Skip to content

Add Vaex comparison #180

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
maccam912 opened this issue Jan 13, 2021 · 5 comments · May be fixed by #243
Open

Add Vaex comparison #180

maccam912 opened this issue Jan 13, 2021 · 5 comments · May be fixed by #243
Labels
new solution

Comments

@maccam912
Copy link

Just found this, thanks for this repo! I see others have offered new projects that could be included in the comparison. Might I suggest Vaex as a future contender?

https://vaex.readthedocs.io/en/latest/

It's lazy, so I'm not sure what magic needs to be done to get actual times for group-by and join. Presumably whatever Spark is doing now that I think about it.

I will now attempt to figure out how to add a tag to this issue.
@jangorecki
Copy link
Contributor

Thank you for filling out this request.
Spark does provide a method for materializing computation. As long as Vaex provides similar one there shouldn't be any issues because of laziness.
Note that author of Vaex @maartenbreddels was already reproducing this benchmark, and was planning to make PR, as mentioned in #150.

@ptomecek
Copy link

@jangorecki @maartenbreddels Looks like #150 is closed, was there ever a PR to include Vax in the benchmarks? I don't see it in the latest generated report.

@jangorecki
Copy link
Contributor

no

@maartenbreddels
Copy link

This is planned 🙂

@maartenbreddels maartenbreddels linked a pull request Feb 9, 2022 that will close this issue
Open
@FullyWashable
Copy link

Hi, I'm looking at this and at the linked pull request, and the reason why the vaex comparison still isn't included is a bit over my head. Is that comparison available somewhere else? Or a similar one that does include vaex?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
new solution
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add Vaex vaexio/db-benchmark
5 participants
@maccam912 @maartenbreddels @jangorecki @FullyWashable @ptomecek