Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support sort merge join with a join condition #398

Open
Tracked by #858
, #843
...
viirya opened this issue May 7, 2024 · 6 comments · Fixed by #553 or #1211
Open
Tracked by #858
, #843
...

Support sort merge join with a join condition #398

viirya opened this issue May 7, 2024 · 6 comments · Fixed by #553 or #1211
Assignees
@comphead
Labels
enhancement New feature or request
Milestone
0.6.0

Comments

@viirya
Copy link
Member

viirya commented May 7, 2024

What is the problem the feature request solves?

Currently SMJ with join condition is not supported by Comet and falls back to Spark. The feature was added into DataFusion but we've not incorporated it yet. We should do it to add the support.

Describe the potential solution

No response

Additional context

No response
@viirya viirya added the enhancement New feature or request label May 7, 2024
@viirya viirya self-assigned this May 7, 2024
@viirya viirya mentioned this issue May 7, 2024
Open
15 tasks
@comphead
Copy link
Contributor

@viirya can I take it as I spent some time with SMJ?

@viirya viirya mentioned this issue Jun 11, 2024
Merged
@andygrove andygrove added this to the 0.2.0 milestone Jun 14, 2024
@andygrove andygrove mentioned this issue Jun 14, 2024
Closed
This was referenced Aug 19, 2024
Closed
Open
@andygrove
Copy link
Member

Re-opening this since we discovered that there are some correctness and performance issues and currently have this feature disabled by default

@andygrove andygrove reopened this Sep 20, 2024
@andygrove andygrove modified the milestones: 0.2.0, 0.4.0 Nov 5, 2024
@andygrove
Copy link
Member

andygrove commented Nov 5, 2024 •
edited
Loading

There is an existing test SortMergeJoin with join filter that we need to enable as part of closing this issue. It was ignored in #1057

@comphead
Copy link
Contributor

comphead commented Nov 5, 2024

Thanks @andygrove I'll take if from now

@comphead comphead assigned comphead and unassigned viirya Nov 5, 2024
@comphead comphead mentioned this issue Nov 5, 2024
Closed
@andygrove andygrove modified the milestones: 0.4.0, 0.5.0 Nov 11, 2024
@comphead
Copy link
Contributor

Waiting for DataFusion 44.0.0

@comphead comphead mentioned this issue Jan 2, 2025
Merged
@andygrove
Copy link
Member

Re-opening this issue because we only enable this feature in tests currently due to poor performance in benchmarks

@andygrove andygrove reopened this Jan 10, 2025
@andygrove andygrove modified the milestones: 0.5.0, 0.6.0 Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@comphead comphead

Labels
enhancement New feature or request
Projects
None yet
Milestone
0.6.0
3 participants
@viirya @andygrove @comphead