Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add multi-join expression support #5081

Merged
merged 1 commit into from
Apr 20, 2020

Conversation

agavra
Copy link
Contributor

@agavra agavra commented Apr 16, 2020

fixes #5062

Description

This change uses the JoinTree to track the equivalence of the join criteria across the different join stages, then uses that information to decide whether or not a repartition is necessary. This opens the door to using different expressions (and non-matching expressions) on different parts of the join.

Testing done

  • unit testing
  • QTT tests

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@vpapavas
Copy link
Member

LGTM, thank you @agavra! We discussed offline but just writing it here so that it doesn't get lost: In absence of join ordering and multi-key partitioning, the approach has the downside of creating inefficient partitioning strategies where at different levels we partition on the same keys and create duplicate work. But given the current limitation, the approach is good enough and gets the job done.

@agavra agavra merged commit 002cd5a into confluentinc:master Apr 20, 2020
@agavra agavra deleted the expressions branch April 20, 2020 22:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

multi-joins on non-constant join conditions
2 participants