Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query Optimizer: Add OUTER --> INNER join conversion #1670

Closed
alamb opened this issue Jan 24, 2022 · 1 comment
Closed

Query Optimizer: Add OUTER --> INNER join conversion #1670

alamb opened this issue Jan 24, 2022 · 1 comment
Labels
datafusion Changes in the datafusion crate enhancement New feature or request

Comments

@alamb
Copy link
Contributor

alamb commented Jan 24, 2022

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

LEFT OUTER, RIGHT OUTER, and FULL OUTER JOINs are often more expensive to evaluate and preclude other optimizations (such as pushing down predicates as can be seen in #1618)

As such, sophisticated optimizers will actually rewrite OUTER joins to INNER joins depending on the predicates of the query to improve performance

Describe the solution you'd like

Add an OptimzierPass pass that will attempt to convert OUTER joins to inner joins.

This will require some non trivial research to figure out under what conditions the joins can be rewritten / converted

Additional context
Relevant discussion: #1618 (comment)

You can see a version of this code in Spark here: https://github.com/apache/spark/blob/aaf0e5e71509a2324e110e45366b753c7926c64b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala#L119-L135

@alamb alamb added enhancement New feature or request datafusion Changes in the datafusion crate labels Jan 24, 2022
@alamb
Copy link
Contributor Author

alamb commented Jan 24, 2022

Whoops -- this is a dupe of #1585

@alamb alamb closed this as completed Jan 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant