Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Rust] [DataFusion] Predicate Pushdown Improvement: treat predicates separated by AND separately #25820

Closed
asfimport opened this issue Aug 17, 2020 · 1 comment

Comments

@asfimport
Copy link
Collaborator

As discussed by @jorgecarleitao and @houqp here: #7880 (review)

If a predicate is a conjunction (aka AND'd) together, each of the clauses can be treated separately (e.g. a single filter expression A > 5 And B < 4 can be broken up and each of A > 5 and B < 4 can be potentially pushed down different levels

The filter pushdown logic works for the following case (when a and b are are separate selections, predicate for a is pushed below the Aggregate in the optimized plan):

********Original plan:
Selection: #b GtEq Int64(1)
  Selection: #a LtEq Int64(1)
    Aggregate: groupBy=[[#a]], aggr=[[MIN(#b)]]
      TableScan: test projection=None

********Optimized plan:
Selection: #b GtEq Int64(1)
  Aggregate: groupBy=[[#a]], aggr=[[MIN(#b)]]
    Selection: #a LtEq Int64(1)
      TableScan: test projection=None

But not for this when a and b are AND'd together

********Original plan:
Selection: #a LtEq Int64(1) And #b GtEq Int64(1)
  Aggregate: groupBy=[[#a]], aggr=[[MIN(#b)]]
    TableScan: test projection=None
********Optimized plan:
Selection: #a LtEq Int64(1) And #b GtEq Int64(1)
  Aggregate: groupBy=[[#a]], aggr=[[MIN(#b)]]
    TableScan: test projection=None

Reporter: Andrew Lamb / @alamb
Assignee: Jorge Leitão / @jorgecarleitao

PRs and other links:

Note: This issue was originally created as ARROW-9771. Please see the migration documentation for further details.

@asfimport
Copy link
Collaborator Author

Andrew Lamb / @alamb:
Issue resolved by pull request 8903
#8903

@asfimport asfimport added this to the 3.0.0 milestone Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants