-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Filter::try_new
with validation
#3796
Conversation
@@ -580,12 +580,12 @@ mod tests { | |||
let table_scan = test_table_scan()?; | |||
|
|||
let plan = LogicalPlanBuilder::from(table_scan) | |||
.filter(col("c"))? | |||
.filter(col("c").gt(lit(1)))? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test was invalid and the new validation checks caught this.
@@ -1930,7 +1930,7 @@ mod tests { | |||
assert_optimized_plan_eq( | |||
&plan, | |||
"\ | |||
Filter: test.b > Int32(1) AS test.b > Int32(1) AND test.b > Int32(1)\ | |||
Filter: test.b > Int32(1)\ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The updated tests here demonstrate that we no longer have aliases for filter predicates.
Filter::try_new
with validationFilter::try_new
with validation
} | ||
} | ||
|
||
// filter predicates should not contain aliased expressions so we remove any aliases here |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the rationale here that the the output of a FilterExpr
has the same schema as its input -- since the Filter never changes its input if we find an alias on the FilterExpr then something is likely wrong as it won't change anything?
I would personally prefer to throw an error rather than stripping the aliases and then track down what is adding aliases
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FilterExec::try_new
now throws an exception when passed an aliased predicate and the rewrite logic has moved to utils::from_plan
which is called from multiple optimizer rules
The CI test is unrelated: #3798 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's ship it!
Benchmark runs are scheduled for baseline = e27e86b and contender = 3af09fb. 3af09fb is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Which issue does this PR close?
Closes #3795
Rationale for this change
Filters should not use aliased expressions
What changes are included in this PR?
Filter::try_new
and makeFilter
attributes private to force the use of the constructor in other cratesAre there any user-facing changes?
Yes. Filter attributes are now
pub(crate)
but accessor methods have been added.