Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate expressions after type coercion #3431

Closed
Tracked by #2355
Dandandan opened this issue Sep 10, 2022 · 0 comments · Fixed by #3444
Closed
Tracked by #2355

Evaluate expressions after type coercion #3431

Dandandan opened this issue Sep 10, 2022 · 0 comments · Fixed by #3444
Labels
enhancement New feature or request performance Make DataFusion faster

Comments

@Dandandan
Copy link
Contributor

Dandandan commented Sep 10, 2022

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The recently added an optimizer pass that performs type coercion on the logical plan.
There is some oppurtunity to simplify the expressions as often it creates casts on constants, like:

CAST(Int64(1) AS Float32)

This could be evaluated to:
Float32(1.0)

Simplifying the plan and leaving more room for further optimization.

Describe the solution you'd like
When wrapping the expression in a cast, we can directly evaluate the result. Alternatively we could run the "simplify_expressions" pass again after type coercion, however this seems unneeded.
It seems we can use the evaluate_to_scalar function / ConstEvaluator to evaluate the created Expr in the type coercion, after applying it to an expression.
https://github.com/apache/arrow-datafusion/blob/master/datafusion/optimizer/src/simplify_expressions.rs#L487

Describe alternatives you've considered
n/a

Additional context

@Dandandan Dandandan added enhancement New feature or request performance Make DataFusion faster labels Sep 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request performance Make DataFusion faster
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant