-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Estimate cost of inequality comparision filters #11518
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/FilterProjectAggregationStatsRule.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
There are some regressions in partitioned |
The only actual regression is in tpch/q04 partitioned (which we already knew about, it's due to bad NDV estimate), the rest is benchmark flakiness. |
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Show resolved
Hide resolved
// left is always lesser than right | ||
if (leftExpressionRange.getHigh() < rightExpressionRange.getLow()) { | ||
PlanNodeStatsEstimate.Builder estimate = PlanNodeStatsEstimate.buildFrom(inputStatistics); | ||
leftExpressionSymbol.ifPresent(symbol -> estimate.addSymbolStatistics(symbol, leftExpressionStatistics)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: can it be empty?
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
What about tpcds/q93, q95, 97, 78? Did plans change for these? |
The inequality estimation code path is executed only for q72 in tpcds and there was no change to plan there. |
d543273
to
48b2735
Compare
You mean partitioned q72 plan? |
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
Updated benchmarks Ignore results from all the TPCDS queries other than q72 as the inequality estimation code path is encountered only in that TPCDS query. |
81aa639
to
1d2e45f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm % comments
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/test/java/io/trino/cost/TestFilterStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm % comments
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
3851aad
to
194059d
Compare
core/trino-main/src/test/java/io/trino/cost/TestComparisonStatsCalculator.java
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/cost/ComparisonStatsCalculator.java
Show resolved
Hide resolved
When there is some overlap between the ranges of the operands of an inequality expression, we approximate it's cost assuming uniform distribution of values within each range and that all pairs of distinct values from both ranges exist.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mind automation (if it fails)
On top of #11469
Description
When there is some overlap between the ranges of the
operands of an inequality expression, we approximate
it's cost assuming uniform distribution of values
within each range and that all pairs of distinct values
from both ranges exist.
Improvement
CBO
Improves query plans in the presence of inequality expressions
Related issues, pull requests, and links
Documentation
(x) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.
Release notes
( ) No release notes entries required.
(x) Release notes entries required with the following suggested text: