Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove ExpressionEquivalence #21276

Merged
merged 1 commit into from
Mar 27, 2024
Merged

Remove ExpressionEquivalence #21276

merged 1 commit into from
Mar 27, 2024

Conversation

martint
Copy link
Member

@martint martint commented Mar 27, 2024

This abstraction doesn't appear to be needed anymore.

Fixes #21235

Release notes

(x) Release notes are required, with the following suggested text:

# Section
* TBD. ({issue}`21235`)

@findepi
Copy link
Member

findepi commented Mar 27, 2024

Remove TestExpressionEquivalence

Did you mean "TestEE" or "EE"?

@martint
Copy link
Member Author

martint commented Mar 27, 2024

Good catch. Yes, ExpressionEquivalence and associated tests.

@martint martint changed the title Remove TestExpressionEquivalence Remove ExpressionEquivalence Mar 27, 2024
@@ -764,7 +762,7 @@ public PlanNode visitSpatialJoin(SpatialJoinNode node, RewriteContext<Expression
PlanNode output = node;
if (leftSource != node.getLeft() ||
rightSource != node.getRight() ||
!areExpressionsEquivalent(newJoinPredicate, joinPredicate)) {
!newJoinPredicate.equals(joinPredicate)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the assumption is that newJoinPredicate cannot be similar to (equivalent), but different from, joinPredicate?

what does guarantee this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a stronger requirement to detect whether the predicate was rewritten by the optimization. Before, it just checked whether they were equivalent based on some semantic-preserving transformations. Now it's based on whether they are exactly the same.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question is, was ExpressionEquivalence more forgiving than equals? Maybe we keep ExpressionEquivalence abstraction but with equals inside?

Copy link
Member Author

@martint martint Mar 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it was more lenient than equality. However, I don't think that matters in practice.

The only canonicalizations this class supports are:

  • AND(AND(x, y), z) => AND(x, y, z). Same for OR.
  • AND(x, x, x, a) => AND(x, a). Same for OR. I believe there's a bug here, though, as it ignores non-deterministic operations that should be evaluated multiple times.
  • a = b => b = a
  • a is distinct from b => b is distinct from a

All of these are already handled by the expression optimizer and canonicalizer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe there's a bug here, though, as it ignores non-deterministic operations that should be evaluated multiple times.

This demonstrates the issue:

ResolvedFunction random = new TestingFunctionResolution().resolveFunction("random", fromTypes());
Expression expression = new IsNull(new Call(random, List.of()));
assertNotEquivalent(
        new Logical(AND, List.of(expression, expression)),
        expression);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these are already handled by the expression optimizer and canonicalizer.

the newJoinPredicate comes thru the optimizer (PredicatePushDown.Rewriter#simplifyExpression), does it also do canonicalization?

@findepi
Copy link
Member

findepi commented Mar 27, 2024

LGTM, but i don't think I know why/when it became obsolete.

@sopel39 do you want to take a look?

@findepi findepi requested a review from sopel39 March 27, 2024 05:19
This abstraction doesn't appear to be needed anymore.
@martint
Copy link
Member Author

martint commented Mar 27, 2024

LGTM, but i don't think I know why/when it became obsolete.

The optimizer performs many of the canonicalizations that this class was doing. I wasn't able to find any queries that produce different plans with or without this code.

@martint martint merged commit b4ad7f1 into trinodb:master Mar 27, 2024
92 of 93 checks passed
@github-actions github-actions bot added this to the 444 milestone Mar 27, 2024
@martint martint deleted the issue-21235 branch March 27, 2024 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Regression: Sql planner fails with contract violation error
5 participants