Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prototype Reduce pushdown through Union. #30633

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

frankmcsherry
Copy link
Contributor

When we have Reduce over Union, we may be able to push down the Reduce if its key expressions can be seen to be disjoint on the union inputs. More generally, if there are multiple connected components of union inputs under the "not incompatible" relation, we can push the reduce down to each connected component and union the results.

The arranged state is not increased, as we are certain it is partitioned among the subexpressions. The pushdown can result in wins when the union inputs have more specific properties that further optimize the reduce (e.g. are constants, have unique keys, certain columns are literals).

There is a risk to the pushdown, which is that if we were relying on the arranged outputs of the Reduce for .. say a join input or the arrangement for a CREATE INDEX command, we would need to re-arrange the results, and we would not feel quite as smart. So, we should just race and merge this. At the same time, the moments that feel bad here are similar to the moments that we know how to work around with "product of sum" join processing (which is tbd, but this is another reason to want it).

Motivation

Tips for reviewer

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant