Prototype Reduce
pushdown through Union
.
#30633
Draft
+281
−132
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When we have
Reduce
overUnion
, we may be able to push down theReduce
if its key expressions can be seen to be disjoint on the union inputs. More generally, if there are multiple connected components of union inputs under the "not incompatible" relation, we can push the reduce down to each connected component and union the results.The arranged state is not increased, as we are certain it is partitioned among the subexpressions. The pushdown can result in wins when the union inputs have more specific properties that further optimize the reduce (e.g. are constants, have unique keys, certain columns are literals).
There is a risk to the pushdown, which is that if we were relying on the arranged outputs of the
Reduce
for .. say a join input or the arrangement for aCREATE INDEX
command, we would need to re-arrange the results, and we would not feel quite as smart. So, we should just race and merge this. At the same time, the moments that feel bad here are similar to the moments that we know how to work around with "product of sum" join processing (which is tbd, but this is another reason to want it).Motivation
Tips for reviewer
Checklist
$T ⇔ Proto$T
mapping (possibly in a backwards-incompatible way), then it is tagged with aT-proto
label.