Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](nereids) fix union all instance number #39999

Merged
merged 2 commits into from
Aug 29, 2024

Conversation

xzj7019
Copy link
Contributor

@xzj7019 xzj7019 commented Aug 27, 2024

In following plan pattern, fragment 11 has 3 child fragments aligning with 3 exchange nodes and the union plan node only have 2 plan children, i.e, the aggr node and the exchange node(4). In current union node's parallelism computing, it used the plan children's size to do iteration to find the max parallel but ignore the last fragment's. Actually, it should use fragment's children(exclude unpartitioned ones) instead of plan node's children to do the iteration.

11:VUNION(703)
|
|----10:VAGGREGATE (update finalize)(697)
| | group by: brand_name[#23], skc[#24]
| | sortByGroupKey:false
| | cardinality=1
| | final projections: '1', brand_name[#25], skc[#26]
| | final project output tuple id: 11
| | distribute expr lists: skc[#24]
| |
| 9:VHASH JOIN(691)
| | join op: INNER JOIN(PARTITIONED)[]
| | equal join conjunct: (skc[#19] = skc_code[#12])
| | cardinality=1
| | vec output tuple id: 9
| | output tuple id: 9
| | vIntermediate tuple ids: 8
| | hash output slot ids: 18 19
| | final projections: brand_name[#20], skc[#21]
| | final project output tuple id: 9
| | distribute expr lists: skc[#19]
| | distribute expr lists: skc_code[#12]
| |
| |----6:VEXCHANGE
| | offset: 0
| | distribute expr lists: skc_code[#12]
| |
| 8:VEXCHANGE
| offset: 0
| distribute expr lists:
|
4:VEXCHANGE
offset: 0
distribute expr lists:

@xzj7019
Copy link
Contributor Author

xzj7019 commented Aug 27, 2024

run buildall

@wm1581066 wm1581066 requested a review from morrySnow August 27, 2024 13:09
@wm1581066 wm1581066 added the usercase Important user case type label label Aug 27, 2024
@xzj7019
Copy link
Contributor Author

xzj7019 commented Aug 28, 2024

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 29, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit 9dbd4d9 into apache:master Aug 29, 2024
31 of 35 checks passed
yiguolei pushed a commit that referenced this pull request Aug 29, 2024
morrySnow pushed a commit that referenced this pull request Sep 2, 2024
dataroaring pushed a commit that referenced this pull request Sep 3, 2024
In following plan pattern, fragment 11 has 3 child fragments aligning
with 3 exchange nodes and the union plan node only have 2 plan children,
i.e, the aggr node and the exchange node(4). In current union node's
parallelism computing, it used the plan children's size to do iteration
to find the max parallel but ignore the last fragment's. Actually, it
should use fragment's children(exclude unpartitioned ones) instead of
plan node's children to do the iteration.

11:VUNION(703)
  |
  |----10:VAGGREGATE (update finalize)(697)
  |    |  group by: brand_name[#23], skc[#24]
  |    |  sortByGroupKey:false
  |    |  cardinality=1
  |    |  final projections: '1', brand_name[#25], skc[#26]
  |    |  final project output tuple id: 11
  |    |  distribute expr lists: skc[#24]
  |    |
  |    9:VHASH JOIN(691)
  |    |  join op: INNER JOIN(PARTITIONED)[]
  |    |  equal join conjunct: (skc[#19] = skc_code[#12])
  |    |  cardinality=1
  |    |  vec output tuple id: 9
  |    |  output tuple id: 9
  |    |  vIntermediate tuple ids: 8
  |    |  hash output slot ids: 18 19
  |    |  final projections: brand_name[#20], skc[#21]
  |    |  final project output tuple id: 9
  |    |  distribute expr lists: skc[#19]
  |    |  distribute expr lists: skc_code[#12]
  |    |
  |    |----6:VEXCHANGE
  |    |       offset: 0
  |    |       distribute expr lists: skc_code[#12]
  |    |
  |    8:VEXCHANGE
  |       offset: 0
  |       distribute expr lists:
  |
  4:VEXCHANGE
     offset: 0
     distribute expr lists:
@yiguolei yiguolei mentioned this pull request Sep 5, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.15-merged dev/2.1.6-merged dev/3.0.2-merged reviewed usercase Important user case type label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants