Skip to content

Commit

Permalink
[fix](plan) only scan node with limit and no predicate can reduce to …
Browse files Browse the repository at this point in the history
…1 instance (#31342)

This PR #25952 introduce a opt that if a scan node has limit and predicates, use only 1 instance to save cup and memory.
But this is wrong because we can not guarantee that the predicates can truly help to prune the data.
So I modify the logic to remove this opt.
Now, only scan node with limit and NO predicate can reduce to only 1 instance.
  • Loading branch information
morningman authored and Doris-Extras committed Feb 23, 2024
1 parent 49842ee commit 481517a
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -1378,6 +1378,8 @@ public boolean getShouldColoScan() {
// If scan is key search, should not enable the shared scan opt to prevent the performance problem
// 1. where contain the eq or in expr of key column slot
// 2. key column slot is distribution column and first column
// FIXME: this is not a good check, we can not guarantee that the predicate we check can truly
// help to prune the data, so we should check the predicate's effect on the data.
protected boolean isKeySearch() {
List<SlotRef> whereSlot = Lists.newArrayList();
for (Expr conjunct : conjuncts) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -735,7 +735,7 @@ public int getScanRangeNum() {
return Integer.MAX_VALUE;
}

public boolean haveLimitAndConjunts() {
return hasLimit() && !conjuncts.isEmpty();
public boolean shouldUseOneInstance() {
return hasLimit() && conjuncts.isEmpty();
}
}
8 changes: 4 additions & 4 deletions fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
Original file line number Diff line number Diff line change
Expand Up @@ -2067,9 +2067,9 @@ private void computeFragmentHosts() throws Exception {
if (node.isPresent() && (!node.get().shouldDisableSharedScan(context)
|| ignoreStorageDataDistribution)) {
expectedInstanceNum = Math.max(expectedInstanceNum, 1);
// if have limit and conjunts, only need 1 instance to save cpu and
// if have limit and no conjuncts, only need 1 instance to save cpu and
// mem resource
if (node.get().haveLimitAndConjunts()) {
if (node.get().shouldUseOneInstance()) {
expectedInstanceNum = 1;
}

Expand All @@ -2080,9 +2080,9 @@ private void computeFragmentHosts() throws Exception {
//the scan instance num should not larger than the tablets num
expectedInstanceNum = Math.min(perNodeScanRanges.size(), parallelExecInstanceNum);
}
// if have limit and conjunts, only need 1 instance to save cpu and
// if have limit and no conjuncts, only need 1 instance to save cpu and
// mem resource
if (node.get().haveLimitAndConjunts()) {
if (node.get().shouldUseOneInstance()) {
expectedInstanceNum = 1;
}

Expand Down

0 comments on commit 481517a

Please sign in to comment.