Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](memory) Fix work load group meaningless GC #27307

Merged
merged 1 commit into from
Nov 21, 2023

Conversation

xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented Nov 20, 2023

Proposed changes

Work load group that do not allow overcommit always trigger GC LOG INFO, but this is meaningless.

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@xinyiZzz
Copy link
Contributor Author

run buildall

@xinyiZzz xinyiZzz requested a review from wangbo November 20, 2023 14:03
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.72 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17098400268 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit 52ed64acde3d81d576ecb589504c5fbc6cfeb25e, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4888	4636	4623	4623
q2	375	149	159	149
q3	2028	1851	1872	1851
q4	1400	1243	1227	1227
q5	3973	3956	4005	3956
q6	247	130	131	130
q7	1416	873	881	873
q8	2776	2801	2797	2797
q9	9880	10520	9645	9645
q10	3490	3582	3549	3549
q11	378	244	244	244
q12	434	294	297	294
q13	4602	3817	3846	3817
q14	321	296	288	288
q15	588	533	519	519
q16	670	578	575	575
q17	1153	969	934	934
q18	7836	7390	7449	7390
q19	1666	1696	1682	1682
q20	568	319	321	319
q21	4349	3966	3962	3962
q22	481	390	388	388
Total cold run time: 53519 ms
Total hot run time: 49212 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4573	4550	4560	4550
q2	337	210	258	210
q3	4024	3984	3995	3984
q4	2688	2685	2689	2685
q5	9802	9725	9758	9725
q6	241	124	120	120
q7	3035	2445	2504	2445
q8	4504	4508	4501	4501
q9	13292	13178	13241	13178
q10	4119	4206	4216	4206
q11	774	669	654	654
q12	988	826	824	824
q13	4260	3564	3557	3557
q14	375	348	345	345
q15	573	522	516	516
q16	735	658	663	658
q17	3844	3920	3828	3828
q18	9699	8912	9047	8912
q19	1799	1778	1812	1778
q20	2359	2067	2067	2067
q21	8807	8679	8235	8235
q22	901	817	827	817
Total cold run time: 81729 ms
Total hot run time: 77795 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.53% (8447/23124)
Line Coverage: 28.87% (68674/237886)
Region Coverage: 27.84% (35509/127559)
Branch Coverage: 24.60% (18111/73618)
Coverage Report: http://coverage.selectdb-in.cc/coverage/52ed64acde3d81d576ecb589504c5fbc6cfeb25e_52ed64acde3d81d576ecb589504c5fbc6cfeb25e/report/index.html

Copy link
Contributor

@wangbo wangbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 21, 2023
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wangbo wangbo merged commit 2548e27 into apache:master Nov 21, 2023
19 of 20 checks passed
superdiaodiao pushed a commit to superdiaodiao/doris that referenced this pull request Nov 21, 2023
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Nov 21, 2023
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 28, 2023
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed workload-group
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants