Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore](Nereids): remove SemiJoinCommute in rewriter #28563

Merged
merged 1 commit into from
Dec 18, 2023

Conversation

jackwener
Copy link
Member

Proposed changes

analyzer already contained SemiJoinCommute

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@jackwener
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit eab8da3b5df063021e7a9efb60cfeaaba0a71b85, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4734	4437	4481	4437
q2	367	147	160	147
q3	1470	1245	1228	1228
q4	1131	944	917	917
q5	3157	3127	3162	3127
q6	253	130	130	130
q7	994	494	496	494
q8	2212	2228	2191	2191
q9	6707	6677	6691	6677
q10	3220	3290	3265	3265
q11	322	201	194	194
q12	359	218	211	211
q13	4525	3807	3822	3807
q14	249	211	210	210
q15	566	525	530	525
q16	441	398	393	393
q17	1003	571	502	502
q18	7193	7036	6856	6856
q19	1540	1445	1425	1425
q20	545	313	298	298
q21	3089	2657	2677	2657
q22	346	279	282	279
Total cold run time: 44423 ms
Total hot run time: 39970 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4407	4414	4396	4396
q2	270	165	173	165
q3	3529	3523	3513	3513
q4	2384	2369	2367	2367
q5	5735	5705	5722	5705
q6	243	122	123	122
q7	2371	1876	1845	1845
q8	3513	3525	3555	3525
q9	9035	9001	8992	8992
q10	3910	3983	3992	3983
q11	504	371	390	371
q12	758	598	604	598
q13	4283	3542	3563	3542
q14	284	243	262	243
q15	570	528	525	525
q16	499	440	458	440
q17	1894	1864	1862	1862
q18	8684	8309	8304	8304
q19	1738	1777	1755	1755
q20	2244	1941	1923	1923
q21	6559	6212	6204	6204
q22	501	418	438	418
Total cold run time: 63915 ms
Total hot run time: 60798 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.94 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.8 seconds inserted 10000000 Rows, about 359K ops/s
storage size: 17219660750 Bytes

@jackwener
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 4182655d9d3fdd9f0e24ce1c237d8d58d522fb54, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4698	4435	4452	4435
q2	362	152	160	152
q3	1463	1290	1269	1269
q4	1106	929	901	901
q5	3170	3214	3153	3153
q6	247	132	129	129
q7	981	488	482	482
q8	2231	2211	2209	2209
q9	6682	6642	6629	6629
q10	3209	3253	3274	3253
q11	317	196	195	195
q12	348	211	205	205
q13	4583	3810	3819	3810
q14	243	212	214	212
q15	567	529	515	515
q16	440	385	382	382
q17	1028	684	586	586
q18	7225	6954	7107	6954
q19	1543	1449	1413	1413
q20	489	313	320	313
q21	3076	2658	2678	2658
q22	354	279	286	279
Total cold run time: 44362 ms
Total hot run time: 40134 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4392	4392	4400	4392
q2	267	164	167	164
q3	3533	3543	3498	3498
q4	2380	2364	2366	2364
q5	5707	5744	5726	5726
q6	239	122	122	122
q7	2403	1900	1868	1868
q8	3530	3530	3519	3519
q9	9032	9044	9084	9044
q10	3902	3988	3997	3988
q11	500	382	386	382
q12	759	590	598	590
q13	4298	3582	3566	3566
q14	294	256	247	247
q15	570	519	528	519
q16	492	450	455	450
q17	1884	1854	1895	1854
q18	8737	8192	8326	8192
q19	1747	1746	1767	1746
q20	2259	1954	1916	1916
q21	6550	6186	6164	6164
q22	499	435	428	428
Total cold run time: 63974 ms
Total hot run time: 60739 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.72 seconds
stream load tsv: 576 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 28 seconds loaded 2358488459 Bytes, about 80 MB/s
stream load orc: 70 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17217921939 Bytes

@jackwener
Copy link
Member Author

run p0

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 18, 2023
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@jackwener jackwener merged commit ddcfba0 into apache:master Dec 18, 2023
17 of 18 checks passed
@jackwener jackwener deleted the chore branch December 18, 2023 12:47
hello-stephen pushed a commit to hello-stephen/doris that referenced this pull request Dec 28, 2023
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants