Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix](group commit) Fix group commit exception in nereids planner #37953

Merged

Conversation

Yukang-Lian
Copy link
Collaborator

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Yukang-Lian
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40129 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 099b3868a13c95103dfa5161a8f492b1e2631015, data reload: false

------ Round 1 ----------------------------------
q1	17908	4467	4374	4374
q2	2610	197	196	196
q3	12252	1171	1171	1171
q4	10278	894	828	828
q5	7549	2744	2706	2706
q6	224	142	142	142
q7	980	603	603	603
q8	9209	2064	2097	2064
q9	8825	6555	6545	6545
q10	8751	3827	3726	3726
q11	452	234	237	234
q12	401	223	224	223
q13	17780	2939	2959	2939
q14	273	236	243	236
q15	541	494	477	477
q16	479	376	380	376
q17	964	706	662	662
q18	7953	7751	7494	7494
q19	4902	1460	1386	1386
q20	713	322	324	322
q21	4965	3146	3276	3146
q22	343	282	279	279
Total cold run time: 118352 ms
Total hot run time: 40129 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4382	4299	4302	4299
q2	390	266	262	262
q3	3035	2775	2798	2775
q4	1896	1606	1618	1606
q5	5288	5316	5297	5297
q6	225	131	131	131
q7	2183	1741	1735	1735
q8	3188	3333	3300	3300
q9	8436	8386	8361	8361
q10	3874	3731	3675	3675
q11	583	483	488	483
q12	770	664	611	611
q13	17471	2967	2965	2965
q14	307	270	280	270
q15	512	479	476	476
q16	467	427	431	427
q17	1782	1477	1453	1453
q18	7665	7568	7272	7272
q19	1645	1485	1558	1485
q20	1989	1788	1753	1753
q21	4835	4719	4637	4637
q22	576	492	493	492
Total cold run time: 71499 ms
Total hot run time: 53765 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172633 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 099b3868a13c95103dfa5161a8f492b1e2631015, data reload: false

query1	899	366	362	362
query2	6458	1863	1853	1853
query3	6686	210	223	210
query4	28517	17341	17486	17341
query5	4218	472	460	460
query6	262	172	165	165
query7	4600	292	294	292
query8	239	190	197	190
query9	8516	2406	2373	2373
query10	444	277	290	277
query11	11648	10236	10075	10075
query12	135	89	81	81
query13	1640	372	362	362
query14	10300	6856	7785	6856
query15	227	169	165	165
query16	7910	331	335	331
query17	1797	554	538	538
query18	2012	288	277	277
query19	199	160	156	156
query20	91	85	82	82
query21	216	133	127	127
query22	4344	4253	4140	4140
query23	33745	33117	33385	33117
query24	12045	2878	2874	2874
query25	697	384	397	384
query26	1825	154	147	147
query27	2978	276	280	276
query28	7504	1992	2008	1992
query29	1175	640	632	632
query30	282	156	150	150
query31	1019	772	746	746
query32	96	55	55	55
query33	790	332	388	332
query34	895	492	484	484
query35	682	597	573	573
query36	1101	921	936	921
query37	294	81	83	81
query38	2892	2755	2774	2755
query39	903	809	832	809
query40	279	121	124	121
query41	49	44	45	44
query42	117	100	98	98
query43	509	492	464	464
query44	1201	723	718	718
query45	194	164	162	162
query46	1095	739	725	725
query47	1854	1796	1760	1760
query48	363	293	308	293
query49	1198	408	404	404
query50	781	393	395	393
query51	6969	6784	6784	6784
query52	106	91	96	91
query53	364	300	302	300
query54	979	454	448	448
query55	76	73	76	73
query56	292	271	297	271
query57	1155	1036	1082	1036
query58	261	251	254	251
query59	2942	2561	2578	2561
query60	304	281	290	281
query61	104	95	94	94
query62	840	655	656	655
query63	332	290	285	285
query64	10438	2256	1672	1672
query65	3166	3095	3077	3077
query66	1379	352	332	332
query67	15345	15250	15200	15200
query68	4692	548	558	548
query69	647	339	328	328
query70	1136	1076	1184	1076
query71	423	299	284	284
query72	8780	5428	5603	5428
query73	746	330	327	327
query74	6312	5629	5663	5629
query75	4174	2697	2699	2697
query76	3274	978	994	978
query77	666	321	308	308
query78	10997	9138	9211	9138
query79	6816	542	527	527
query80	1230	489	491	489
query81	581	229	220	220
query82	1495	142	143	142
query83	343	169	168	168
query84	282	85	91	85
query85	1930	304	289	289
query86	405	323	322	322
query87	3281	3157	3109	3109
query88	4849	2394	2378	2378
query89	519	409	400	400
query90	1885	213	196	196
query91	133	100	103	100
query92	65	52	53	52
query93	5450	516	507	507
query94	1078	247	216	216
query95	407	334	316	316
query96	624	279	273	273
query97	3203	3046	3034	3034
query98	207	215	189	189
query99	1572	1295	1246	1246
Total cold run time: 301175 ms
Total hot run time: 172633 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.08 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 099b3868a13c95103dfa5161a8f492b1e2631015, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.03	0.03
query3	0.22	0.05	0.05
query4	1.67	0.07	0.07
query5	0.53	0.47	0.50
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.06
query9	0.55	0.48	0.49
query10	0.55	0.55	0.54
query11	0.14	0.11	0.11
query12	0.14	0.12	0.13
query13	0.59	0.58	0.58
query14	0.76	0.77	0.78
query15	0.86	0.79	0.81
query16	0.38	0.37	0.37
query17	1.00	0.98	0.98
query18	0.23	0.22	0.22
query19	1.80	1.74	1.79
query20	0.01	0.01	0.01
query21	15.40	0.75	0.67
query22	4.62	6.81	1.48
query23	18.27	1.38	1.24
query24	2.07	0.23	0.23
query25	0.15	0.09	0.08
query26	0.29	0.20	0.20
query27	0.46	0.23	0.23
query28	13.22	1.01	1.00
query29	12.64	3.37	3.32
query30	0.25	0.07	0.06
query31	2.85	0.38	0.40
query32	3.28	0.49	0.47
query33	2.89	2.86	2.89
query34	16.92	4.28	4.35
query35	4.45	4.39	4.43
query36	0.74	0.46	0.48
query37	0.19	0.16	0.16
query38	0.15	0.15	0.15
query39	0.05	0.03	0.04
query40	0.16	0.12	0.13
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110 s
Total hot run time: 30.08 s

dataroaring
dataroaring previously approved these changes Jul 17, 2024
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 17, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

@Yukang-Lian
Copy link
Collaborator Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 17, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 39937 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b1799c26893126ef3753c15036d9bcbf99b55465, data reload: false

------ Round 1 ----------------------------------
q1	18097	4471	4436	4436
q2	2471	199	185	185
q3	11106	1187	1076	1076
q4	10516	759	792	759
q5	7692	2724	2699	2699
q6	224	138	142	138
q7	975	612	620	612
q8	9529	2113	2081	2081
q9	8762	6612	6545	6545
q10	8773	3740	3800	3740
q11	463	240	236	236
q12	394	221	222	221
q13	17767	2997	2967	2967
q14	275	242	241	241
q15	540	478	495	478
q16	490	405	385	385
q17	974	604	665	604
q18	8070	7423	7386	7386
q19	7571	1443	1382	1382
q20	700	332	337	332
q21	5066	3149	3256	3149
q22	349	297	285	285
Total cold run time: 120804 ms
Total hot run time: 39937 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4388	4300	4271	4271
q2	384	259	274	259
q3	3058	2742	2732	2732
q4	1900	1603	1631	1603
q5	5314	5312	5334	5312
q6	226	132	134	132
q7	2153	1766	1717	1717
q8	3211	3337	3298	3298
q9	8406	8407	8436	8407
q10	3942	3721	3645	3645
q11	589	491	488	488
q12	794	598	590	590
q13	16415	3046	2996	2996
q14	308	278	270	270
q15	519	475	482	475
q16	474	414	430	414
q17	1787	1446	1465	1446
q18	7779	7510	7322	7322
q19	1652	1495	1660	1495
q20	2016	1832	1775	1775
q21	4782	4789	4630	4630
q22	603	514	522	514
Total cold run time: 70700 ms
Total hot run time: 53791 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 171637 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b1799c26893126ef3753c15036d9bcbf99b55465, data reload: false

query1	909	377	363	363
query2	6490	1929	1802	1802
query3	6673	211	218	211
query4	24715	17256	17630	17256
query5	4180	489	504	489
query6	275	188	173	173
query7	4592	302	306	302
query8	266	207	192	192
query9	8618	2450	2419	2419
query10	432	287	269	269
query11	12803	10059	9969	9969
query12	134	84	82	82
query13	1632	368	359	359
query14	9854	7600	6924	6924
query15	220	165	177	165
query16	7757	315	308	308
query17	1804	545	520	520
query18	1908	278	291	278
query19	214	153	156	153
query20	93	86	121	86
query21	211	123	131	123
query22	4494	3967	4041	3967
query23	33889	33048	33020	33020
query24	12122	2924	2809	2809
query25	664	371	380	371
query26	1869	152	153	152
query27	3006	274	279	274
query28	7743	2028	2011	2011
query29	1174	625	621	621
query30	286	148	144	144
query31	954	732	745	732
query32	91	55	54	54
query33	769	303	296	296
query34	983	487	503	487
query35	685	577	584	577
query36	1106	953	945	945
query37	278	79	79	79
query38	2867	2761	2732	2732
query39	847	804	818	804
query40	275	127	121	121
query41	47	45	46	45
query42	118	100	107	100
query43	529	480	479	479
query44	1215	746	735	735
query45	193	161	159	159
query46	1105	758	754	754
query47	1923	1773	1790	1773
query48	379	299	292	292
query49	1196	419	416	416
query50	782	401	398	398
query51	6872	6855	6712	6712
query52	106	91	94	91
query53	365	289	280	280
query54	920	451	453	451
query55	76	75	76	75
query56	310	270	271	270
query57	1194	1052	1067	1052
query58	260	263	270	263
query59	2963	2658	2532	2532
query60	308	282	307	282
query61	101	113	120	113
query62	833	662	667	662
query63	367	288	300	288
query64	10488	2246	1663	1663
query65	3214	3127	3139	3127
query66	1370	362	357	357
query67	15296	15200	14888	14888
query68	4516	553	554	553
query69	502	326	332	326
query70	1153	1140	1137	1137
query71	404	289	286	286
query72	7680	5309	5976	5309
query73	753	332	324	324
query74	6188	5648	5635	5635
query75	3451	2745	2716	2716
query76	2703	951	914	914
query77	481	319	318	318
query78	9748	9259	9000	9000
query79	2172	524	520	520
query80	2266	485	471	471
query81	600	226	222	222
query82	792	136	136	136
query83	335	171	170	170
query84	257	88	89	88
query85	2172	342	304	304
query86	500	342	325	325
query87	3329	3109	3089	3089
query88	4129	2371	2380	2371
query89	498	393	396	393
query90	1849	194	190	190
query91	134	101	102	101
query92	67	51	51	51
query93	2409	520	511	511
query94	1283	213	210	210
query95	416	313	322	313
query96	609	275	271	271
query97	3180	3020	3085	3020
query98	211	203	191	191
query99	1528	1296	1276	1276
Total cold run time: 286727 ms
Total hot run time: 171637 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.74 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b1799c26893126ef3753c15036d9bcbf99b55465, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.08
query5	0.50	0.48	0.50
query6	1.15	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.05	0.04
query9	0.55	0.49	0.48
query10	0.55	0.55	0.55
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.60	0.58	0.59
query14	0.76	0.78	0.78
query15	0.85	0.81	0.82
query16	0.34	0.38	0.37
query17	0.96	0.97	0.96
query18	0.22	0.22	0.21
query19	1.74	1.68	1.74
query20	0.01	0.01	0.02
query21	15.40	0.78	0.65
query22	3.93	7.49	2.07
query23	18.38	1.35	1.25
query24	2.06	0.22	0.25
query25	0.14	0.09	0.09
query26	0.29	0.20	0.21
query27	0.46	0.23	0.22
query28	13.34	1.03	1.00
query29	12.58	3.27	3.30
query30	0.26	0.06	0.06
query31	2.87	0.40	0.39
query32	3.27	0.48	0.46
query33	2.87	2.97	2.95
query34	16.91	4.36	4.38
query35	4.45	4.41	4.44
query36	0.66	0.46	0.50
query37	0.19	0.16	0.15
query38	0.15	0.15	0.15
query39	0.05	0.04	0.04
query40	0.15	0.12	0.12
query41	0.09	0.04	0.05
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.23 s
Total hot run time: 30.74 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 17, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@dataroaring dataroaring merged commit 3d3ceae into apache:master Jul 18, 2024
28 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants