Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[refactor](pipeline) Refactor logics #40576

Merged
merged 2 commits into from
Sep 10, 2024
Merged

[refactor](pipeline) Refactor logics #40576

merged 2 commits into from
Sep 10, 2024

Conversation

Gabriel39
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Gabriel39
Copy link
Contributor Author

run buildall

@Gabriel39
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38154 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 25a9c5878140837044bcb9ee0e26707c30190fec, data reload: false

------ Round 1 ----------------------------------
q1	17634	4508	4332	4332
q2	2035	183	181	181
q3	11684	977	1147	977
q4	10515	694	801	694
q5	7776	2903	2834	2834
q6	230	141	141	141
q7	984	620	609	609
q8	9326	2115	2116	2115
q9	7068	6619	6603	6603
q10	7008	2220	2207	2207
q11	466	243	243	243
q12	398	223	227	223
q13	18233	3109	3119	3109
q14	290	237	226	226
q15	542	496	483	483
q16	501	441	423	423
q17	996	713	745	713
q18	7444	6861	6941	6861
q19	1389	1155	1035	1035
q20	682	334	342	334
q21	3978	3145	2839	2839
q22	1141	972	1012	972
Total cold run time: 110320 ms
Total hot run time: 38154 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4361	4365	4292	4292
q2	391	278	277	277
q3	2905	2705	2732	2705
q4	1935	1634	1699	1634
q5	5727	5726	5848	5726
q6	229	139	139	139
q7	2238	1866	1815	1815
q8	3327	3513	3547	3513
q9	8908	8920	8977	8920
q10	3674	3394	3363	3363
q11	625	507	515	507
q12	795	647	671	647
q13	15282	3333	3274	3274
q14	325	280	293	280
q15	549	532	497	497
q16	543	485	519	485
q17	1867	1567	1569	1567
q18	8206	7860	7900	7860
q19	1803	1667	1636	1636
q20	2200	1931	1969	1931
q21	5718	5527	5509	5509
q22	1129	1009	1055	1009
Total cold run time: 72737 ms
Total hot run time: 57586 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.86% (9396/25490)
Line Coverage: 28.25% (77517/274382)
Region Coverage: 27.64% (40010/144745)
Branch Coverage: 24.27% (20351/83866)
Coverage Report: http://coverage.selectdb-in.cc/coverage/25a9c5878140837044bcb9ee0e26707c30190fec_25a9c5878140837044bcb9ee0e26707c30190fec/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 196875 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 25a9c5878140837044bcb9ee0e26707c30190fec, data reload: false

query1	1251	921	902	902
query2	6429	1970	1872	1872
query3	10613	3957	4076	3957
query4	60374	28810	23238	23238
query5	4972	494	482	482
query6	385	179	153	153
query7	5620	292	297	292
query8	307	213	208	208
query9	7722	2474	2465	2465
query10	434	276	262	262
query11	17574	15366	15418	15366
query12	152	103	107	103
query13	1467	398	406	398
query14	10602	7370	6881	6881
query15	215	191	174	174
query16	6690	478	504	478
query17	1094	577	546	546
query18	1470	286	316	286
query19	200	152	157	152
query20	120	108	112	108
query21	221	108	105	105
query22	4673	4529	4558	4529
query23	34277	33680	33722	33680
query24	6005	2983	2810	2810
query25	512	381	385	381
query26	589	152	153	152
query27	1586	280	275	275
query28	3889	2024	2016	2016
query29	637	409	406	406
query30	242	153	151	151
query31	930	741	780	741
query32	74	51	52	51
query33	407	292	288	288
query34	887	469	491	469
query35	831	719	726	719
query36	1068	933	953	933
query37	152	85	95	85
query38	4045	3985	3869	3869
query39	1440	1410	1420	1410
query40	194	113	112	112
query41	46	76	44	44
query42	117	95	100	95
query43	502	470	460	460
query44	1135	749	736	736
query45	198	163	164	163
query46	1073	768	745	745
query47	1927	1764	1779	1764
query48	352	286	291	286
query49	764	428	442	428
query50	826	411	419	411
query51	7080	6858	6894	6858
query52	103	86	87	86
query53	250	179	180	179
query54	551	450	453	450
query55	76	73	73	73
query56	274	257	252	252
query57	1215	1064	1102	1064
query58	217	237	236	236
query59	2954	2863	2878	2863
query60	283	281	269	269
query61	139	100	97	97
query62	774	668	638	638
query63	217	186	194	186
query64	1324	679	670	670
query65	3258	3165	3152	3152
query66	675	351	343	343
query67	15655	15508	15207	15207
query68	1592	536	537	536
query69	439	281	274	274
query70	1232	1103	1131	1103
query71	325	274	266	266
query72	4597	3939	4160	3939
query73	734	317	329	317
query74	9122	8866	8847	8847
query75	3359	2686	2676	2676
query76	1354	970	948	948
query77	493	316	315	315
query78	9916	9916	9306	9306
query79	935	859	840	840
query80	845	801	780	780
query81	531	259	256	256
query82	264	259	262	259
query83	190	191	186	186
query84	263	103	102	102
query85	712	384	379	379
query86	308	314	327	314
query87	4376	4442	4422	4422
query88	4799	3993	4065	3993
query89	373	360	356	356
query90	2175	306	304	304
query91	124	147	123	123
query92	76	74	75	74
query93	895	893	891	891
query94	710	342	390	342
query95	436	404	398	398
query96	460	459	460	459
query97	3125	3117	3115	3115
query98	234	232	224	224
query99	1488	1290	1271	1271
Total cold run time: 298817 ms
Total hot run time: 196875 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.88 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 25a9c5878140837044bcb9ee0e26707c30190fec, data reload: false

query1	0.05	0.04	0.03
query2	0.09	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.08	0.08
query5	0.50	0.52	0.51
query6	1.13	0.72	0.74
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.50	0.48
query10	0.55	0.57	0.55
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.61	0.59	0.59
query14	1.42	1.42	1.41
query15	0.85	0.84	0.82
query16	0.37	0.38	0.38
query17	1.05	1.04	1.01
query18	0.22	0.21	0.20
query19	1.86	1.70	1.74
query20	0.02	0.01	0.01
query21	15.41	0.66	0.66
query22	4.18	6.72	2.23
query23	18.27	1.40	1.34
query24	2.09	0.22	0.21
query25	0.16	0.10	0.08
query26	0.25	0.18	0.17
query27	0.08	0.07	0.07
query28	13.28	1.02	1.01
query29	12.60	3.41	3.42
query30	0.25	0.06	0.06
query31	2.87	0.41	0.41
query32	3.24	0.47	0.48
query33	2.98	3.02	3.03
query34	17.12	4.46	4.40
query35	4.46	4.44	4.44
query36	0.66	0.46	0.49
query37	0.18	0.15	0.15
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.15	0.12	0.12
query41	0.08	0.04	0.04
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.16 s
Total hot run time: 31.88 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 10, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@Gabriel39 Gabriel39 merged commit cd902b6 into apache:master Sep 10, 2024
22 of 28 checks passed
Gabriel39 added a commit to Gabriel39/incubator-doris that referenced this pull request Oct 11, 2024
@Gabriel39
[refactor](pipeline) Refactor logics (apache#40576)
778355a
@Gabriel39 Gabriel39 mentioned this pull request Oct 11, 2024
Merged
Gabriel39 added a commit that referenced this pull request Oct 11, 2024
@Gabriel39
[branch-3.0](pick) Pick 6 commits (#41715)
8260737 
pick #39982 #41292 #40576 #41555 #41589 #41592
Gabriel39 added a commit to Gabriel39/incubator-doris that referenced this pull request Oct 14, 2024
@Gabriel39
[refactor](pipeline) Refactor logics (apache#40576)
03439a0
dataroaring pushed a commit that referenced this pull request Oct 15, 2024
@Gabriel39
[pick](branch-3.0) pick #39982 #40576 #41555 #41592 #41676 #41740 (#4…
54b4fa3 
…1808)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Mryange Mryange Mryange approved these changes

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Gabriel39 @doris-robot @BiteTheDDDDt @Mryange