Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](function) Avoid create_column inside cast_column function #41775

Merged
merged 1 commit into from
Oct 15, 2024
Merged

[opt](function) Avoid create_column inside cast_column function #41775

merged 1 commit into from
Oct 15, 2024

Conversation

Mryange
Copy link
Contributor

@Mryange Mryange commented Oct 13, 2024 •
edited
Loading

Proposed changes

In the cast function, we don't need to initialize the result column.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Mryange
Copy link
Contributor Author

Mryange commented Oct 13, 2024

run buildall

@Mryange Mryange marked this pull request as draft October 13, 2024 13:56
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.43% (9681/25862)
Line Coverage: 28.69% (80300/279922)
Region Coverage: 28.12% (41527/147678)
Branch Coverage: 24.70% (21112/85460)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ac9537d713a26a831236fa2ba1529991cb78c7bf_ac9537d713a26a831236fa2ba1529991cb78c7bf/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 40843 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ac9537d713a26a831236fa2ba1529991cb78c7bf, data reload: false

------ Round 1 ----------------------------------
q1	17640	8112	7237	7237
q2	2017	275	269	269
q3	12031	1068	1170	1068
q4	10557	744	776	744
q5	7757	2836	2825	2825
q6	239	145	149	145
q7	985	630	649	630
q8	9343	1870	1949	1870
q9	6532	6465	6369	6369
q10	7081	2308	2291	2291
q11	425	242	240	240
q12	400	222	222	222
q13	17804	3031	2979	2979
q14	257	211	206	206
q15	573	528	527	527
q16	656	579	593	579
q17	971	550	511	511
q18	7270	6839	6862	6839
q19	1354	966	932	932
q20	481	205	205	205
q21	4057	3253	3143	3143
q22	1149	1012	1014	1012
Total cold run time: 109579 ms
Total hot run time: 40843 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7377	7379	7337	7337
q2	327	240	250	240
q3	3126	2982	2942	2942
q4	2112	1877	1838	1838
q5	5806	5957	5856	5856
q6	234	142	147	142
q7	2338	1974	1862	1862
q8	3445	3663	3547	3547
q9	9265	9129	8924	8924
q10	3604	3570	3532	3532
q11	576	491	503	491
q12	828	661	610	610
q13	15225	3113	3171	3113
q14	296	269	290	269
q15	554	521	513	513
q16	710	645	644	644
q17	1850	1657	1608	1608
q18	8206	7612	7743	7612
q19	1693	1519	1495	1495
q20	2133	1894	1892	1892
q21	5569	5377	5271	5271
q22	1134	1050	1044	1044
Total cold run time: 76408 ms
Total hot run time: 60782 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191015 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ac9537d713a26a831236fa2ba1529991cb78c7bf, data reload: false

query1	980	376	398	376
query2	6261	2079	2058	2058
query3	8687	193	200	193
query4	34264	23552	23398	23398
query5	3660	473	467	467
query6	271	165	176	165
query7	4187	301	311	301
query8	278	208	221	208
query9	9677	2672	2677	2672
query10	463	285	284	284
query11	17790	15176	15220	15176
query12	159	108	98	98
query13	1573	473	429	429
query14	9619	6423	7032	6423
query15	250	174	176	174
query16	7997	485	496	485
query17	1641	618	591	591
query18	2115	314	323	314
query19	253	155	153	153
query20	121	120	124	120
query21	207	109	108	108
query22	4986	4616	4775	4616
query23	35046	34090	34177	34090
query24	11009	2830	2853	2830
query25	628	399	405	399
query26	1188	162	158	158
query27	2574	299	295	295
query28	7549	2423	2403	2403
query29	847	437	431	431
query30	282	152	152	152
query31	1095	803	817	803
query32	94	53	56	53
query33	756	299	304	299
query34	906	517	510	510
query35	889	732	752	732
query36	1083	946	938	938
query37	150	91	81	81
query38	4083	4038	3867	3867
query39	1536	1423	1423	1423
query40	208	96	95	95
query41	46	45	47	45
query42	115	94	94	94
query43	528	487	483	483
query44	1233	807	811	807
query45	192	170	165	165
query46	1134	716	722	716
query47	1950	1872	1836	1836
query48	432	347	353	347
query49	964	415	409	409
query50	805	418	406	406
query51	7055	6951	6919	6919
query52	98	87	92	87
query53	255	180	180	180
query54	1077	488	472	472
query55	73	75	75	75
query56	272	257	267	257
query57	1239	1123	1118	1118
query58	237	235	251	235
query59	3218	3059	2935	2935
query60	290	259	268	259
query61	107	100	102	100
query62	830	676	655	655
query63	209	194	182	182
query64	3930	636	608	608
query65	3443	3220	3195	3195
query66	837	293	291	291
query67	15932	15618	15590	15590
query68	4740	559	551	551
query69	525	291	296	291
query70	1186	1111	1126	1111
query71	339	270	276	270
query72	7377	3984	3951	3951
query73	787	342	350	342
query74	10443	8919	9025	8919
query75	3458	2660	2640	2640
query76	2954	889	919	889
query77	510	304	282	282
query78	10521	9682	9545	9545
query79	2240	591	608	591
query80	2229	433	446	433
query81	596	236	236	236
query82	731	143	138	138
query83	290	138	133	133
query84	277	77	83	77
query85	1226	295	280	280
query86	318	255	298	255
query87	4467	4391	4301	4301
query88	4105	2381	2350	2350
query89	404	283	284	283
query90	2087	180	181	180
query91	138	106	105	105
query92	62	49	50	49
query93	1227	564	557	557
query94	917	297	287	287
query95	353	258	243	243
query96	604	274	275	274
query97	3295	3136	3168	3136
query98	210	193	191	191
query99	1724	1291	1305	1291
Total cold run time: 302203 ms
Total hot run time: 191015 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.92 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ac9537d713a26a831236fa2ba1529991cb78c7bf, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.02	0.03
query3	0.23	0.07	0.06
query4	1.64	0.10	0.10
query5	0.50	0.51	0.51
query6	1.14	0.72	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.58	0.49	0.50
query10	0.55	0.54	0.53
query11	0.14	0.10	0.11
query12	0.13	0.10	0.10
query13	0.60	0.60	0.60
query14	2.71	2.71	2.70
query15	0.91	0.82	0.81
query16	0.39	0.38	0.38
query17	1.07	1.00	1.01
query18	0.20	0.19	0.20
query19	1.86	1.78	1.83
query20	0.02	0.01	0.01
query21	15.35	0.58	0.56
query22	2.59	1.94	2.09
query23	17.05	0.94	0.92
query24	3.44	1.67	1.19
query25	0.35	0.14	0.06
query26	0.47	0.14	0.13
query27	0.05	0.04	0.03
query28	9.70	1.10	1.08
query29	12.56	3.26	3.20
query30	0.25	0.06	0.07
query31	2.86	0.38	0.39
query32	3.27	0.46	0.46
query33	2.96	2.99	3.09
query34	17.08	4.49	4.50
query35	4.57	4.55	4.47
query36	0.66	0.50	0.50
query37	0.08	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 106.47 s
Total hot run time: 32.92 s

@Mryange Mryange changed the title [only test] [opt](function) Avoid create_column inside cast_column function Oct 14, 2024
@Mryange Mryange marked this pull request as ready for review October 14, 2024 02:42
@Mryange
Copy link
Contributor Author

Mryange commented Oct 14, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.46% (9702/25897)
Line Coverage: 28.74% (80525/280228)
Region Coverage: 28.17% (41657/147863)
Branch Coverage: 24.74% (21170/85556)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ac9537d713a26a831236fa2ba1529991cb78c7bf_ac9537d713a26a831236fa2ba1529991cb78c7bf/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41381 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ac9537d713a26a831236fa2ba1529991cb78c7bf, data reload: false

------ Round 1 ----------------------------------
q1	17672	7576	7275	7275
q2	2046	282	270	270
q3	12098	1066	1149	1066
q4	10560	863	922	863
q5	7764	3143	3083	3083
q6	242	150	148	148
q7	1045	654	605	605
q8	9358	1933	2000	1933
q9	6570	6478	6441	6441
q10	7098	2464	2424	2424
q11	440	247	246	246
q12	421	220	225	220
q13	17806	3044	3019	3019
q14	235	210	210	210
q15	574	524	517	517
q16	654	594	610	594
q17	980	595	473	473
q18	7372	6763	6827	6763
q19	1343	1055	898	898
q20	483	192	186	186
q21	3948	3321	3174	3174
q22	1121	1014	973	973
Total cold run time: 109830 ms
Total hot run time: 41381 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7291	7265	7237	7237
q2	334	228	235	228
q3	3070	2945	2888	2888
q4	2130	1888	1856	1856
q5	5764	5803	5796	5796
q6	235	142	147	142
q7	2265	1855	1825	1825
q8	3425	3524	3518	3518
q9	8889	8935	8870	8870
q10	3584	3624	3535	3535
q11	606	503	487	487
q12	868	644	656	644
q13	8700	3200	3186	3186
q14	324	276	271	271
q15	587	518	515	515
q16	699	657	649	649
q17	1849	1641	1615	1615
q18	8497	7969	7703	7703
q19	1749	1524	1492	1492
q20	2138	1880	1909	1880
q21	5636	5420	5453	5420
q22	1123	1071	1065	1065
Total cold run time: 69763 ms
Total hot run time: 60822 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191550 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ac9537d713a26a831236fa2ba1529991cb78c7bf, data reload: false

query1	889	404	397	397
query2	6248	2189	2135	2135
query3	8685	202	204	202
query4	34251	23705	23528	23528
query5	3419	486	484	484
query6	268	165	172	165
query7	4186	286	285	285
query8	285	232	226	226
query9	9585	2636	2646	2636
query10	454	275	258	258
query11	17768	15195	15291	15195
query12	150	100	95	95
query13	1573	445	414	414
query14	9639	6685	7510	6685
query15	249	167	174	167
query16	8018	441	471	441
query17	1633	650	606	606
query18	2166	312	321	312
query19	319	164	162	162
query20	126	120	119	119
query21	209	111	106	106
query22	4954	4458	4583	4458
query23	35320	34356	35137	34356
query24	11198	2771	2746	2746
query25	559	406	405	405
query26	756	157	165	157
query27	2284	300	292	292
query28	7458	2419	2411	2411
query29	665	432	422	422
query30	307	157	154	154
query31	1035	831	778	778
query32	94	56	58	56
query33	769	304	288	288
query34	924	528	518	518
query35	893	752	735	735
query36	1120	977	940	940
query37	145	87	103	87
query38	4047	3927	4109	3927
query39	1539	1441	1429	1429
query40	208	97	98	97
query41	48	45	45	45
query42	126	101	102	101
query43	542	517	484	484
query44	1293	819	802	802
query45	197	165	165	165
query46	1162	704	701	701
query47	1965	1863	1856	1856
query48	405	332	323	323
query49	1010	411	404	404
query50	823	383	393	383
query51	7215	6853	6951	6853
query52	101	88	88	88
query53	260	184	186	184
query54	1244	423	431	423
query55	86	81	75	75
query56	300	262	275	262
query57	1276	1146	1156	1146
query58	244	243	251	243
query59	3243	3022	3014	3014
query60	285	260	260	260
query61	106	102	98	98
query62	873	676	659	659
query63	226	187	191	187
query64	3885	645	617	617
query65	3318	3219	3206	3206
query66	832	293	297	293
query67	16013	15622	15724	15622
query68	4652	563	549	549
query69	468	287	294	287
query70	1183	1090	1144	1090
query71	343	274	263	263
query72	7039	4057	4081	4057
query73	776	349	355	349
query74	10393	9015	8980	8980
query75	3433	2700	2680	2680
query76	2812	925	917	917
query77	626	295	310	295
query78	10529	9581	9605	9581
query79	2681	596	606	596
query80	2548	447	453	447
query81	587	242	239	239
query82	717	134	135	134
query83	298	136	138	136
query84	289	73	72	72
query85	1419	297	290	290
query86	409	293	302	293
query87	4520	4231	4318	4231
query88	3834	2189	2155	2155
query89	418	285	290	285
query90	1852	192	186	186
query91	143	103	117	103
query92	61	52	49	49
query93	1612	531	540	531
query94	997	289	287	287
query95	349	242	244	242
query96	607	274	284	274
query97	3241	3153	3119	3119
query98	221	195	200	195
query99	1800	1270	1296	1270
Total cold run time: 302330 ms
Total hot run time: 191550 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.12 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ac9537d713a26a831236fa2ba1529991cb78c7bf, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.06	0.06
query4	1.64	0.10	0.10
query5	0.52	0.51	0.50
query6	1.14	0.73	0.72
query7	0.01	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.52	0.49
query10	0.56	0.55	0.55
query11	0.14	0.11	0.11
query12	0.12	0.11	0.11
query13	0.62	0.59	0.60
query14	2.83	2.75	2.70
query15	0.89	0.82	0.84
query16	0.39	0.38	0.38
query17	1.08	1.03	1.02
query18	0.20	0.20	0.20
query19	1.99	1.91	2.03
query20	0.01	0.00	0.01
query21	15.40	0.56	0.57
query22	2.60	2.55	1.24
query23	16.97	0.93	0.77
query24	3.44	1.09	2.22
query25	0.33	0.13	0.04
query26	0.43	0.14	0.13
query27	0.04	0.04	0.04
query28	9.36	1.08	1.07
query29	12.56	3.21	3.18
query30	0.25	0.06	0.06
query31	2.89	0.38	0.37
query32	3.28	0.46	0.46
query33	2.99	3.02	3.03
query34	16.99	4.47	4.43
query35	4.49	4.52	4.52
query36	0.65	0.48	0.49
query37	0.08	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.03
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.2 s
Total hot run time: 32.12 s

eldenmoon
eldenmoon approved these changes Oct 14, 2024
View reviewed changes
Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 14, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@amorynan
Copy link
Contributor

add testcases ?

amorynan
amorynan approved these changes Oct 15, 2024
View reviewed changes
Copy link
Contributor

@amorynan amorynan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

qidaye
qidaye approved these changes Oct 15, 2024
View reviewed changes
Copy link
Contributor

@qidaye qidaye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eldenmoon eldenmoon merged commit 6e7ad0c into apache:master Oct 15, 2024
29 of 32 checks passed
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Oct 24, 2024
@Mryange @eldenmoon
[opt](function) Avoid create_column inside cast_column function (apac…
1837029 
…he#41775)

## Proposed changes

 In the cast function, we don't need to initialize the result column.


<!--Describe your changes.-->
@eldenmoon eldenmoon mentioned this pull request Oct 24, 2024
Merged
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Oct 24, 2024
@Mryange @eldenmoon
[opt](function) Avoid create_column inside cast_column function (apac…
218507c 
…he#41775)

## Proposed changes

 In the cast function, we don't need to initialize the result column.


<!--Describe your changes.-->
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Oct 24, 2024
@Mryange @eldenmoon
[opt](function) Avoid create_column inside cast_column function (apac…
92e9cac 
…he#41775)

## Proposed changes

 In the cast function, we don't need to initialize the result column.


<!--Describe your changes.-->
@eldenmoon eldenmoon mentioned this pull request Oct 24, 2024
Merged
eldenmoon added a commit that referenced this pull request Oct 24, 2024
@eldenmoon @Mryange
[opt](function) Avoid create_column inside cast_column function (#41775…
37b377f 
…) (#42438)

(#41775)

Co-authored-by: Mryange <[email protected]>
eldenmoon added a commit that referenced this pull request Oct 24, 2024
@eldenmoon @Mryange
[opt](function) Avoid create_column inside cast_column function (#41775…
ffffbec 
…) (#42432)

#41775

Co-authored-by: Mryange <[email protected]>
@gavinchou gavinchou added cir and removed approved Indicates a PR has been approved by one committer. labels Nov 11, 2024
@gavinchou gavinchou mentioned this pull request Nov 26, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eldenmoon eldenmoon eldenmoon approved these changes

@qidaye qidaye qidaye approved these changes

@amorynan amorynan amorynan approved these changes

Assignees
No one assigned
Labels
cir dev/2.1.7-merged dev/3.0.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@Mryange @doris-robot @amorynan @qidaye @eldenmoon @gavinchou