[Fix](Nereids) fix append_trailing_char_if_absent function return null #40820

LiBinfeng-01 · 2024-09-13T11:14:55Z

example: select append_trailing_char_if_absent('it','a') would return null in original design, it can not return null when folding constant on fe any time

doris-robot · 2024-09-13T11:15:00Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

LiBinfeng-01 · 2024-09-13T11:15:04Z

run buildall

morrySnow · 2024-09-13T11:21:41Z

...in/java/org/apache/doris/nereids/trees/expressions/functions/executable/TimeRoundSeries.java

@@ -91,7 +92,7 @@ private static LocalDateTime getDateCeilOrFloor(DATE tag, LocalDateTime date, in
                break;
            }
            default: {
-                return null;
+                throw new AnalysisException("Unexpected tag: " + tag);


why change this? return null is correct because we do not want to throw exception

here return null is reasonable, I check caller, they have deal with null. Let me restore this behavior

LiBinfeng-01 · 2024-09-13T11:30:36Z

run buildall

doris-robot · 2024-09-13T11:51:45Z

TPC-H: Total hot run time: 42772 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e60ce35353c212025fbb1e7ea12794ab07cff7b, data reload: false

------ Round 1 ----------------------------------
q1	19249	7440	7235	7235
q2	2035	188	196	188
q3	10445	1313	1407	1313
q4	10169	959	1008	959
q5	7710	3165	3108	3108
q6	240	156	151	151
q7	1035	635	633	633
q8	9470	1993	2025	1993
q9	6806	6285	6318	6285
q10	7013	2493	2480	2480
q11	436	260	260	260
q12	416	230	233	230
q13	17762	3053	3061	3053
q14	281	256	258	256
q15	586	528	515	515
q16	523	421	424	421
q17	992	949	941	941
q18	7351	6674	6739	6674
q19	1384	1239	1222	1222
q20	610	342	328	328
q21	3913	3562	3537	3537
q22	1083	990	1035	990
Total cold run time: 109509 ms
Total hot run time: 42772 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7222	7278	7173	7173
q2	338	246	229	229
q3	2918	2872	2942	2872
q4	1966	1974	1958	1958
q5	5443	5384	5448	5384
q6	233	147	145	145
q7	2049	1661	1680	1661
q8	3247	3309	3325	3309
q9	8428	8417	8399	8399
q10	3405	3481	3470	3470
q11	571	471	471	471
q12	792	546	575	546
q13	5492	3056	3072	3056
q14	291	269	286	269
q15	569	521	516	516
q16	493	454	442	442
q17	1736	1693	1687	1687
q18	7857	7621	7445	7445
q19	1715	1693	1684	1684
q20	2035	1799	1833	1799
q21	5751	5427	5464	5427
q22	1094	1051	986	986
Total cold run time: 63645 ms
Total hot run time: 58928 ms

doris-robot · 2024-09-13T12:03:35Z

TPC-DS: Total hot run time: 195890 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7e60ce35353c212025fbb1e7ea12794ab07cff7b, data reload: false

query1	924	380	369	369
query2	6454	1792	1754	1754
query3	6681	212	225	212
query4	25633	24118	24089	24089
query5	5082	557	535	535
query6	263	179	165	165
query7	4591	311	314	311
query8	295	226	226	226
query9	8507	2562	2581	2562
query10	483	293	283	283
query11	16267	15684	15876	15684
query12	160	100	103	100
query13	1686	409	383	383
query14	11672	6920	7061	6920
query15	223	172	177	172
query16	7624	499	431	431
query17	1559	591	570	570
query18	1984	303	296	296
query19	197	151	154	151
query20	124	114	115	114
query21	218	107	104	104
query22	4428	4397	4302	4302
query23	34794	33791	33706	33706
query24	9486	3114	3096	3096
query25	676	415	414	414
query26	1340	161	161	161
query27	2801	287	288	287
query28	6823	2114	2084	2084
query29	949	440	434	434
query30	304	159	164	159
query31	1021	793	816	793
query32	110	57	60	57
query33	748	308	304	304
query34	907	479	503	479
query35	887	767	765	765
query36	1075	894	923	894
query37	167	88	87	87
query38	4032	3860	3970	3860
query39	1509	1414	1398	1398
query40	279	121	119	119
query41	50	50	47	47
query42	120	97	98	97
query43	514	425	455	425
query44	1259	813	776	776
query45	199	174	178	174
query46	1123	830	832	830
query47	1905	1804	1828	1804
query48	375	294	296	294
query49	1127	471	464	464
query50	912	435	444	435
query51	7076	6961	6996	6961
query52	104	88	90	88
query53	258	186	194	186
query54	814	482	475	475
query55	82	76	80	76
query56	295	286	285	285
query57	1207	1094	1091	1091
query58	356	244	270	244
query59	3027	2673	2451	2451
query60	325	274	276	274
query61	105	116	104	104
query62	925	662	704	662
query63	228	192	196	192
query64	5152	689	710	689
query65	3296	3216	3187	3187
query66	1400	300	299	299
query67	16055	15539	15514	15514
query68	3173	863	858	858
query69	428	329	324	324
query70	1182	1180	1187	1180
query71	354	351	348	348
query72	5928	3367	3385	3367
query73	586	583	589	583
query74	9289	9322	9249	9249
query75	3149	3026	3036	3026
query76	1904	842	840	840
query77	439	404	405	404
query78	9373	9275	9315	9275
query79	924	897	871	871
query80	847	833	826	826
query81	455	261	263	261
query82	271	261	262	261
query83	194	189	198	189
query84	238	112	107	107
query85	661	413	398	398
query86	317	319	295	295
query87	4306	4368	4413	4368
query88	4135	4077	4053	4053
query89	374	372	377	372
query90	1315	331	318	318
query91	124	126	127	126
query92	80	74	77	74
query93	1038	1057	1027	1027
query94	622	404	392	392
query95	480	446	423	423
query96	477	470	466	466
query97	3102	3133	3159	3133
query98	236	226	225	225
query99	1585	1317	1297	1297
Total cold run time: 278838 ms
Total hot run time: 195890 ms

doris-robot · 2024-09-13T12:09:05Z

ClickBench: Total hot run time: 31.06 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e60ce35353c212025fbb1e7ea12794ab07cff7b, data reload: false

query1	0.04	0.05	0.04
query2	0.08	0.04	0.05
query3	0.22	0.04	0.05
query4	1.68	0.07	0.06
query5	0.49	0.50	0.51
query6	1.16	0.73	0.72
query7	0.02	0.01	0.02
query8	0.06	0.04	0.04
query9	0.55	0.51	0.50
query10	0.56	0.58	0.56
query11	0.16	0.12	0.12
query12	0.15	0.14	0.12
query13	0.63	0.61	0.60
query14	1.47	1.49	1.50
query15	0.92	0.87	0.88
query16	0.36	0.36	0.36
query17	1.00	1.06	1.04
query18	0.25	0.20	0.21
query19	1.89	1.83	1.77
query20	0.01	0.01	0.02
query21	15.40	0.68	0.68
query22	4.29	8.14	1.24
query23	17.73	1.27	1.31
query24	2.29	0.25	0.22
query25	0.19	0.08	0.07
query26	0.28	0.18	0.18
query27	0.08	0.08	0.08
query28	13.18	1.14	1.12
query29	12.54	3.39	3.31
query30	0.25	0.06	0.06
query31	2.86	0.41	0.41
query32	3.24	0.48	0.50
query33	2.99	3.05	3.09
query34	15.43	4.34	4.34
query35	4.37	4.34	4.40
query36	0.67	0.50	0.49
query37	0.19	0.17	0.17
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.17	0.14	0.14
query41	0.10	0.05	0.04
query42	0.06	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 108.27 s
Total hot run time: 31.06 s

LiBinfeng-01 · 2024-09-14T02:04:13Z

run cloud_p0

LiBinfeng-01 · 2024-09-17T10:00:52Z

run buildall

doris-robot · 2024-09-17T10:46:27Z

TPC-H: Total hot run time: 41505 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 87e4de4d86c44281670e40eee95ab8596995684a, data reload: false

------ Round 1 ----------------------------------
q1	17623	7551	7422	7422
q2	2062	165	154	154
q3	10647	1056	1101	1056
q4	10521	708	731	708
q5	7781	3033	3047	3033
q6	237	147	146	146
q7	993	611	597	597
q8	9449	2033	2044	2033
q9	6814	6405	6408	6405
q10	7002	2268	2267	2267
q11	439	239	247	239
q12	412	217	223	217
q13	17763	2966	3002	2966
q14	238	216	215	215
q15	570	525	528	525
q16	682	617	605	605
q17	962	808	828	808
q18	7227	6581	6565	6565
q19	1412	1035	997	997
q20	579	279	279	279
q21	3918	3318	3267	3267
q22	1094	1012	1001	1001
Total cold run time: 108425 ms
Total hot run time: 41505 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7269	7276	7332	7276
q2	316	225	228	225
q3	2975	2893	2988	2893
q4	1963	1797	1807	1797
q5	5638	5589	5592	5589
q6	228	150	151	150
q7	2247	1806	1791	1791
q8	3307	3434	3467	3434
q9	8720	8912	8754	8754
q10	3477	3430	3468	3430
q11	585	501	478	478
q12	824	622	670	622
q13	11098	3146	3142	3142
q14	307	281	286	281
q15	579	509	527	509
q16	725	671	660	660
q17	1809	1581	1570	1570
q18	8137	7771	7837	7771
q19	1740	1588	1568	1568
q20	2101	1886	1914	1886
q21	5509	5320	5491	5320
q22	1117	1057	1077	1057
Total cold run time: 70671 ms
Total hot run time: 60203 ms

doris-robot · 2024-09-17T10:58:58Z

TPC-DS: Total hot run time: 198636 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 87e4de4d86c44281670e40eee95ab8596995684a, data reload: false

query1	1275	888	862	862
query2	6398	2054	2068	2054
query3	10746	4099	3859	3859
query4	65519	28321	23516	23516
query5	5058	469	479	469
query6	422	175	169	169
query7	5478	304	298	298
query8	321	221	227	221
query9	8532	2667	2634	2634
query10	424	284	281	281
query11	16861	15079	15786	15079
query12	156	100	100	100
query13	1542	435	423	423
query14	10554	7712	7805	7712
query15	196	183	173	173
query16	6524	511	485	485
query17	1150	615	579	579
query18	1460	340	313	313
query19	211	150	153	150
query20	136	113	111	111
query21	215	105	109	105
query22	4843	4501	4620	4501
query23	34767	33952	33735	33735
query24	6079	2926	2848	2848
query25	534	407	392	392
query26	609	158	158	158
query27	1595	290	288	288
query28	4093	2463	2416	2416
query29	669	432	435	432
query30	232	160	155	155
query31	972	779	777	777
query32	72	57	57	57
query33	420	289	295	289
query34	904	493	479	479
query35	857	743	720	720
query36	1048	933	962	933
query37	147	87	92	87
query38	4018	3920	3867	3867
query39	1474	1405	1399	1399
query40	209	95	98	95
query41	50	48	49	48
query42	115	100	95	95
query43	532	490	472	472
query44	1151	819	791	791
query45	192	163	165	163
query46	1145	774	827	774
query47	1906	1792	1814	1792
query48	464	368	388	368
query49	673	391	390	390
query50	829	409	408	408
query51	7077	7101	6980	6980
query52	98	87	86	86
query53	246	187	181	181
query54	563	443	453	443
query55	81	82	74	74
query56	281	264	258	258
query57	1165	1102	1082	1082
query58	220	233	251	233
query59	3126	3005	3104	3005
query60	286	262	267	262
query61	103	102	100	100
query62	767	663	635	635
query63	214	181	177	177
query64	1319	656	617	617
query65	3267	3197	3168	3168
query66	604	297	286	286
query67	16010	15675	15255	15255
query68	1470	564	552	552
query69	434	287	296	287
query70	1159	1132	1136	1132
query71	323	253	260	253
query72	6054	3953	3986	3953
query73	757	327	331	327
query74	9404	8972	8958	8958
query75	3323	2715	2691	2691
query76	1413	815	940	815
query77	479	291	333	291
query78	10105	9588	9235	9235
query79	1082	891	866	866
query80	816	593	572	572
query81	504	253	256	253
query82	1306	229	225	225
query83	221	163	159	159
query84	286	106	101	101
query85	743	371	362	362
query86	361	312	317	312
query87	4497	4278	4318	4278
query88	4355	4037	4021	4021
query89	393	361	358	358
query90	1819	310	303	303
query91	161	163	163	163
query92	76	70	70	70
query93	951	888	884	884
query94	720	383	383	383
query95	449	409	416	409
query96	496	482	480	480
query97	3166	3088	3195	3088
query98	224	225	215	215
query99	1480	1283	1276	1276
Total cold run time: 308231 ms
Total hot run time: 198636 ms

doris-robot · 2024-09-17T11:04:30Z

ClickBench: Total hot run time: 32.39 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 87e4de4d86c44281670e40eee95ab8596995684a, data reload: false

query1	0.05	0.05	0.04
query2	0.06	0.03	0.02
query3	0.22	0.06	0.06
query4	1.66	0.10	0.10
query5	0.51	0.51	0.52
query6	1.16	0.72	0.72
query7	0.02	0.01	0.02
query8	0.03	0.03	0.03
query9	0.56	0.50	0.50
query10	0.56	0.57	0.53
query11	0.17	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.60	0.60
query14	3.06	3.13	2.95
query15	0.90	0.80	0.82
query16	0.38	0.38	0.38
query17	1.00	1.01	0.96
query18	0.19	0.19	0.20
query19	1.98	1.81	2.01
query20	0.01	0.01	0.01
query21	15.38	0.60	0.61
query22	2.80	3.17	1.80
query23	17.47	0.82	0.82
query24	2.50	0.66	1.33
query25	0.30	0.18	0.15
query26	0.29	0.14	0.15
query27	0.04	0.04	0.04
query28	11.06	1.09	1.07
query29	12.56	3.22	3.21
query30	0.25	0.06	0.06
query31	2.88	0.39	0.40
query32	3.24	0.45	0.45
query33	3.00	3.04	3.01
query34	16.80	4.37	4.42
query35	4.41	4.45	4.41
query36	0.66	0.48	0.49
query37	0.08	0.06	0.06
query38	0.04	0.03	0.04
query39	0.04	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.38 s
Total hot run time: 32.39 s

LiBinfeng-01 · 2024-09-17T11:58:11Z

run cloud_p0

github-actions · 2024-09-18T02:32:51Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-09-18T02:32:52Z

PR approved by anyone and no changes requested.

#40820) (#40890) cherry-pick： #40820 example: select append_trailing_char_if_absent('it','a') would return null in original design, it can not return null when folding constant on fe any time

#41157) cherry-pick： #40820 example: select append_trailing_char_if_absent('it','a') would return null in original design, it can not return null when folding constant on fe any time

…40745)(40820) (#41837) cherry-pick from master #40441 #40772 #40744 #40745 #40820

#40820) (#41160) cherry-pick from master #40820 example: select append_trailing_char_if_absent('it','a') would return null in original design, it can not return null when folding constant on fe any time

…40820)(#40745) (#41930) cherry-pick from master (#40441)(#40772)(#40744)(#40820)(#40745)

morrySnow reviewed Sep 13, 2024

View reviewed changes

[Fix](Nereids) fix append_trailing_char_if_absent function return null

7e60ce3

LiBinfeng-01 force-pushed the fix_function_append_trailing_char_if_absent branch from 1ff781d to 7e60ce3 Compare September 13, 2024 11:30

add more cases and fix bug

87e4de4

LiBinfeng-01 mentioned this pull request Sep 18, 2024

[Fix](Nereids) fix append_trailing_char_if_absent function return null (#40820) #40890

Merged

englefly approved these changes Sep 18, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 18, 2024

github-actions bot added the reviewed label Sep 18, 2024

morrySnow approved these changes Sep 18, 2024

View reviewed changes

morrySnow merged commit 1501597 into apache:master Sep 18, 2024
25 of 28 checks passed

morrySnow added dev/2.1.x dev/2.0.x labels Sep 20, 2024

This was referenced Sep 23, 2024

[Fix](Nereids) fix append_trailing_char_if_absent function return null #41157

Merged

[Fix](Nereids) fix append_trailing_char_if_absent function return null (#40820) #41160

Merged

yiguolei added dev/2.1.7-merged and removed dev/2.1.x labels Sep 26, 2024

dataroaring added the dev/3.0.2-merged label Oct 13, 2024

This was referenced Oct 16, 2024

[Feat](Nereids) support fold constant by fe (#40441)(#40772)(#40744)(#40745)(40820) #41837

Merged

[Feat](Nereids) support fold constant by fe (#40441)(#40772)(#40744)(40820)(#40745) #41930

Merged

morrySnow pushed a commit that referenced this pull request Oct 17, 2024

[Feat](Nereids) support fold constant by fe (#40441)(#40772)(#40744)(#…

5fe37c0

…40745)(40820) (#41837) cherry-pick from master #40441 #40772 #40744 #40745 #40820

morrySnow added dev/2.0.16-merged and removed dev/2.0.x labels Oct 21, 2024

morrySnow pushed a commit that referenced this pull request Oct 21, 2024

[Feat](Nereids) support fold constant by fe (#40441)(#40772)(#40744)(…

ef7714b

…40820)(#40745) (#41930) cherry-pick from master (#40441)(#40772)(#40744)(#40820)(#40745)

yiguolei mentioned this pull request Nov 6, 2024

Release Note 2.1.7 #43319

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fix](Nereids) fix append_trailing_char_if_absent function return null #40820

[Fix](Nereids) fix append_trailing_char_if_absent function return null #40820

LiBinfeng-01 commented Sep 13, 2024

doris-robot commented Sep 13, 2024

LiBinfeng-01 commented Sep 13, 2024

morrySnow Sep 13, 2024

LiBinfeng-01 Sep 13, 2024

LiBinfeng-01 Sep 13, 2024

LiBinfeng-01 commented Sep 13, 2024

doris-robot commented Sep 13, 2024

doris-robot commented Sep 13, 2024

doris-robot commented Sep 13, 2024

LiBinfeng-01 commented Sep 14, 2024

LiBinfeng-01 commented Sep 17, 2024

doris-robot commented Sep 17, 2024

doris-robot commented Sep 17, 2024

doris-robot commented Sep 17, 2024

LiBinfeng-01 commented Sep 17, 2024

github-actions bot commented Sep 18, 2024

github-actions bot commented Sep 18, 2024

[Fix](Nereids) fix append_trailing_char_if_absent function return null #40820

[Fix](Nereids) fix append_trailing_char_if_absent function return null #40820

Conversation

LiBinfeng-01 commented Sep 13, 2024

doris-robot commented Sep 13, 2024

LiBinfeng-01 commented Sep 13, 2024

morrySnow Sep 13, 2024

Choose a reason for hiding this comment

LiBinfeng-01 Sep 13, 2024

Choose a reason for hiding this comment

LiBinfeng-01 Sep 13, 2024

Choose a reason for hiding this comment

LiBinfeng-01 commented Sep 13, 2024

doris-robot commented Sep 13, 2024

doris-robot commented Sep 13, 2024

doris-robot commented Sep 13, 2024

LiBinfeng-01 commented Sep 14, 2024

LiBinfeng-01 commented Sep 17, 2024

doris-robot commented Sep 17, 2024

doris-robot commented Sep 17, 2024

doris-robot commented Sep 17, 2024

LiBinfeng-01 commented Sep 17, 2024

github-actions bot commented Sep 18, 2024

github-actions bot commented Sep 18, 2024