Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improvemen](overflow) Provide the user with a suggestion to avoid th… #39631

Merged
merged 3 commits into from
Aug 25, 2024
Merged

[improvemen](overflow) Provide the user with a suggestion to avoid th… #39631

merged 3 commits into from
Aug 25, 2024

Conversation

dataroaring
Copy link
Contributor

…e error.

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@dataroaring
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 38490 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4d5d531616f9df8761d6c58eb7327fa853b0d2ae, data reload: false

------ Round 1 ----------------------------------
q1	18410	4888	4411	4411
q2	2871	209	206	206
q3	11065	1158	1114	1114
q4	11167	766	745	745
q5	7891	2928	2906	2906
q6	289	162	162	162
q7	1033	661	644	644
q8	9400	2146	2091	2091
q9	7161	6568	6598	6568
q10	7096	2189	2230	2189
q11	489	278	264	264
q12	426	246	250	246
q13	18592	2975	2994	2975
q14	304	253	243	243
q15	552	519	529	519
q16	534	406	409	406
q17	1004	650	739	650
q18	7552	6890	6721	6721
q19	5395	1151	1146	1146
q20	683	357	347	347
q21	4139	2902	2949	2902
q22	1157	1039	1035	1035
Total cold run time: 117210 ms
Total hot run time: 38490 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4601	4293	4272	4272
q2	399	299	303	299
q3	2902	2605	2648	2605
q4	1916	1687	1627	1627
q5	5381	5396	5398	5396
q6	231	150	147	147
q7	2085	1728	1758	1728
q8	3244	3429	3386	3386
q9	8522	8415	8397	8397
q10	3398	3205	3205	3205
q11	626	546	517	517
q12	810	662	640	640
q13	17565	3008	3032	3008
q14	331	293	301	293
q15	573	526	522	522
q16	505	453	460	453
q17	1813	1497	1514	1497
q18	7778	7743	7439	7439
q19	1724	1650	1659	1650
q20	2048	1839	1850	1839
q21	10324	5254	5190	5190
q22	1148	1073	1030	1030
Total cold run time: 77924 ms
Total hot run time: 55140 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191501 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4d5d531616f9df8761d6c58eb7327fa853b0d2ae, data reload: false

query1	993	395	398	395
query2	6734	1983	1949	1949
query3	6668	229	248	229
query4	30634	23364	23165	23165
query5	4382	729	700	700
query6	308	245	208	208
query7	4612	336	319	319
query8	482	430	419	419
query9	8570	2527	2508	2508
query10	520	360	353	353
query11	17892	15043	15034	15034
query12	191	140	135	135
query13	1710	424	425	424
query14	9606	7205	7110	7110
query15	271	197	189	189
query16	7853	490	468	468
query17	1618	593	576	576
query18	2081	331	338	331
query19	230	164	164	164
query20	149	135	138	135
query21	254	142	142	142
query22	4177	4131	4016	4016
query23	34041	33230	33277	33230
query24	11651	2940	2971	2940
query25	673	421	420	420
query26	1197	186	180	180
query27	2852	299	297	297
query28	7624	2123	2136	2123
query29	896	457	436	436
query30	341	189	190	189
query31	1077	853	817	817
query32	118	82	80	80
query33	815	355	342	342
query34	985	507	498	498
query35	910	763	765	763
query36	1182	949	973	949
query37	178	105	104	104
query38	3948	3833	3882	3833
query39	1503	1445	1473	1445
query40	243	154	153	153
query41	140	137	138	137
query42	139	117	118	117
query43	553	507	494	494
query44	1267	766	779	766
query45	225	199	191	191
query46	1215	795	814	795
query47	1942	1819	1818	1818
query48	419	357	334	334
query49	1245	576	626	576
query50	924	470	458	458
query51	7210	7118	7090	7090
query52	120	117	109	109
query53	311	230	224	224
query54	946	516	493	493
query55	89	90	89	89
query56	320	302	309	302
query57	1232	1111	1105	1105
query58	310	286	323	286
query59	3078	2904	2798	2798
query60	351	327	346	327
query61	148	146	142	142
query62	923	696	699	696
query63	266	228	222	222
query64	5577	2471	2312	2312
query65	3282	3177	3177	3177
query66	1184	678	705	678
query67	15721	14974	14985	14974
query68	6363	593	589	589
query69	652	436	358	358
query70	1248	1134	1175	1134
query71	545	316	310	310
query72	6645	2358	2084	2084
query73	875	365	361	361
query74	9502	8813	8823	8813
query75	4041	2723	2768	2723
query76	4376	1179	1076	1076
query77	845	431	435	431
query78	9922	9288	9219	9219
query79	4962	564	548	548
query80	1134	621	598	598
query81	622	264	263	263
query82	326	169	169	169
query83	254	213	216	213
query84	296	100	98	98
query85	1061	362	348	348
query86	380	319	328	319
query87	4367	4422	4167	4167
query88	4842	2484	2453	2453
query89	466	327	323	323
query90	2037	232	241	232
query91	159	127	124	124
query92	98	80	77	77
query93	4500	555	547	547
query94	988	319	330	319
query95	387	288	289	288
query96	670	280	278	278
query97	3257	3099	3041	3041
query98	251	226	229	226
query99	1735	1311	1294	1294
Total cold run time: 307112 ms
Total hot run time: 191501 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.3 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4d5d531616f9df8761d6c58eb7327fa853b0d2ae, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.05
query3	0.23	0.05	0.06
query4	1.67	0.08	0.07
query5	0.51	0.49	0.50
query6	1.13	0.72	0.73
query7	0.03	0.02	0.02
query8	0.06	0.05	0.05
query9	0.56	0.48	0.50
query10	0.55	0.55	0.54
query11	0.15	0.12	0.12
query12	0.16	0.13	0.14
query13	0.63	0.59	0.59
query14	0.76	0.78	0.79
query15	0.83	0.83	0.85
query16	0.37	0.35	0.38
query17	0.98	1.05	1.06
query18	0.20	0.20	0.20
query19	1.88	1.69	1.70
query20	0.02	0.01	0.01
query21	15.40	0.65	0.65
query22	4.90	6.62	1.99
query23	18.25	1.38	1.32
query24	2.08	0.24	0.23
query25	0.16	0.09	0.08
query26	0.28	0.18	0.19
query27	0.09	0.09	0.08
query28	13.25	1.04	1.03
query29	12.71	3.42	3.40
query30	0.44	0.24	0.23
query31	2.80	0.41	0.40
query32	3.26	0.48	0.49
query33	2.97	2.97	2.96
query34	16.93	4.37	4.50
query35	4.47	4.42	4.45
query36	0.66	0.48	0.51
query37	0.21	0.18	0.18
query38	0.17	0.17	0.17
query39	0.07	0.05	0.06
query40	0.19	0.15	0.15
query41	0.12	0.07	0.07
query42	0.08	0.07	0.07
query43	0.07	0.06	0.06
Total cold run time: 110.41 s
Total hot run time: 31.3 s

@dataroaring dataroaring force-pushed the column_overflow_prompt branch from 4d5d531 to 6eef526 Compare August 24, 2024 10:31
@dataroaring
Copy link
Contributor Author

run buildall

@dataroaring dataroaring force-pushed the column_overflow_prompt branch from 6eef526 to 29ad578 Compare August 24, 2024 10:35
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 24, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38372 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 29ad5784ffb5edc50405763b019aebc36cfa925a, data reload: false

------ Round 1 ----------------------------------
q1	18151	4692	4409	4409
q2	2047	183	180	180
q3	11763	969	1163	969
q4	10376	780	784	780
q5	7742	2835	2821	2821
q6	226	139	140	139
q7	969	616	602	602
q8	9325	2128	2094	2094
q9	7443	6586	6598	6586
q10	7004	2265	2199	2199
q11	463	250	242	242
q12	404	227	232	227
q13	18944	3053	3072	3053
q14	284	231	231	231
q15	526	495	504	495
q16	504	404	393	393
q17	996	695	717	695
q18	7599	6876	6984	6876
q19	1405	1064	995	995
q20	677	331	353	331
q21	3978	3035	3152	3035
q22	1116	1020	1050	1020
Total cold run time: 111942 ms
Total hot run time: 38372 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4395	4361	4326	4326
q2	392	283	288	283
q3	2914	2661	2684	2661
q4	1956	1678	1699	1678
q5	5695	5693	5764	5693
q6	224	140	142	140
q7	2255	1829	1822	1822
q8	3338	3467	3541	3467
q9	8898	8800	8919	8800
q10	3666	3338	3348	3338
q11	604	539	497	497
q12	871	663	673	663
q13	15427	3266	3299	3266
q14	316	314	302	302
q15	529	503	486	486
q16	505	457	453	453
q17	1880	1563	1541	1541
q18	8071	7840	7960	7840
q19	1731	1569	1597	1569
q20	2153	1932	1900	1900
q21	5825	5479	5456	5456
q22	1152	1070	1061	1061
Total cold run time: 72797 ms
Total hot run time: 57242 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192396 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 29ad5784ffb5edc50405763b019aebc36cfa925a, data reload: false

query1	1246	890	869	869
query2	6285	1897	1850	1850
query3	10603	4110	3822	3822
query4	60098	23876	23266	23266
query5	5492	511	508	508
query6	429	171	175	171
query7	5766	302	294	294
query8	294	207	202	202
query9	8647	2485	2491	2485
query10	475	279	273	273
query11	17799	15014	15287	15014
query12	164	108	108	108
query13	1524	397	401	397
query14	10705	7479	7367	7367
query15	225	175	167	167
query16	7634	505	435	435
query17	1188	568	584	568
query18	2091	295	290	290
query19	299	151	148	148
query20	126	118	113	113
query21	205	101	97	97
query22	4430	4462	4324	4324
query23	34972	34325	33453	33453
query24	5957	2970	2929	2929
query25	506	408	399	399
query26	684	170	162	162
query27	1752	291	282	282
query28	3722	2077	2057	2057
query29	654	443	433	433
query30	239	155	146	146
query31	937	787	758	758
query32	84	56	59	56
query33	471	292	296	292
query34	872	484	482	482
query35	847	706	730	706
query36	1079	928	959	928
query37	144	81	83	81
query38	3911	3888	3930	3888
query39	1463	1383	1402	1383
query40	201	121	120	120
query41	48	47	46	46
query42	116	100	100	100
query43	505	472	479	472
query44	1134	766	765	765
query45	202	166	165	165
query46	1098	752	745	745
query47	1888	1796	1795	1795
query48	393	308	299	299
query49	786	432	447	432
query50	825	441	430	430
query51	7214	7073	7010	7010
query52	103	91	88	88
query53	261	185	189	185
query54	612	462	467	462
query55	79	77	78	77
query56	294	269	260	260
query57	1204	1052	1058	1052
query58	232	260	250	250
query59	2932	2783	2908	2783
query60	292	279	271	271
query61	104	102	97	97
query62	803	646	630	630
query63	222	191	198	191
query64	3969	2270	1796	1796
query65	3227	3148	3171	3148
query66	665	333	331	331
query67	15291	15282	15280	15280
query68	4336	577	571	571
query69	430	275	276	275
query70	1215	1140	1128	1128
query71	385	303	283	283
query72	2554	2107	2089	2089
query73	729	327	331	327
query74	9239	8776	8854	8776
query75	3386	2724	2767	2724
query76	1881	985	948	948
query77	560	318	319	318
query78	9786	9044	9030	9030
query79	1055	544	551	544
query80	735	514	536	514
query81	547	236	225	225
query82	278	137	137	137
query83	173	148	148	148
query84	267	81	78	78
query85	777	300	285	285
query86	318	309	295	295
query87	4471	4226	4330	4226
query88	3056	2407	2414	2407
query89	394	281	283	281
query90	2005	197	196	196
query91	123	104	98	98
query92	63	50	54	50
query93	1103	541	550	541
query94	750	305	299	299
query95	360	265	268	265
query96	607	275	272	272
query97	3217	3109	3138	3109
query98	211	204	201	201
query99	1550	1273	1296	1273
Total cold run time: 305280 ms
Total hot run time: 192396 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.24 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 29ad5784ffb5edc50405763b019aebc36cfa925a, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.05	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.07
query5	0.50	0.50	0.51
query6	1.12	0.74	0.73
query7	0.02	0.01	0.01
query8	0.05	0.05	0.05
query9	0.54	0.49	0.49
query10	0.54	0.54	0.54
query11	0.17	0.12	0.11
query12	0.15	0.12	0.13
query13	0.61	0.59	0.60
query14	0.76	0.82	0.78
query15	0.85	0.82	0.83
query16	0.37	0.39	0.36
query17	1.06	1.08	1.06
query18	0.22	0.21	0.21
query19	1.88	1.76	1.74
query20	0.01	0.01	0.00
query21	15.39	0.67	0.67
query22	4.25	6.91	2.07
query23	18.55	1.42	1.33
query24	2.18	0.23	0.25
query25	0.16	0.08	0.08
query26	0.28	0.18	0.18
query27	0.07	0.07	0.08
query28	13.24	1.04	1.02
query29	12.60	3.38	3.34
query30	0.25	0.06	0.05
query31	2.89	0.40	0.40
query32	3.23	0.49	0.48
query33	3.00	3.00	3.08
query34	17.78	4.44	4.51
query35	4.59	4.49	4.52
query36	0.66	0.48	0.49
query37	0.20	0.16	0.15
query38	0.15	0.15	0.15
query39	0.05	0.04	0.03
query40	0.16	0.13	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.05
Total cold run time: 110.77 s
Total hot run time: 31.24 s

@dataroaring dataroaring merged commit 116130e into apache:master Aug 25, 2024
29 of 32 checks passed
dataroaring added a commit to dataroaring/incubator-doris that referenced this pull request Aug 25, 2024
@dataroaring
[improvemen](overflow) Provide the user with a suggestion to avoid th… (
bcb0ae1 
apache#39631)
dataroaring added a commit to dataroaring/incubator-doris that referenced this pull request Aug 25, 2024
@dataroaring
[improvemen](overflow) Provide the user with a suggestion to avoid th… (
6e6c598 
apache#39631)
@dataroaring dataroaring mentioned this pull request Aug 25, 2024
Merged
yiguolei pushed a commit that referenced this pull request Aug 26, 2024
@dataroaring
[improvemen](overflow) Provide the user with a suggestion to avoid th… (
4c1c67e 
#39631) (#39897)

cherry-pick #39631 to branch-2.1
dataroaring added a commit that referenced this pull request Oct 9, 2024
@dataroaring
[improvemen](overflow) Provide the user with a suggestion to avoid th… (
c5c7d76 
#39631)
@gavinchou gavinchou mentioned this pull request Nov 26, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yiguolei yiguolei yiguolei approved these changes

@gavinchou gavinchou gavinchou approved these changes

@morningman morningman morningman approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@dataroaring @doris-robot @morningman @gavinchou @yiguolei