Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improve](statistics)Record index row count update time. #39788

Merged
merged 1 commit into from
Aug 23, 2024
Merged

[improve](statistics)Record index row count update time. #39788

merged 1 commit into from
Aug 23, 2024

Conversation

Jibing-Li
Copy link
Contributor

Record index row count update time. So the planner could use it to decide which row count to use, reported row count or analyzed row count.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Jibing-Li Jibing-Li marked this pull request as ready for review August 22, 2024 10:02
@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38290 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7af9993ba09876042ce9d390614fbccb0c83fe57, data reload: false

------ Round 1 ----------------------------------
q1	18506	4599	4490	4490
q2	2845	185	177	177
q3	11087	1124	1179	1124
q4	10534	750	762	750
q5	7781	2887	2873	2873
q6	232	146	140	140
q7	982	639	628	628
q8	9668	2064	2068	2064
q9	7145	6539	6570	6539
q10	7021	2176	2280	2176
q11	468	240	248	240
q12	398	228	223	223
q13	18250	3018	3021	3018
q14	283	233	237	233
q15	511	505	499	499
q16	494	393	389	389
q17	975	701	671	671
q18	7374	6823	6787	6787
q19	1385	1012	1069	1012
q20	683	333	340	333
q21	4093	2942	3113	2942
q22	1122	982	1018	982
Total cold run time: 111837 ms
Total hot run time: 38290 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4427	4362	4354	4354
q2	372	270	278	270
q3	2886	2662	2634	2634
q4	1972	1654	1678	1654
q5	5420	5397	5394	5394
q6	218	130	130	130
q7	2135	1751	1706	1706
q8	3188	3379	3368	3368
q9	8494	8450	8490	8450
q10	3445	3200	3195	3195
q11	591	508	491	491
q12	789	606	602	602
q13	10541	3046	3038	3038
q14	316	283	280	280
q15	529	470	465	465
q16	474	446	430	430
q17	1774	1500	1486	1486
q18	7800	7414	7583	7414
q19	1673	1583	1630	1583
q20	2060	1819	1835	1819
q21	5474	5280	5169	5169
q22	1105	1021	1025	1021
Total cold run time: 65683 ms
Total hot run time: 54953 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186544 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7af9993ba09876042ce9d390614fbccb0c83fe57, data reload: false

query1	930	376	374	374
query2	6448	1880	1845	1845
query3	6657	209	214	209
query4	33439	23216	23154	23154
query5	4146	522	484	484
query6	262	165	160	160
query7	4588	295	293	293
query8	258	202	205	202
query9	8575	2440	2430	2430
query10	435	283	273	273
query11	16421	15178	15091	15091
query12	144	98	100	98
query13	1636	385	377	377
query14	9375	6851	6397	6397
query15	235	183	169	169
query16	7818	447	446	446
query17	1604	578	579	578
query18	2085	290	280	280
query19	191	145	140	140
query20	116	108	106	106
query21	210	102	101	101
query22	4466	4229	4164	4164
query23	34225	33625	33453	33453
query24	11216	2910	2917	2910
query25	621	393	391	391
query26	1100	157	155	155
query27	2381	285	282	282
query28	7193	2027	2004	2004
query29	793	431	398	398
query30	291	161	153	153
query31	975	777	787	777
query32	102	55	60	55
query33	761	294	286	286
query34	990	469	493	469
query35	833	744	715	715
query36	1093	892	935	892
query37	156	85	87	85
query38	4067	3922	3881	3881
query39	1424	1415	1403	1403
query40	211	118	117	117
query41	48	48	45	45
query42	117	97	96	96
query43	511	452	465	452
query44	1231	740	760	740
query45	195	171	170	170
query46	1117	753	768	753
query47	1918	1781	1790	1781
query48	389	299	293	293
query49	1110	443	440	440
query50	816	426	411	411
query51	7280	7081	6992	6992
query52	101	94	88	88
query53	255	180	180	180
query54	985	467	500	467
query55	77	73	76	73
query56	273	258	250	250
query57	1221	1039	1051	1039
query58	246	227	240	227
query59	3017	2797	2737	2737
query60	303	266	271	266
query61	106	100	98	98
query62	831	619	672	619
query63	213	183	185	183
query64	5553	2280	1729	1729
query65	3232	3203	3160	3160
query66	1392	346	342	342
query67	15581	15358	15280	15280
query68	3559	589	576	576
query69	401	278	289	278
query70	1170	1128	1094	1094
query71	336	278	279	278
query72	6286	2321	2107	2107
query73	743	323	329	323
query74	9288	8820	8917	8820
query75	3433	2660	2761	2660
query76	1883	1007	1032	1007
query77	491	319	310	310
query78	9640	9080	9309	9080
query79	1034	539	536	536
query80	722	511	495	495
query81	449	236	234	234
query82	238	141	137	137
query83	173	156	156	156
query84	240	80	83	80
query85	688	295	300	295
query86	317	300	305	300
query87	4433	4272	4159	4159
query88	3059	2383	2367	2367
query89	384	295	285	285
query90	1900	275	199	199
query91	125	99	100	99
query92	58	53	50	50
query93	1031	553	535	535
query94	726	281	301	281
query95	355	252	258	252
query96	588	272	266	266
query97	3224	3088	3107	3088
query98	218	213	208	208
query99	1473	1281	1302	1281
Total cold run time: 285254 ms
Total hot run time: 186544 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.9 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7af9993ba09876042ce9d390614fbccb0c83fe57, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.69	0.08	0.07
query5	0.52	0.50	0.49
query6	1.12	0.73	0.72
query7	0.02	0.02	0.01
query8	0.06	0.05	0.05
query9	0.55	0.48	0.48
query10	0.56	0.55	0.53
query11	0.15	0.11	0.12
query12	0.15	0.11	0.12
query13	0.61	0.58	0.58
query14	0.76	0.78	0.79
query15	0.87	0.82	0.81
query16	0.37	0.38	0.38
query17	1.07	1.01	0.98
query18	0.23	0.21	0.22
query19	1.79	1.74	1.78
query20	0.02	0.01	0.01
query21	15.39	0.67	0.67
query22	4.31	6.98	2.11
query23	18.28	1.39	1.30
query24	2.09	0.23	0.22
query25	0.14	0.09	0.09
query26	0.29	0.18	0.17
query27	0.08	0.07	0.08
query28	13.29	1.02	0.99
query29	12.65	3.35	3.34
query30	0.24	0.06	0.05
query31	2.87	0.40	0.38
query32	3.29	0.48	0.48
query33	2.95	2.99	3.00
query34	16.92	4.39	4.35
query35	4.39	4.42	4.47
query36	0.66	0.47	0.49
query37	0.19	0.17	0.16
query38	0.16	0.16	0.15
query39	0.05	0.05	0.04
query40	0.17	0.12	0.12
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.51 s
Total hot run time: 30.9 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 23, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@Jibing-Li Jibing-Li merged commit d71379c into apache:master Aug 23, 2024
28 of 30 checks passed
@Jibing-Li Jibing-Li deleted the rowtime branch August 23, 2024 04:15
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
@Jibing-Li @dataroaring
[improve](statistics)Record index row count update time. (#39788)
29488bd 
Record index row count update time. So the planner could use it to
decide which row count to use, reported row count or analyzed row count.
@gavinchou gavinchou mentioned this pull request Nov 26, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow morrySnow approved these changes

@englefly englefly englefly approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Jibing-Li @doris-robot @englefly @morrySnow