Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](nestedtype)fix nested type with is_exclusive #40398

Merged
merged 3 commits into from
Sep 5, 2024

Conversation

amorynan
Copy link
Contributor

@amorynan amorynan commented Sep 5, 2024

Proposed changes

is_exclusive in column_array/map/struct has wrong semantic , we should make sure it's nested column is also is_exclusive which can make behavior right in some operator like join.
Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@amorynan
Copy link
Contributor Author

amorynan commented Sep 5, 2024

run buildall

Copy link
Contributor

github-actions bot commented Sep 5, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

  • regression-test/suites/datatype_p0/nested_types/query/sql/nested_with_join.sql

Consider using git-lfs to manage large files.

@github-actions github-actions bot added the lfs-detected! Warning Label for use when LFS is detected in the commits of a Pull Request label Sep 5, 2024
Copy link
Contributor

github-actions bot commented Sep 5, 2024

clang-tidy review says "All clean, LGTM! 👍"

Gabriel39
Gabriel39 previously approved these changes Sep 5, 2024
Copy link
Contributor

github-actions bot commented Sep 5, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

  • regression-test/suites/datatype_p0/nested_types/query/sql/nested_with_join.sql

Consider using git-lfs to manage large files.

Copy link
Contributor

github-actions bot commented Sep 5, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 5, 2024
Copy link
Contributor

github-actions bot commented Sep 5, 2024

PR approved by anyone and no changes requested.

@amorynan
Copy link
Contributor Author

amorynan commented Sep 5, 2024

run buildall

@amorynan amorynan requested a review from eldenmoon September 5, 2024 02:57
Copy link
Contributor

github-actions bot commented Sep 5, 2024

clang-tidy review says "All clean, LGTM! 👍"

Copy link
Contributor

github-actions bot commented Sep 5, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

  • regression-test/suites/datatype_p0/nested_types/query/sql/nested_with_join.sql

Consider using git-lfs to manage large files.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Sep 5, 2024
@amorynan
Copy link
Contributor Author

amorynan commented Sep 5, 2024

run buildall

Copy link
Contributor

github-actions bot commented Sep 5, 2024

clang-tidy review says "All clean, LGTM! 👍"

Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

github-actions bot commented Sep 5, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 5, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 38388 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 271340573c1a2565363112b2bf0e5cf112a783ae, data reload: false

------ Round 1 ----------------------------------
q1	18424	4553	4390	4390
q2	3000	183	180	180
q3	10833	1240	1147	1147
q4	10479	788	756	756
q5	8124	2901	2875	2875
q6	227	140	139	139
q7	962	636	598	598
q8	9311	2075	2038	2038
q9	7201	6574	6527	6527
q10	7007	2255	2205	2205
q11	478	241	245	241
q12	390	218	220	218
q13	18876	3061	3071	3061
q14	271	235	247	235
q15	533	499	481	481
q16	584	502	511	502
q17	975	741	657	657
q18	7433	6892	6832	6832
q19	1390	1014	1072	1014
q20	697	339	329	329
q21	3953	2926	2941	2926
q22	1128	1038	1037	1037
Total cold run time: 112276 ms
Total hot run time: 38388 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4332	4305	4286	4286
q2	388	267	266	266
q3	2908	2673	2712	2673
q4	1938	1651	1612	1612
q5	5405	5393	5386	5386
q6	223	128	128	128
q7	2127	1746	1771	1746
q8	3199	3385	3332	3332
q9	8403	8415	8389	8389
q10	3457	3201	3161	3161
q11	585	501	507	501
q12	765	609	626	609
q13	10870	3063	3065	3063
q14	303	281	282	281
q15	536	486	477	477
q16	612	562	544	544
q17	1805	1510	1480	1480
q18	7788	7487	7438	7438
q19	1666	1581	1607	1581
q20	2060	1820	1841	1820
q21	5475	5097	5213	5097
q22	1118	1070	1025	1025
Total cold run time: 65963 ms
Total hot run time: 54895 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.83% (9393/25502)
Line Coverage: 28.27% (77448/273946)
Region Coverage: 27.67% (39966/144460)
Branch Coverage: 24.31% (20339/83666)
Coverage Report: http://coverage.selectdb-in.cc/coverage/271340573c1a2565363112b2bf0e5cf112a783ae_271340573c1a2565363112b2bf0e5cf112a783ae/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 188383 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 271340573c1a2565363112b2bf0e5cf112a783ae, data reload: false

query1	915	372	358	358
query2	6480	1935	1931	1931
query3	6651	203	223	203
query4	31376	23070	23078	23070
query5	4188	485	490	485
query6	261	171	178	171
query7	4591	285	294	285
query8	243	203	201	201
query9	8568	2509	2502	2502
query10	435	270	261	261
query11	17936	15029	14996	14996
query12	146	97	97	97
query13	1626	392	372	372
query14	9846	7195	7291	7195
query15	240	170	162	162
query16	8061	429	446	429
query17	1569	566	547	547
query18	2110	282	284	282
query19	191	141	138	138
query20	115	105	109	105
query21	212	102	101	101
query22	4544	4350	4272	4272
query23	34032	33503	33595	33503
query24	11230	2876	2783	2783
query25	636	415	403	403
query26	1206	159	160	159
query27	2360	283	286	283
query28	7202	2075	2075	2075
query29	844	438	419	419
query30	307	162	154	154
query31	986	767	779	767
query32	96	57	56	56
query33	769	295	293	293
query34	985	486	485	485
query35	852	741	731	731
query36	1109	941	950	941
query37	167	105	94	94
query38	3906	3877	3792	3792
query39	1430	1424	1393	1393
query40	205	117	117	117
query41	52	48	49	48
query42	119	95	100	95
query43	492	471	482	471
query44	1170	758	744	744
query45	202	169	177	169
query46	1105	766	748	748
query47	1925	1856	1845	1845
query48	390	303	307	303
query49	1111	444	459	444
query50	809	417	403	403
query51	7045	7014	6946	6946
query52	99	88	86	86
query53	256	188	180	180
query54	1028	469	479	469
query55	77	75	74	74
query56	310	274	267	267
query57	1207	1099	1086	1086
query58	252	235	234	234
query59	2903	2699	2870	2699
query60	308	280	272	272
query61	121	118	119	118
query62	847	671	653	653
query63	223	181	187	181
query64	5447	757	741	741
query65	3289	3151	3179	3151
query66	1443	343	359	343
query67	15834	15390	15672	15390
query68	4602	596	562	562
query69	419	287	275	275
query70	1109	1139	1164	1139
query71	341	271	287	271
query72	6746	3965	4028	3965
query73	743	329	329	329
query74	9126	8711	8824	8711
query75	3430	2730	2657	2657
query76	2456	1067	1050	1050
query77	512	306	313	306
query78	10311	9141	9141	9141
query79	2351	542	554	542
query80	1029	507	501	501
query81	586	235	232	232
query82	607	143	152	143
query83	232	154	150	150
query84	225	79	79	79
query85	1524	296	317	296
query86	483	301	269	269
query87	4373	4150	4194	4150
query88	3667	2382	2357	2357
query89	385	285	286	285
query90	1835	184	182	182
query91	121	99	100	99
query92	61	49	47	47
query93	2071	549	549	549
query94	888	287	279	279
query95	351	251	250	250
query96	591	271	261	261
query97	3279	3064	3080	3064
query98	222	205	202	202
query99	1668	1269	1286	1269
Total cold run time: 293034 ms
Total hot run time: 188383 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 271340573c1a2565363112b2bf0e5cf112a783ae, data reload: false

query1	0.05	0.05	0.04
query2	0.08	0.04	0.03
query3	0.22	0.05	0.05
query4	1.68	0.08	0.08
query5	0.51	0.50	0.48
query6	1.13	0.73	0.73
query7	0.02	0.01	0.02
query8	0.06	0.04	0.05
query9	0.54	0.49	0.48
query10	0.53	0.53	0.55
query11	0.15	0.11	0.11
query12	0.16	0.13	0.12
query13	0.60	0.59	0.60
query14	2.10	2.06	2.10
query15	0.90	0.83	0.82
query16	0.35	0.38	0.37
query17	1.02	1.05	1.03
query18	0.21	0.20	0.20
query19	1.83	1.72	1.77
query20	0.01	0.01	0.01
query21	15.40	0.66	0.65
query22	4.33	6.90	2.20
query23	18.28	1.35	1.33
query24	2.15	0.22	0.21
query25	0.14	0.07	0.08
query26	0.28	0.19	0.18
query27	0.07	0.08	0.07
query28	13.21	1.01	1.00
query29	12.59	3.31	3.27
query30	0.24	0.07	0.05
query31	2.87	0.42	0.40
query32	3.24	0.51	0.48
query33	2.99	3.00	3.07
query34	17.12	4.42	4.38
query35	4.50	4.44	4.43
query36	0.66	0.47	0.46
query37	0.19	0.16	0.16
query38	0.15	0.14	0.14
query39	0.05	0.04	0.04
query40	0.16	0.12	0.12
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.05
Total cold run time: 110.98 s
Total hot run time: 32.28 s

@eldenmoon eldenmoon merged commit 32557f8 into apache:master Sep 5, 2024
23 of 29 checks passed
amorynan added a commit to amorynan/doris that referenced this pull request Sep 5, 2024
is_exclusive in column_array/map/struct has wrong semantic , we should
make sure it's nested column is also is_exclusive which can make
behavior right in some operator like join.
yiguolei pushed a commit to amorynan/doris that referenced this pull request Sep 10, 2024
is_exclusive in column_array/map/struct has wrong semantic , we should
make sure it's nested column is also is_exclusive which can make
behavior right in some operator like join.
yiguolei pushed a commit to amorynan/doris that referenced this pull request Sep 15, 2024
is_exclusive in column_array/map/struct has wrong semantic , we should
make sure it's nested column is also is_exclusive which can make
behavior right in some operator like join.
yiguolei pushed a commit that referenced this pull request Sep 19, 2024
is_exclusive in column_array/map/struct has wrong semantic , we should
make sure it's nested column is also is_exclusive which can make
behavior right in some operator like join.

## Proposed changes
backport: #40398
Issue Number: close #xxx

<!--Describe your changes.-->
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
is_exclusive in column_array/map/struct has wrong semantic , we should
make sure it's nested column is also is_exclusive which can make
behavior right in some operator like join.
amorynan added a commit to amorynan/doris that referenced this pull request Oct 10, 2024
is_exclusive in column_array/map/struct has wrong semantic , we should
make sure it's nested column is also is_exclusive which can make
behavior right in some operator like join.
eldenmoon pushed a commit that referenced this pull request Oct 15, 2024
## Proposed changes
backport: #40398
Issue Number: close #xxx

<!--Describe your changes.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.16-merged dev/2.1.7-merged dev/3.0.3-merged lfs-detected! Warning Label for use when LFS is detected in the commits of a Pull Request reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants