Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](function) Refine crypto functions signature to fix wrong result #40285

Merged
merged 2 commits into from
Sep 4, 2024
Merged

[Fix](function) Refine crypto functions signature to fix wrong result #40285

merged 2 commits into from
Sep 4, 2024

Conversation

zclllyybb
Copy link
Contributor

@zclllyybb zclllyybb commented Sep 2, 2024 •
edited
Loading

Proposed changes

Issue Number: close #xxx

  1. remove all aes/sm4_{encrypt/decrypt}_v2 functions signatures.
  2. before for 2 args, always use XXX_128_ECB, now fixed it. we will use block_encryption_mode when 2/3 args are given.
    2.1. before this, we wrongly used AES_128_ECB for SM4_ENCRYPT
  3. for AES_ENCRYPT with 2 args, we wrongly use 'AES_128_ECB' as its default initial string. now fixed it.

docs: apache/doris-website#1083

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37867 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e0911e50cdd0192168ebf51081c967c2d73af34, data reload: false

------ Round 1 ----------------------------------
q1	17927	4455	4304	4304
q2	2041	179	191	179
q3	11643	935	1121	935
q4	10520	766	709	709
q5	7744	2839	2763	2763
q6	232	140	140	140
q7	953	616	608	608
q8	9346	2078	2072	2072
q9	6968	6518	6569	6518
q10	7007	2190	2152	2152
q11	491	249	241	241
q12	394	223	229	223
q13	17784	3111	3056	3056
q14	291	226	239	226
q15	520	488	482	482
q16	599	506	506	506
q17	977	658	732	658
q18	7299	6835	6805	6805
q19	1385	994	1105	994
q20	670	337	326	326
q21	3957	3127	2979	2979
q22	1111	991	991	991
Total cold run time: 109859 ms
Total hot run time: 37867 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4384	4318	4316	4316
q2	378	279	269	269
q3	2840	2684	2647	2647
q4	1952	1634	1607	1607
q5	5579	5698	5676	5676
q6	230	137	152	137
q7	2244	1820	1810	1810
q8	3300	3404	3427	3404
q9	8880	8838	8844	8838
q10	3561	3395	3340	3340
q11	624	513	518	513
q12	845	668	660	660
q13	13512	3198	3300	3198
q14	323	289	276	276
q15	562	480	499	480
q16	650	569	572	569
q17	1833	1550	1546	1546
q18	8183	7742	7732	7732
q19	1757	1525	1533	1525
q20	2145	1922	1914	1914
q21	5843	5490	5310	5310
q22	1159	1031	996	996
Total cold run time: 70784 ms
Total hot run time: 56763 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192218 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7e0911e50cdd0192168ebf51081c967c2d73af34, data reload: false

query1	1250	889	871	871
query2	6434	2050	2029	2029
query3	10605	3935	3946	3935
query4	58578	24946	23320	23320
query5	5297	499	486	486
query6	415	166	186	166
query7	5741	293	298	293
query8	284	203	201	201
query9	8719	2487	2448	2448
query10	467	278	262	262
query11	17949	15073	15301	15073
query12	157	100	103	100
query13	1561	400	390	390
query14	10258	7289	7324	7289
query15	229	170	185	170
query16	7242	489	435	435
query17	1123	560	585	560
query18	1977	288	314	288
query19	281	147	144	144
query20	122	115	114	114
query21	208	106	105	105
query22	4555	4483	4282	4282
query23	34327	33606	33262	33262
query24	5938	2889	2879	2879
query25	535	404	401	401
query26	670	157	158	157
query27	1733	286	279	279
query28	3786	2058	2062	2058
query29	693	420	426	420
query30	253	153	159	153
query31	969	769	789	769
query32	74	54	59	54
query33	461	300	292	292
query34	869	467	481	467
query35	834	742	708	708
query36	1060	941	962	941
query37	145	90	91	90
query38	4002	3882	3941	3882
query39	1506	1419	1381	1381
query40	198	117	115	115
query41	51	47	46	46
query42	120	95	97	95
query43	522	484	464	464
query44	1094	745	741	741
query45	201	170	177	170
query46	1086	735	735	735
query47	1887	1795	1807	1795
query48	373	304	304	304
query49	778	443	448	443
query50	842	427	422	422
query51	6982	6820	6812	6812
query52	102	86	89	86
query53	252	178	180	178
query54	584	461	467	461
query55	78	76	75	75
query56	284	257	275	257
query57	1306	1064	1081	1064
query58	217	241	231	231
query59	3141	2857	2731	2731
query60	289	266	265	265
query61	106	119	95	95
query62	746	632	657	632
query63	218	188	187	187
query64	2390	681	646	646
query65	3192	3112	3152	3112
query66	681	347	332	332
query67	15548	15193	15241	15193
query68	3706	578	572	572
query69	408	278	266	266
query70	1178	1079	1115	1079
query71	355	273	279	273
query72	6316	3764	3975	3764
query73	746	330	337	330
query74	9199	8733	8829	8733
query75	3378	2726	2682	2682
query76	1547	1025	938	938
query77	519	323	329	323
query78	9828	9240	9055	9055
query79	1024	540	535	535
query80	707	518	519	518
query81	537	239	232	232
query82	283	144	145	144
query83	174	149	147	147
query84	260	79	73	73
query85	746	283	273	273
query86	325	306	292	292
query87	4424	4399	4351	4351
query88	3559	2318	2318	2318
query89	420	280	280	280
query90	1921	189	189	189
query91	128	102	103	102
query92	65	50	51	50
query93	1073	535	537	535
query94	702	294	297	294
query95	339	255	253	253
query96	583	269	268	268
query97	3192	3058	3056	3056
query98	222	198	192	192
query99	1516	1288	1260	1260
Total cold run time: 303955 ms
Total hot run time: 192218 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.95 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e0911e50cdd0192168ebf51081c967c2d73af34, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.04	0.04
query3	0.23	0.06	0.05
query4	1.67	0.09	0.10
query5	0.50	0.48	0.49
query6	1.14	0.74	0.73
query7	0.02	0.01	0.02
query8	0.04	0.04	0.05
query9	0.55	0.48	0.50
query10	0.52	0.54	0.53
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.61	0.59	0.59
query14	2.03	2.06	2.11
query15	0.86	0.83	0.83
query16	0.38	0.38	0.38
query17	1.00	1.05	1.06
query18	0.21	0.19	0.20
query19	1.98	1.82	1.80
query20	0.01	0.01	0.01
query21	15.41	0.66	0.66
query22	4.43	7.34	1.81
query23	18.28	1.37	1.31
query24	2.05	0.22	0.22
query25	0.15	0.09	0.08
query26	0.27	0.18	0.17
query27	0.08	0.08	0.07
query28	13.32	1.01	1.00
query29	12.61	3.31	3.29
query30	0.23	0.07	0.06
query31	2.88	0.40	0.40
query32	3.24	0.48	0.47
query33	2.95	3.01	3.03
query34	17.07	4.38	4.35
query35	4.49	4.43	4.39
query36	0.66	0.48	0.46
query37	0.19	0.16	0.15
query38	0.16	0.14	0.15
query39	0.05	0.04	0.03
query40	0.16	0.13	0.13
query41	0.09	0.04	0.04
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 111.07 s
Total hot run time: 31.95 s

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38444 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 52a087925fbfd635a510485efca4182b2eb67a52, data reload: false

------ Round 1 ----------------------------------
q1	17642	4912	4368	4368
q2	2013	180	178	178
q3	10460	1212	1141	1141
q4	10134	769	735	735
q5	7761	2903	2839	2839
q6	233	134	134	134
q7	971	607	593	593
q8	9329	2076	2077	2076
q9	7241	6548	6550	6548
q10	7030	2170	2278	2170
q11	452	246	250	246
q12	392	217	218	217
q13	17770	3076	3064	3064
q14	288	240	235	235
q15	531	483	488	483
q16	569	528	508	508
q17	982	687	669	669
q18	7579	6929	6978	6929
q19	1383	1044	1076	1044
q20	669	336	331	331
q21	3959	2998	2934	2934
q22	1122	1002	1026	1002
Total cold run time: 108510 ms
Total hot run time: 38444 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4369	4303	4315	4303
q2	387	281	271	271
q3	2842	2658	2626	2626
q4	1960	1645	1631	1631
q5	5410	5406	5435	5406
q6	219	129	127	127
q7	2103	1731	1760	1731
q8	3212	3373	3318	3318
q9	8425	8452	8422	8422
q10	3445	3211	3179	3179
q11	595	506	488	488
q12	820	603	622	603
q13	13766	3078	3089	3078
q14	305	269	276	269
q15	519	480	492	480
q16	590	585	570	570
q17	1794	1507	1472	1472
q18	7727	7599	7531	7531
q19	1669	1655	1534	1534
q20	2056	1813	1842	1813
q21	5307	5233	5248	5233
q22	1148	1034	1048	1034
Total cold run time: 68668 ms
Total hot run time: 55119 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187442 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 52a087925fbfd635a510485efca4182b2eb67a52, data reload: false

query1	920	377	374	374
query2	6463	1965	1881	1881
query3	6648	202	213	202
query4	34164	23246	23034	23034
query5	4157	521	481	481
query6	258	169	174	169
query7	4585	294	278	278
query8	262	204	197	197
query9	8485	2442	2450	2442
query10	435	271	267	267
query11	17916	14980	15045	14980
query12	147	98	99	98
query13	1635	369	353	353
query14	9497	7252	7437	7252
query15	289	173	171	171
query16	7831	454	472	454
query17	1602	577	569	569
query18	1599	296	315	296
query19	275	144	142	142
query20	124	110	112	110
query21	212	102	101	101
query22	4562	4093	4112	4093
query23	33994	33671	33240	33240
query24	10994	2857	2780	2780
query25	660	386	383	383
query26	1127	154	156	154
query27	2710	280	270	270
query28	7028	2045	2028	2028
query29	740	414	396	396
query30	301	156	151	151
query31	959	746	770	746
query32	103	56	54	54
query33	769	295	287	287
query34	972	473	476	473
query35	848	741	712	712
query36	1080	914	916	914
query37	167	94	96	94
query38	3952	3832	3886	3832
query39	1469	1410	1438	1410
query40	200	116	115	115
query41	47	46	44	44
query42	118	94	99	94
query43	493	471	466	466
query44	1228	735	728	728
query45	194	166	165	165
query46	1100	812	747	747
query47	1934	1795	1845	1795
query48	363	291	289	289
query49	1099	432	422	422
query50	811	413	397	397
query51	7008	6865	6937	6865
query52	100	85	84	84
query53	247	181	182	181
query54	957	466	459	459
query55	77	73	74	73
query56	288	278	260	260
query57	1209	1092	1125	1092
query58	255	239	230	230
query59	2907	2838	2835	2835
query60	292	268	270	268
query61	105	100	101	100
query62	830	658	653	653
query63	217	181	175	175
query64	5045	682	735	682
query65	3218	3154	3131	3131
query66	1296	333	327	327
query67	15550	15344	15608	15344
query68	3541	563	553	553
query69	398	275	275	275
query70	1209	1045	1073	1045
query71	330	273	275	273
query72	6410	4087	4104	4087
query73	761	325	329	325
query74	9254	8764	8880	8764
query75	3370	2690	2700	2690
query76	2263	952	1007	952
query77	481	311	313	311
query78	9660	9122	10299	9122
query79	993	542	519	519
query80	671	512	519	512
query81	448	245	239	239
query82	245	151	151	151
query83	182	157	160	157
query84	229	82	78	78
query85	743	287	287	287
query86	297	288	269	269
query87	4433	4253	4162	4162
query88	3007	2336	2325	2325
query89	366	285	293	285
query90	1809	193	189	189
query91	125	100	98	98
query92	61	49	50	49
query93	1025	532	521	521
query94	685	259	301	259
query95	325	252	254	252
query96	586	262	263	262
query97	3188	3116	3104	3104
query98	220	206	199	199
query99	1463	1258	1239	1239
Total cold run time: 285905 ms
Total hot run time: 187442 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.96 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 52a087925fbfd635a510485efca4182b2eb67a52, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.08	0.07
query5	0.51	0.49	0.51
query6	1.12	0.74	0.72
query7	0.02	0.01	0.01
query8	0.06	0.04	0.04
query9	0.54	0.49	0.48
query10	0.56	0.54	0.54
query11	0.15	0.12	0.12
query12	0.16	0.12	0.12
query13	0.60	0.59	0.59
query14	2.04	2.06	2.09
query15	0.89	0.82	0.82
query16	0.38	0.37	0.38
query17	0.99	0.99	0.99
query18	0.22	0.21	0.21
query19	1.95	1.78	1.81
query20	0.01	0.01	0.01
query21	15.42	0.72	0.69
query22	4.03	7.48	1.90
query23	18.26	1.43	1.38
query24	2.19	0.23	0.22
query25	0.16	0.08	0.08
query26	0.27	0.19	0.18
query27	0.07	0.07	0.07
query28	13.13	1.02	1.00
query29	12.64	3.30	3.26
query30	0.25	0.06	0.05
query31	2.89	0.39	0.41
query32	3.24	0.47	0.47
query33	2.98	2.95	3.03
query34	17.14	4.38	4.43
query35	4.45	4.43	4.34
query36	0.65	0.47	0.46
query37	0.18	0.16	0.16
query38	0.15	0.14	0.15
query39	0.05	0.04	0.04
query40	0.17	0.13	0.12
query41	0.09	0.05	0.05
query42	0.07	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 110.76 s
Total hot run time: 31.96 s

HappenLee
HappenLee approved these changes Sep 4, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 4, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 4, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 4, 2024

PR approved by anyone and no changes requested.

@zhangstar333 zhangstar333 merged commit 08a3e56 into apache:master Sep 4, 2024
27 of 29 checks passed
@zclllyybb zclllyybb deleted the crypto branch September 4, 2024 05:03
@cambyzju cambyzju mentioned this pull request Sep 5, 2024
Merged
zclllyybb added a commit to zclllyybb/doris that referenced this pull request Sep 11, 2024
@zclllyybb
[Fix](function) Refine crypto functions signature to fix wrong result (…
f604a09 
…apache#40285)

## Proposed changes

Issue Number: close #xxx

1. remove all `aes/sm4_{encrypt/decrypt}_v2` functions signatures.
2. before for 2 args, always use `XXX_128_ECB`, now fixed it. we will
use `block_encryption_mode` when 2/3 args are given.
2.1. before this, we wrongly used `AES_128_ECB` for SM4_ENCRYPT
4. for `AES_ENCRYPT` with 2 args, we wrongly use `'AES_128_ECB'` as its
default initial string. now fixed it.

docs: apache/doris-website#1083
@zclllyybb zclllyybb mentioned this pull request Sep 11, 2024
Merged
yiguolei pushed a commit that referenced this pull request Sep 11, 2024
@zclllyybb
[branch-2.1](function) Refine crypto functions signature to fix wrong…
3246baa 
… result(#40285) (#40648)

pick #40285
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
@zclllyybb @dataroaring
[Fix](function) Refine crypto functions signature to fix wrong result (…
c09eb5a 
…#40285)

## Proposed changes

Issue Number: close #xxx

1. remove all `aes/sm4_{encrypt/decrypt}_v2` functions signatures.
2. before for 2 args, always use `XXX_128_ECB`, now fixed it. we will
use `block_encryption_mode` when 2/3 args are given.
2.1. before this, we wrongly used `AES_128_ECB` for SM4_ENCRYPT
4. for `AES_ENCRYPT` with 2 args, we wrongly use `'AES_128_ECB'` as its
default initial string. now fixed it.

docs: apache/doris-website#1083
@gavinchou gavinchou mentioned this pull request Nov 26, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhangstar333 zhangstar333 zhangstar333 approved these changes

@HappenLee HappenLee HappenLee approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.3-merged kind/behavior-changed reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@zclllyybb @doris-robot @HappenLee @zhangstar333 @yiguolei @dataroaring