Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhance](auth)The priority of attributes is higher than session variable #47185

Merged
merged 15 commits into from
Feb 13, 2025

Conversation

zddr
Copy link
Contributor

@zddr zddr commented Jan 17, 2025

What problem does this PR solve?

The current priority is that session variable is higher than user property, which is incorrect because session variable can be set freely by ordinary users and will bypass the restrictions set by administrators

Scope of Impact:

  • getInsertTimeoutS
  • getQueryTimeoutS
  • getMaxExecMemByte

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zddr
Copy link
Contributor Author

zddr commented Jan 17, 2025

run buildall

@morrySnow morrySnow added the usercase Important user case type label label Jan 17, 2025
Copy link
Contributor

@morrySnow morrySnow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add test cases

@doris-robot
Copy link

TPC-H: Total hot run time: 32245 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 359fa20ef11c20bcce6e03698b6b3235cd2f6a4d, data reload: false

------ Round 1 ----------------------------------
q1	17588	5448	5354	5354
q2	2045	302	189	189
q3	10408	1267	724	724
q4	10222	974	542	542
q5	7540	2406	2152	2152
q6	195	171	141	141
q7	924	755	604	604
q8	9235	1381	1217	1217
q9	5388	4991	4886	4886
q10	6839	2326	1886	1886
q11	472	281	281	281
q12	345	351	222	222
q13	17773	3691	3135	3135
q14	232	242	207	207
q15	515	473	471	471
q16	619	634	591	591
q17	556	854	335	335
q18	6997	6450	6294	6294
q19	1220	953	521	521
q20	327	336	191	191
q21	2881	2282	1987	1987
q22	372	339	315	315
Total cold run time: 102693 ms
Total hot run time: 32245 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5488	5495	5459	5459
q2	240	331	238	238
q3	2275	2644	2288	2288
q4	1437	1796	1379	1379
q5	4375	4735	4672	4672
q6	160	156	124	124
q7	2044	1966	1797	1797
q8	2680	2824	2669	2669
q9	7312	7213	7259	7213
q10	2993	3239	2772	2772
q11	592	523	521	521
q12	713	770	663	663
q13	3470	3869	3232	3232
q14	275	311	282	282
q15	515	478	457	457
q16	647	688	661	661
q17	1236	1721	1286	1286
q18	7628	7421	7424	7421
q19	789	1172	1090	1090
q20	2005	2003	1910	1910
q21	5742	5268	4870	4870
q22	600	612	525	525
Total cold run time: 53216 ms
Total hot run time: 51529 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186522 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 359fa20ef11c20bcce6e03698b6b3235cd2f6a4d, data reload: false

query1	977	379	362	362
query2	6549	2104	2112	2104
query3	6793	219	214	214
query4	33423	23142	22814	22814
query5	4353	624	456	456
query6	267	195	173	173
query7	4613	489	291	291
query8	287	229	232	229
query9	9529	2571	2554	2554
query10	466	302	234	234
query11	18183	15300	15004	15004
query12	155	108	101	101
query13	1640	498	375	375
query14	9856	6795	6751	6751
query15	217	205	191	191
query16	7873	635	463	463
query17	1613	751	563	563
query18	2019	406	302	302
query19	228	186	164	164
query20	120	115	108	108
query21	220	125	105	105
query22	4432	4303	4347	4303
query23	34132	33462	32689	32689
query24	6881	2300	2275	2275
query25	484	450	368	368
query26	1210	271	154	154
query27	2058	494	326	326
query28	5349	2421	2378	2378
query29	695	546	411	411
query30	233	181	155	155
query31	965	838	792	792
query32	76	58	64	58
query33	512	364	331	331
query34	733	865	482	482
query35	817	829	748	748
query36	991	1025	928	928
query37	117	101	80	80
query38	4287	4090	4045	4045
query39	1502	1384	1443	1384
query40	203	114	99	99
query41	78	56	56	56
query42	116	98	101	98
query43	520	528	501	501
query44	1301	799	790	790
query45	181	176	164	164
query46	839	1036	636	636
query47	1811	1856	1791	1791
query48	380	393	327	327
query49	784	488	385	385
query50	631	661	397	397
query51	6874	6948	6791	6791
query52	99	106	93	93
query53	222	246	183	183
query54	488	505	399	399
query55	88	82	85	82
query56	261	260	236	236
query57	1165	1135	1076	1076
query58	257	239	237	237
query59	2999	3105	2883	2883
query60	280	288	245	245
query61	118	116	116	116
query62	817	705	669	669
query63	226	194	186	186
query64	4419	1004	644	644
query65	3244	3156	3176	3156
query66	1076	414	304	304
query67	15988	15655	15342	15342
query68	5366	819	518	518
query69	476	282	258	258
query70	1213	1196	1116	1116
query71	378	283	250	250
query72	5759	3827	3768	3768
query73	641	739	352	352
query74	10358	8935	8839	8839
query75	3189	3073	2665	2665
query76	3178	1174	767	767
query77	466	350	275	275
query78	9986	9977	9348	9348
query79	3077	797	589	589
query80	682	543	450	450
query81	504	279	239	239
query82	392	145	121	121
query83	188	175	149	149
query84	243	88	71	71
query85	776	340	305	305
query86	391	320	302	302
query87	4416	4443	4419	4419
query88	4915	2129	2161	2129
query89	384	326	307	307
query90	1873	189	195	189
query91	139	139	106	106
query92	67	56	51	51
query93	2348	877	538	538
query94	672	404	307	307
query95	331	277	250	250
query96	483	615	280	280
query97	2753	2857	2771	2771
query98	226	211	202	202
query99	1268	1404	1240	1240
Total cold run time: 287458 ms
Total hot run time: 186522 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.83 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 359fa20ef11c20bcce6e03698b6b3235cd2f6a4d, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.63	0.11	0.10
query5	0.42	0.44	0.42
query6	1.16	0.65	0.64
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.60	0.51	0.50
query10	0.55	0.57	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.10
query13	0.61	0.60	0.59
query14	2.69	2.88	2.72
query15	0.89	0.82	0.82
query16	0.40	0.39	0.39
query17	1.04	0.96	1.08
query18	0.24	0.21	0.21
query19	1.98	1.85	2.00
query20	0.02	0.01	0.01
query21	15.35	0.96	0.57
query22	0.76	0.77	0.64
query23	15.39	1.44	0.54
query24	2.54	0.36	1.37
query25	0.21	0.13	0.12
query26	0.38	0.16	0.13
query27	0.06	0.04	0.04
query28	13.32	1.04	0.43
query29	12.59	3.97	3.28
query30	0.26	0.09	0.06
query31	2.85	0.59	0.40
query32	3.22	0.55	0.46
query33	2.98	2.95	3.02
query34	16.62	5.12	4.43
query35	4.52	4.51	4.48
query36	0.61	0.49	0.50
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 105.09 s
Total hot run time: 29.83 s

@zddr
Copy link
Contributor Author

zddr commented Jan 21, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32351 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 70bf08b01a9916e9466c2eabd52a67bbb22a845e, data reload: false

------ Round 1 ----------------------------------
q1	17565	5840	5740	5740
q2	2043	303	165	165
q3	10875	1264	780	780
q4	10246	956	516	516
q5	7964	2360	2120	2120
q6	190	164	130	130
q7	881	754	605	605
q8	9255	1341	1151	1151
q9	5107	4792	4832	4792
q10	6828	2329	1910	1910
q11	472	266	255	255
q12	343	347	218	218
q13	17746	3702	3071	3071
q14	236	226	219	219
q15	516	454	473	454
q16	646	634	584	584
q17	562	844	331	331
q18	6931	6350	6399	6350
q19	2578	941	531	531
q20	293	311	182	182
q21	2804	2179	1939	1939
q22	365	334	308	308
Total cold run time: 104446 ms
Total hot run time: 32351 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6582	5917	6002	5917
q2	245	330	234	234
q3	2219	2610	2299	2299
q4	1410	1857	1350	1350
q5	4268	4729	4773	4729
q6	176	160	127	127
q7	2034	1992	1801	1801
q8	2585	2832	2704	2704
q9	7305	7237	7302	7237
q10	3034	3289	2743	2743
q11	559	518	495	495
q12	682	796	590	590
q13	3478	3874	3273	3273
q14	286	308	272	272
q15	540	461	475	461
q16	628	711	644	644
q17	1222	1758	1255	1255
q18	7769	7503	7426	7426
q19	791	1116	1062	1062
q20	1964	2037	1892	1892
q21	5649	5119	4970	4970
q22	617	590	551	551
Total cold run time: 54043 ms
Total hot run time: 52032 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193499 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 70bf08b01a9916e9466c2eabd52a67bbb22a845e, data reload: false

query1	1337	949	928	928
query2	6317	2139	2047	2047
query3	11088	4542	4480	4480
query4	61272	29127	23206	23206
query5	5458	600	455	455
query6	437	201	193	193
query7	5566	519	300	300
query8	318	230	229	229
query9	8090	2595	2576	2576
query10	452	315	270	270
query11	17566	14935	15641	14935
query12	168	110	109	109
query13	1476	556	412	412
query14	10846	6453	7425	6453
query15	222	218	186	186
query16	7466	671	514	514
query17	1137	761	585	585
query18	1911	421	323	323
query19	198	199	165	165
query20	115	120	109	109
query21	209	120	108	108
query22	4567	4735	4613	4613
query23	34177	33555	33300	33300
query24	5580	2321	2302	2302
query25	472	464	392	392
query26	645	286	157	157
query27	1780	469	340	340
query28	4357	2476	2427	2427
query29	531	574	419	419
query30	211	188	167	167
query31	916	881	840	840
query32	70	56	60	56
query33	493	359	305	305
query34	744	869	504	504
query35	857	850	772	772
query36	1054	1091	968	968
query37	120	106	84	84
query38	4242	4486	4348	4348
query39	1566	1473	1432	1432
query40	221	119	102	102
query41	54	53	55	53
query42	121	104	109	104
query43	539	535	518	518
query44	1293	823	827	823
query45	182	181	176	176
query46	880	1077	661	661
query47	1946	1942	1895	1895
query48	410	395	329	329
query49	723	489	407	407
query50	660	675	394	394
query51	7016	7078	7056	7056
query52	101	100	97	97
query53	231	259	180	180
query54	493	518	429	429
query55	89	78	87	78
query56	261	259	268	259
query57	1220	1190	1105	1105
query58	258	224	221	221
query59	3120	3156	2912	2912
query60	283	267	248	248
query61	117	112	108	108
query62	737	716	673	673
query63	221	183	180	180
query64	1301	1025	671	671
query65	3225	3228	3145	3145
query66	759	394	304	304
query67	15859	15656	15479	15479
query68	5186	824	521	521
query69	479	292	261	261
query70	1228	1096	1160	1096
query71	420	280	265	265
query72	6045	3903	3538	3538
query73	795	753	360	360
query74	9930	9163	8771	8771
query75	3179	3139	2671	2671
query76	3774	1168	874	874
query77	477	360	284	284
query78	10148	9961	9297	9297
query79	3617	822	583	583
query80	1823	534	458	458
query81	553	287	238	238
query82	710	153	121	121
query83	262	178	151	151
query84	293	99	70	70
query85	808	347	294	294
query86	467	316	299	299
query87	4392	4564	4328	4328
query88	4400	2151	2173	2151
query89	450	335	291	291
query90	1612	188	188	188
query91	145	136	111	111
query92	69	57	54	54
query93	2994	890	520	520
query94	876	386	300	300
query95	331	267	247	247
query96	490	623	282	282
query97	2855	2863	2782	2782
query98	234	202	193	193
query99	1317	1427	1284	1284
Total cold run time: 316710 ms
Total hot run time: 193499 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.92 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 70bf08b01a9916e9466c2eabd52a67bbb22a845e, data reload: false

query1	0.05	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.07	0.06
query4	1.62	0.10	0.10
query5	0.40	0.43	0.40
query6	1.15	0.67	0.65
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.61	0.51	0.49
query10	0.57	0.56	0.54
query11	0.14	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.60	0.60
query14	2.71	2.83	2.73
query15	0.91	0.83	0.83
query16	0.38	0.37	0.37
query17	1.03	1.06	1.06
query18	0.22	0.21	0.21
query19	1.87	1.77	2.02
query20	0.01	0.01	0.01
query21	15.36	0.85	0.59
query22	0.74	0.85	0.63
query23	15.31	1.41	0.50
query24	2.70	1.33	1.82
query25	0.19	0.14	0.04
query26	0.20	0.14	0.14
query27	0.06	0.05	0.05
query28	14.44	1.00	0.43
query29	12.60	3.97	3.30
query30	0.25	0.10	0.06
query31	2.83	0.60	0.40
query32	3.23	0.55	0.46
query33	3.00	3.04	3.06
query34	16.61	5.15	4.57
query35	4.63	4.48	4.56
query36	0.64	0.48	0.47
query37	0.09	0.06	0.05
query38	0.04	0.03	0.04
query39	0.04	0.03	0.03
query40	0.16	0.13	0.12
query41	0.09	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.09 s
Total hot run time: 30.92 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 22, 2025
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit aebd17f into apache:master Feb 13, 2025
33 checks passed
zddr added a commit to zddr/incubator-doris that referenced this pull request Feb 13, 2025
…able (apache#47185)

The current priority is that session variable is higher than user
property, which is incorrect because session variable can be set freely
by ordinary users and will bypass the restrictions set by administrators

Scope of Impact:
- getInsertTimeoutS
- getQueryTimeoutS
- getMaxExecMemByte
lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
…able (apache#47185)

### What problem does this PR solve?
The current priority is that session variable is higher than user
property, which is incorrect because session variable can be set freely
by ordinary users and will bypass the restrictions set by administrators

Scope of Impact:
- getInsertTimeoutS
- getQueryTimeoutS
- getMaxExecMemByte
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.9-merged dev/3.0.x dev/3.0.x-conflict kind/behavior-changed reviewed usercase Important user case type label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants