Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](fe) Use table.readLock in TabletStatMgr for updating statistics #39612

Merged
merged 2 commits into from
Aug 23, 2024
Merged

[opt](fe) Use table.readLock in TabletStatMgr for updating statistics #39612

merged 2 commits into from
Aug 23, 2024

Conversation

SWJTU-ZhangLei
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@SWJTU-ZhangLei
Copy link
Contributor Author

run buildall

dataroaring
dataroaring previously approved these changes Aug 20, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 20, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@SWJTU-ZhangLei
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Aug 20, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 38611 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d7d793a71eeaad3700ad30450f6f725044f12967, data reload: false

------ Round 1 ----------------------------------
q1	17873	4383	4287	4287
q2	2183	211	210	210
q3	11819	964	1243	964
q4	10539	805	870	805
q5	7785	2858	2818	2818
q6	269	163	159	159
q7	1023	656	649	649
q8	9598	2107	2146	2107
q9	8795	6636	6606	6606
q10	7067	2263	2246	2246
q11	507	275	267	267
q12	433	272	268	268
q13	17928	3052	3050	3050
q14	298	253	254	253
q15	557	518	512	512
q16	529	428	409	409
q17	989	638	666	638
q18	7462	6987	6849	6849
q19	7259	1140	1193	1140
q20	754	356	367	356
q21	3920	2949	2958	2949
q22	1206	1069	1095	1069
Total cold run time: 118793 ms
Total hot run time: 38611 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4751	4547	4385	4385
q2	423	317	307	307
q3	3033	2899	2893	2893
q4	2051	1761	1690	1690
q5	5703	5798	5694	5694
q6	243	156	159	156
q7	2238	1784	1832	1784
q8	3367	3520	3420	3420
q9	8703	8756	8797	8756
q10	3572	3268	3266	3266
q11	610	506	523	506
q12	850	691	650	650
q13	16881	3132	3080	3080
q14	321	304	305	304
q15	563	543	524	524
q16	497	441	474	441
q17	1844	1586	1543	1543
q18	8430	7795	7813	7795
q19	4214	1538	1629	1538
q20	2139	1911	1869	1869
q21	14536	5276	5253	5253
q22	1177	1089	1107	1089
Total cold run time: 86146 ms
Total hot run time: 56943 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196340 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d7d793a71eeaad3700ad30450f6f725044f12967, data reload: false

query1	1290	892	901	892
query2	6560	1981	1922	1922
query3	10646	3753	3902	3753
query4	58368	25969	23356	23356
query5	6259	727	717	717
query6	552	210	209	209
query7	6447	326	332	326
query8	549	433	430	430
query9	9571	2545	2515	2515
query10	640	346	337	337
query11	18979	15064	15525	15064
query12	194	136	127	127
query13	1693	446	458	446
query14	12471	7577	7566	7566
query15	262	200	196	196
query16	7844	516	510	510
query17	1249	620	631	620
query18	1840	356	357	356
query19	311	177	168	168
query20	148	141	136	136
query21	254	147	149	147
query22	4504	4611	4318	4318
query23	34384	33784	33942	33784
query24	5710	3068	3050	3050
query25	589	428	430	428
query26	726	188	181	181
query27	1795	317	306	306
query28	3884	2189	2094	2094
query29	719	440	446	440
query30	221	180	179	179
query31	1034	834	779	779
query32	123	78	81	78
query33	560	343	338	338
query34	900	513	499	499
query35	860	764	774	764
query36	1079	933	968	933
query37	160	102	100	100
query38	3968	3788	3806	3788
query39	1543	1489	1451	1451
query40	227	157	155	155
query41	138	136	137	136
query42	131	120	116	116
query43	542	513	507	507
query44	1138	800	803	800
query45	230	193	195	193
query46	1117	771	748	748
query47	1913	1814	1856	1814
query48	449	343	337	337
query49	958	577	577	577
query50	869	468	480	468
query51	7193	7029	7091	7029
query52	124	112	109	109
query53	299	228	225	225
query54	617	503	502	502
query55	90	92	94	92
query56	330	309	306	306
query57	1240	1120	1137	1120
query58	299	301	310	301
query59	3058	2962	2733	2733
query60	356	333	326	326
query61	147	144	144	144
query62	797	676	693	676
query63	257	258	228	228
query64	4398	2375	1847	1847
query65	3308	3179	3207	3179
query66	1045	663	665	663
query67	15341	15210	15024	15024
query68	8031	589	603	589
query69	729	432	319	319
query70	1567	1170	1095	1095
query71	561	321	319	319
query72	6972	2372	2089	2089
query73	2660	360	363	360
query74	9214	8740	8769	8740
query75	4078	2725	2740	2725
query76	5023	1029	1032	1029
query77	828	445	441	441
query78	11644	9582	8995	8995
query79	11657	562	559	559
query80	1381	618	617	617
query81	614	267	262	262
query82	510	158	158	158
query83	279	215	210	210
query84	296	95	96	95
query85	859	354	353	353
query86	362	293	333	293
query87	4418	4208	4199	4199
query88	4312	2580	2498	2498
query89	497	322	321	321
query90	2197	234	234	234
query91	155	129	127	127
query92	85	73	76	73
query93	3661	563	556	556
query94	1020	333	295	295
query95	395	296	294	294
query96	618	289	289	289
query97	3197	3044	3069	3044
query98	244	225	226	225
query99	1598	1279	1309	1279
Total cold run time: 345159 ms
Total hot run time: 196340 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.91 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d7d793a71eeaad3700ad30450f6f725044f12967, data reload: false

query1	0.04	0.05	0.04
query2	0.08	0.05	0.04
query3	0.23	0.05	0.06
query4	1.67	0.08	0.07
query5	0.48	0.50	0.50
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.06	0.05	0.04
query9	0.56	0.49	0.49
query10	0.54	0.54	0.54
query11	0.16	0.12	0.12
query12	0.16	0.13	0.13
query13	0.62	0.59	0.60
query14	0.75	0.79	0.79
query15	0.84	0.82	0.81
query16	0.38	0.38	0.37
query17	1.04	0.97	0.97
query18	0.22	0.21	0.21
query19	1.82	1.78	1.68
query20	0.02	0.01	0.01
query21	15.41	0.66	0.65
query22	4.04	7.02	1.91
query23	18.31	1.36	1.25
query24	2.13	0.24	0.22
query25	0.16	0.09	0.09
query26	0.26	0.18	0.18
query27	0.08	0.08	0.08
query28	13.18	1.03	1.01
query29	12.68	3.30	3.31
query30	0.43	0.25	0.24
query31	2.79	0.42	0.40
query32	3.24	0.49	0.49
query33	2.95	2.93	2.92
query34	17.04	4.37	4.42
query35	4.42	4.41	4.43
query36	0.67	0.50	0.48
query37	0.21	0.17	0.18
query38	0.18	0.17	0.16
query39	0.07	0.06	0.05
query40	0.18	0.15	0.14
query41	0.12	0.07	0.07
query42	0.08	0.07	0.07
query43	0.06	0.06	0.07
Total cold run time: 109.52 s
Total hot run time: 30.91 s

@SWJTU-ZhangLei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38154 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d7d793a71eeaad3700ad30450f6f725044f12967, data reload: false

------ Round 1 ----------------------------------
q1	17852	4338	4336	4336
q2	2055	216	221	216
q3	11791	1020	1065	1020
q4	10526	735	801	735
q5	7779	2855	2755	2755
q6	280	159	162	159
q7	1015	651	648	648
q8	9569	2091	2097	2091
q9	7930	6565	6602	6565
q10	7041	2215	2235	2215
q11	485	269	272	269
q12	425	261	256	256
q13	17907	3042	3025	3025
q14	296	259	258	258
q15	554	532	524	524
q16	517	430	413	413
q17	1002	669	712	669
q18	7429	6920	6812	6812
q19	6348	1074	1120	1074
q20	742	349	359	349
q21	3933	2896	2732	2732
q22	1146	1067	1033	1033
Total cold run time: 116622 ms
Total hot run time: 38154 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4624	4291	4353	4291
q2	421	325	330	325
q3	2878	2624	2620	2620
q4	2006	1706	1749	1706
q5	5681	5686	5707	5686
q6	236	146	144	144
q7	2180	1778	1757	1757
q8	3302	3480	3471	3471
q9	8884	8736	8757	8736
q10	3588	3378	3294	3294
q11	601	530	537	530
q12	851	664	652	652
q13	16312	3230	3125	3125
q14	317	280	284	280
q15	565	518	529	518
q16	517	461	464	461
q17	1844	1580	1574	1574
q18	8218	7884	7587	7587
q19	9587	1454	1730	1454
q20	2220	1898	1874	1874
q21	12229	5280	5305	5280
q22	1175	1045	1070	1045
Total cold run time: 88236 ms
Total hot run time: 56410 ms

@SWJTU-ZhangLei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38535 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d7d793a71eeaad3700ad30450f6f725044f12967, data reload: false

------ Round 1 ----------------------------------
q1	17863	4374	4311	4311
q2	2057	222	229	222
q3	10379	1163	1143	1143
q4	10163	796	777	777
q5	7995	2924	2864	2864
q6	278	164	169	164
q7	1025	694	665	665
q8	9548	2079	2104	2079
q9	6942	6626	6580	6580
q10	7129	2293	2233	2233
q11	534	291	284	284
q12	434	270	272	270
q13	19103	3007	3065	3007
q14	312	267	266	266
q15	575	548	532	532
q16	560	425	418	418
q17	1019	711	756	711
q18	7503	6912	6665	6665
q19	6325	1051	1115	1051
q20	732	350	353	350
q21	4087	3155	2901	2901
q22	1138	1083	1042	1042
Total cold run time: 115701 ms
Total hot run time: 38535 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4519	4314	4261	4261
q2	420	329	334	329
q3	2887	2612	2650	2612
q4	1926	1654	1716	1654
q5	5405	5429	5411	5411
q6	246	157	160	157
q7	2093	1709	1745	1709
q8	3238	3430	3388	3388
q9	8486	8432	8440	8432
q10	3438	3170	3187	3170
q11	623	561	543	543
q12	818	644	640	640
q13	16695	3025	3010	3010
q14	333	309	296	296
q15	545	519	511	511
q16	511	453	451	451
q17	1794	1484	1490	1484
q18	7821	7408	7494	7408
q19	1716	1626	1535	1535
q20	2104	1857	1819	1819
q21	5354	5208	5131	5131
q22	1178	1079	1052	1052
Total cold run time: 72150 ms
Total hot run time: 55003 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191866 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d7d793a71eeaad3700ad30450f6f725044f12967, data reload: false

query1	966	422	389	389
query2	6744	1988	1949	1949
query3	6675	233	244	233
query4	34351	23548	23252	23252
query5	4434	727	713	713
query6	322	223	218	218
query7	4616	336	332	332
query8	482	435	430	430
query9	8675	2536	2517	2517
query10	521	356	342	342
query11	17398	15110	15176	15110
query12	188	138	141	138
query13	1722	459	467	459
query14	10204	7292	7209	7209
query15	272	194	198	194
query16	8059	494	486	486
query17	1620	624	594	594
query18	2123	349	342	342
query19	247	176	196	176
query20	142	138	139	138
query21	256	144	141	141
query22	4227	4092	3962	3962
query23	34051	33365	33237	33237
query24	11451	2981	2951	2951
query25	662	435	424	424
query26	1351	188	202	188
query27	2778	320	301	301
query28	7732	2133	2104	2104
query29	904	449	441	441
query30	329	190	183	183
query31	1056	851	843	843
query32	121	82	82	82
query33	827	346	346	346
query34	923	523	533	523
query35	926	770	787	770
query36	1108	956	916	916
query37	167	106	106	106
query38	4030	3944	3829	3829
query39	1555	1465	1466	1465
query40	315	158	161	158
query41	144	137	140	137
query42	132	116	122	116
query43	539	518	508	508
query44	1278	797	808	797
query45	226	194	201	194
query46	1135	780	774	774
query47	1903	1842	1852	1842
query48	413	342	334	334
query49	1245	588	597	588
query50	875	475	477	475
query51	7292	7153	7089	7089
query52	127	110	111	110
query53	300	231	230	230
query54	927	518	510	510
query55	90	90	92	90
query56	342	317	323	317
query57	1216	1162	1166	1162
query58	326	330	315	315
query59	3034	2776	3021	2776
query60	358	340	331	331
query61	152	147	151	147
query62	886	710	692	692
query63	261	228	232	228
query64	5454	2409	1897	1897
query65	3254	3169	3163	3163
query66	1261	672	675	672
query67	15643	14948	15034	14948
query68	6504	597	598	597
query69	751	425	354	354
query70	1200	1103	1172	1103
query71	574	313	318	313
query72	7663	2298	2087	2087
query73	855	363	362	362
query74	9400	8954	9062	8954
query75	5035	2752	2723	2723
query76	4865	1031	1021	1021
query77	937	444	435	435
query78	9785	9209	9099	9099
query79	3918	555	551	551
query80	1173	606	610	606
query81	618	259	258	258
query82	870	158	158	158
query83	379	218	214	214
query84	293	98	105	98
query85	1487	373	360	360
query86	460	322	331	322
query87	4337	4264	4273	4264
query88	4820	2525	2517	2517
query89	478	332	330	330
query90	2026	236	235	235
query91	155	130	131	130
query92	91	76	75	75
query93	4785	557	550	550
query94	912	320	352	320
query95	398	298	295	295
query96	621	285	290	285
query97	3256	3093	3116	3093
query98	254	233	224	224
query99	1619	1300	1314	1300
Total cold run time: 313862 ms
Total hot run time: 191866 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.48 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d7d793a71eeaad3700ad30450f6f725044f12967, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.66	0.08	0.07
query5	0.52	0.49	0.49
query6	1.14	0.73	0.74
query7	0.02	0.01	0.02
query8	0.06	0.04	0.05
query9	0.55	0.47	0.49
query10	0.56	0.55	0.53
query11	0.15	0.13	0.13
query12	0.16	0.12	0.12
query13	0.62	0.60	0.60
query14	0.76	0.80	0.79
query15	0.85	0.82	0.82
query16	0.37	0.39	0.36
query17	1.05	1.02	1.02
query18	0.21	0.20	0.20
query19	1.75	1.70	1.72
query20	0.01	0.02	0.01
query21	15.41	0.67	0.66
query22	4.56	6.88	2.33
query23	18.33	1.36	1.29
query24	2.08	0.23	0.22
query25	0.15	0.08	0.09
query26	0.28	0.18	0.18
query27	0.09	0.09	0.08
query28	13.27	1.05	1.01
query29	12.67	3.35	3.36
query30	0.43	0.24	0.25
query31	2.84	0.40	0.40
query32	3.23	0.50	0.49
query33	2.96	2.99	3.00
query34	16.93	4.33	4.35
query35	4.39	4.41	4.46
query36	0.68	0.50	0.48
query37	0.20	0.17	0.18
query38	0.16	0.17	0.17
query39	0.06	0.06	0.05
query40	0.18	0.16	0.14
query41	0.11	0.07	0.07
query42	0.07	0.06	0.07
query43	0.07	0.06	0.06
Total cold run time: 109.95 s
Total hot run time: 31.48 s

@SWJTU-ZhangLei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38826 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c84a2455da938baabe71bf411a2f96577e5f066f, data reload: false

------ Round 1 ----------------------------------
q1	18116	4438	4362	4362
q2	2889	228	215	215
q3	10950	1205	1180	1180
q4	10354	780	768	768
q5	7859	2927	2862	2862
q6	269	169	158	158
q7	1049	675	651	651
q8	9487	2093	2040	2040
q9	7195	6543	6580	6543
q10	7094	2253	2293	2253
q11	489	269	277	269
q12	441	264	254	254
q13	17918	3013	2993	2993
q14	300	260	269	260
q15	575	518	537	518
q16	536	414	412	412
q17	991	729	752	729
q18	7582	6830	6843	6830
q19	6502	1122	1096	1096
q20	712	355	356	355
q21	3902	3015	3036	3015
q22	1140	1065	1063	1063
Total cold run time: 116350 ms
Total hot run time: 38826 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4598	4370	4291	4291
q2	423	313	315	313
q3	2867	2603	2620	2603
q4	1900	1572	1583	1572
q5	5358	5370	5599	5370
q6	240	149	147	147
q7	2084	1757	1731	1731
q8	3227	3409	3382	3382
q9	8496	8388	8384	8384
q10	3435	3172	3213	3172
q11	632	523	546	523
q12	799	625	681	625
q13	17153	2971	2994	2971
q14	328	299	289	289
q15	558	509	508	508
q16	509	426	466	426
q17	1799	1495	1492	1492
q18	7775	7581	7446	7446
q19	2886	1573	1517	1517
q20	2071	1829	1811	1811
q21	13983	5060	4911	4911
q22	1135	1059	1030	1030
Total cold run time: 82256 ms
Total hot run time: 54514 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192435 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c84a2455da938baabe71bf411a2f96577e5f066f, data reload: false

query1	990	411	385	385
query2	6734	2063	1961	1961
query3	6658	233	245	233
query4	33703	23275	23220	23220
query5	4309	662	642	642
query6	294	197	192	192
query7	4615	336	337	336
query8	491	442	441	441
query9	8581	2519	2495	2495
query10	523	359	341	341
query11	17272	14925	15137	14925
query12	185	140	140	140
query13	1707	466	461	461
query14	9930	7416	7358	7358
query15	288	196	191	191
query16	8042	526	514	514
query17	1624	607	585	585
query18	2075	335	343	335
query19	333	171	168	168
query20	143	136	135	135
query21	244	146	163	146
query22	4744	4193	4242	4193
query23	34135	33387	33307	33307
query24	11074	2938	2929	2929
query25	648	419	426	419
query26	1151	179	187	179
query27	2586	307	303	303
query28	7102	2157	2101	2101
query29	840	454	448	448
query30	342	187	185	185
query31	1040	857	853	853
query32	121	85	84	84
query33	820	354	334	334
query34	906	510	522	510
query35	889	774	787	774
query36	1114	940	939	939
query37	169	113	102	102
query38	3963	3784	3815	3784
query39	1526	1457	1460	1457
query40	245	160	156	156
query41	146	143	142	142
query42	141	118	118	118
query43	558	500	516	500
query44	1228	781	803	781
query45	239	195	197	195
query46	1133	778	807	778
query47	1945	1859	1860	1859
query48	422	345	347	345
query49	1256	588	586	586
query50	863	455	469	455
query51	7153	7075	7131	7075
query52	126	108	109	108
query53	300	236	224	224
query54	966	517	511	511
query55	90	90	89	89
query56	331	327	308	308
query57	1206	1139	1149	1139
query58	342	297	318	297
query59	2933	2949	2872	2872
query60	353	322	334	322
query61	152	152	166	152
query62	872	703	703	703
query63	267	228	220	220
query64	5309	2391	3787	2391
query65	3230	3161	3191	3161
query66	1164	700	734	700
query67	15579	15055	15022	15022
query68	8150	595	605	595
query69	754	465	375	375
query70	1202	1161	1064	1064
query71	558	314	313	313
query72	7607	2361	2086	2086
query73	840	359	356	356
query74	9001	8867	8758	8758
query75	5301	2732	2690	2690
query76	5118	1111	1045	1045
query77	919	448	439	439
query78	9890	9282	9346	9282
query79	7307	550	555	550
query80	1057	613	605	605
query81	633	266	263	263
query82	306	160	160	160
query83	408	217	217	217
query84	293	102	98	98
query85	1065	367	357	357
query86	372	333	312	312
query87	4312	4348	4205	4205
query88	4612	2503	2559	2503
query89	497	328	318	318
query90	2092	238	230	230
query91	157	127	126	126
query92	87	76	75	75
query93	5256	565	595	565
query94	987	328	338	328
query95	396	297	293	293
query96	620	289	292	289
query97	3255	3092	3098	3092
query98	238	227	233	227
query99	1616	1352	1305	1305
Total cold run time: 315366 ms
Total hot run time: 192435 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c84a2455da938baabe71bf411a2f96577e5f066f, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.05
query3	0.23	0.06	0.06
query4	1.63	0.08	0.08
query5	0.51	0.52	0.51
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.07	0.05	0.05
query9	0.54	0.48	0.48
query10	0.55	0.54	0.55
query11	0.16	0.12	0.12
query12	0.16	0.12	0.13
query13	0.63	0.60	0.58
query14	0.77	0.80	0.79
query15	0.83	0.82	0.84
query16	0.39	0.36	0.38
query17	0.98	0.98	1.00
query18	0.21	0.22	0.22
query19	1.81	1.74	1.72
query20	0.02	0.01	0.01
query21	15.60	0.66	0.65
query22	4.41	7.55	1.93
query23	18.21	1.38	1.24
query24	2.11	0.23	0.24
query25	0.15	0.08	0.08
query26	0.27	0.17	0.17
query27	0.08	0.07	0.08
query28	13.23	1.04	1.01
query29	12.61	3.32	3.32
query30	0.43	0.21	0.20
query31	2.81	0.40	0.39
query32	3.25	0.48	0.47
query33	2.98	2.95	2.92
query34	17.13	4.40	4.42
query35	4.45	4.43	4.44
query36	0.66	0.50	0.49
query37	0.20	0.18	0.19
query38	0.18	0.17	0.17
query39	0.07	0.06	0.05
query40	0.18	0.15	0.15
query41	0.12	0.07	0.07
query42	0.08	0.07	0.06
query43	0.07	0.07	0.06
Total cold run time: 110.05 s
Total hot run time: 31 s

@SWJTU-ZhangLei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39278 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 86237b89a881c56648e68050821447217a29e309, data reload: false

------ Round 1 ----------------------------------
q1	17994	4429	4367	4367
q2	2048	220	221	220
q3	11563	1105	1186	1105
q4	10545	796	765	765
q5	7791	2888	2882	2882
q6	269	159	163	159
q7	1025	667	657	657
q8	10072	2195	2199	2195
q9	7205	6696	6703	6696
q10	7202	2331	2342	2331
q11	502	287	286	286
q12	439	274	275	274
q13	18510	3046	3109	3046
q14	323	281	311	281
q15	569	533	549	533
q16	718	445	436	436
q17	1147	718	692	692
q18	8068	6858	6888	6858
q19	3152	1107	1009	1009
q20	738	365	367	365
q21	3900	3070	3163	3070
q22	1140	1051	1062	1051
Total cold run time: 114920 ms
Total hot run time: 39278 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4442	4346	4303	4303
q2	430	308	312	308
q3	2915	2692	2709	2692
q4	2069	1774	1801	1774
q5	5700	5664	5746	5664
q6	241	146	151	146
q7	2262	1819	1823	1819
q8	3315	3548	3513	3513
q9	8794	8804	8756	8756
q10	3642	3409	3297	3297
q11	612	516	523	516
q12	833	702	676	676
q13	16232	3313	3206	3206
q14	326	301	296	296
q15	577	517	520	517
q16	542	485	475	475
q17	1854	1536	1568	1536
q18	8077	7756	7790	7756
q19	1789	1700	1523	1523
q20	2183	1949	1924	1924
q21	5775	5435	5445	5435
q22	1178	1102	1091	1091
Total cold run time: 73788 ms
Total hot run time: 57223 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198434 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 86237b89a881c56648e68050821447217a29e309, data reload: false

query1	1316	921	882	882
query2	6568	2046	2005	2005
query3	10636	4160	3939	3939
query4	58926	23972	23357	23357
query5	5628	731	723	723
query6	479	215	205	205
query7	5812	329	327	327
query8	513	434	433	433
query9	9088	2573	2560	2560
query10	569	344	348	344
query11	17017	15184	15287	15184
query12	190	145	147	145
query13	1693	441	460	441
query14	11178	7207	7035	7035
query15	245	192	196	192
query16	7313	519	523	519
query17	1162	634	628	628
query18	1961	366	343	343
query19	314	170	171	170
query20	158	142	138	138
query21	254	144	148	144
query22	4582	4579	4361	4361
query23	34564	34345	34267	34267
query24	6062	3087	3001	3001
query25	570	429	432	429
query26	691	191	191	191
query27	1705	317	321	317
query28	3922	2189	2147	2147
query29	701	461	455	455
query30	228	193	204	193
query31	1027	863	851	851
query32	99	75	78	75
query33	507	360	352	352
query34	897	520	542	520
query35	911	783	763	763
query36	1095	975	989	975
query37	163	109	112	109
query38	4165	3870	3919	3870
query39	1520	1478	1457	1457
query40	235	156	174	156
query41	140	138	140	138
query42	142	116	114	114
query43	558	531	520	520
query44	1165	792	799	792
query45	224	195	200	195
query46	1145	773	756	756
query47	1957	1855	1864	1855
query48	422	344	341	341
query49	914	584	585	584
query50	873	487	476	476
query51	7164	7182	7196	7182
query52	119	106	106	106
query53	301	224	223	223
query54	619	524	513	513
query55	91	90	92	90
query56	330	319	323	319
query57	1257	1146	1145	1145
query58	306	347	317	317
query59	3184	2837	2961	2837
query60	354	322	333	322
query61	150	145	146	145
query62	803	702	703	702
query63	256	228	227	227
query64	3016	1899	1892	1892
query65	3272	3192	3231	3192
query66	967	682	690	682
query67	15357	15195	15274	15195
query68	4493	604	619	604
query69	484	323	322	322
query70	1247	1122	1139	1122
query71	398	325	320	320
query72	6476	2341	2082	2082
query73	804	366	366	366
query74	9153	8846	8868	8846
query75	3477	2754	2774	2754
query76	1956	1046	1084	1046
query77	668	466	452	452
query78	9772	9106	9127	9106
query79	1105	565	576	565
query80	882	636	627	627
query81	571	266	266	266
query82	313	157	166	157
query83	236	211	212	211
query84	276	100	98	98
query85	795	363	368	363
query86	339	327	309	309
query87	4411	4319	4425	4319
query88	3710	2568	2533	2533
query89	429	329	321	321
query90	1988	238	234	234
query91	152	128	130	128
query92	86	80	78	78
query93	1108	550	559	550
query94	753	339	339	339
query95	437	306	305	305
query96	608	285	288	285
query97	3254	3137	3112	3112
query98	259	234	232	232
query99	1612	1342	1319	1319
Total cold run time: 312002 ms
Total hot run time: 198434 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 86237b89a881c56648e68050821447217a29e309, data reload: false

query1	0.05	0.05	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.08	0.08
query5	0.52	0.50	0.49
query6	1.15	0.74	0.73
query7	0.02	0.01	0.01
query8	0.06	0.05	0.05
query9	0.55	0.48	0.49
query10	0.55	0.55	0.55
query11	0.16	0.12	0.11
query12	0.16	0.14	0.13
query13	0.63	0.60	0.58
query14	0.78	0.78	0.78
query15	0.84	0.81	0.84
query16	0.37	0.38	0.39
query17	1.07	0.96	1.06
query18	0.22	0.20	0.20
query19	1.82	1.86	1.85
query20	0.01	0.02	0.01
query21	15.39	0.67	0.66
query22	3.99	6.39	2.54
query23	18.28	1.42	1.34
query24	2.14	0.24	0.23
query25	0.16	0.09	0.09
query26	0.28	0.19	0.19
query27	0.08	0.08	0.09
query28	13.26	1.03	1.01
query29	12.62	3.44	3.40
query30	0.43	0.25	0.25
query31	2.82	0.40	0.39
query32	3.25	0.48	0.49
query33	3.01	3.03	3.01
query34	16.91	4.39	4.33
query35	4.46	4.47	4.44
query36	0.69	0.48	0.50
query37	0.22	0.17	0.18
query38	0.18	0.17	0.17
query39	0.07	0.06	0.06
query40	0.19	0.16	0.15
query41	0.12	0.07	0.07
query42	0.08	0.07	0.07
query43	0.07	0.07	0.06
Total cold run time: 109.65 s
Total hot run time: 32 s

dataroaring
dataroaring approved these changes Aug 22, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 22, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

freemandealer
freemandealer approved these changes Aug 23, 2024
View reviewed changes
Copy link
Contributor

@freemandealer freemandealer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit e0793c9 into apache:master Aug 23, 2024
28 of 30 checks passed
dataroaring pushed a commit that referenced this pull request Aug 23, 2024
@SWJTU-ZhangLei @dataroaring
[opt](fe) Use table.readLock in TabletStatMgr for updating statis…
ae83968 
…tics (#39612)
@gavinchou gavinchou mentioned this pull request Sep 11, 2024
Closed
SWJTU-ZhangLei added a commit to SWJTU-ZhangLei/incubator-doris that referenced this pull request Nov 13, 2024
@SWJTU-ZhangLei
[opt](fe) Use table.readLock in TabletStatMgr for updating statis…
c7b1020 
…tics (apache#39612)
SWJTU-ZhangLei added a commit to SWJTU-ZhangLei/incubator-doris that referenced this pull request Nov 14, 2024
@SWJTU-ZhangLei
[opt](fe) Use table.readLock in TabletStatMgr for updating statis…
b3ead9e 
…tics (apache#39612)
@SWJTU-ZhangLei SWJTU-ZhangLei mentioned this pull request Nov 14, 2024
Merged
16 tasks
dataroaring pushed a commit that referenced this pull request Nov 14, 2024
@SWJTU-ZhangLei
[opt](fe) Use table.readLock in TabletStatMgr for updating statis…
8554c06 
…tics (#39612) (#43937)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@freemandealer freemandealer freemandealer approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.2-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@SWJTU-ZhangLei @doris-robot @freemandealer @dataroaring