Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Improvement](load) Do no block in group commit sink #36612

Merged
merged 2 commits into from
Jun 24, 2024
Merged

[Improvement](load) Do no block in group commit sink #36612

merged 2 commits into from
Jun 24, 2024

Conversation

Gabriel39
Copy link
Contributor

@Gabriel39 Gabriel39 commented Jun 20, 2024 •
edited
Loading

Proposed changes

Group commit sink operator will create an internal loading task before starting. This is a blocking stop now to create task by RPC which is not allowed on pipeline engine.
This PR makes this blocking step a dependency.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Gabriel39
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.47% (9007/24698)
Line Coverage: 28.01% (73865/263663)
Region Coverage: 27.50% (38364/139485)
Branch Coverage: 24.20% (19554/80786)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a1e4fefbae6b34484cd96b6b989ce81822c07fd7_a1e4fefbae6b34484cd96b6b989ce81822c07fd7/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 39615 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a1e4fefbae6b34484cd96b6b989ce81822c07fd7, data reload: false

------ Round 1 ----------------------------------
q1	17615	4357	4250	4250
q2	2045	199	205	199
q3	10441	1075	1117	1075
q4	10194	752	778	752
q5	7474	2687	2659	2659
q6	224	136	134	134
q7	951	618	595	595
q8	9222	2061	2097	2061
q9	9033	6485	6441	6441
q10	9023	3735	3733	3733
q11	433	236	247	236
q12	451	233	220	220
q13	17787	2939	3010	2939
q14	253	217	215	215
q15	530	488	474	474
q16	530	385	390	385
q17	958	691	731	691
q18	8001	7382	7346	7346
q19	3002	1412	1544	1412
q20	638	319	332	319
q21	5006	3238	3142	3142
q22	396	348	337	337
Total cold run time: 114207 ms
Total hot run time: 39615 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4416	4256	4228	4228
q2	370	284	262	262
q3	3001	2758	2820	2758
q4	2019	1725	1715	1715
q5	5473	5599	5526	5526
q6	219	136	138	136
q7	2189	1807	1822	1807
q8	3287	3430	3396	3396
q9	8673	8678	8698	8678
q10	4116	3798	3926	3798
q11	584	481	500	481
q12	778	617	608	608
q13	17060	3189	3178	3178
q14	318	279	284	279
q15	537	479	502	479
q16	504	439	446	439
q17	1823	1565	1493	1493
q18	8096	8046	7733	7733
q19	1870	1592	1759	1592
q20	2114	1892	1898	1892
q21	5057	5063	4872	4872
q22	657	545	543	543
Total cold run time: 73161 ms
Total hot run time: 55893 ms

@Gabriel39
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39872 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e6c5ce6a9f953cdc944ed2463814a69ffbfe52f5, data reload: false

------ Round 1 ----------------------------------
q1	11434	4326	4281	4281
q2	1497	194	196	194
q3	3727	1124	1074	1074
q4	1006	779	747	747
q5	2758	2608	2982	2608
q6	241	139	147	139
q7	942	601	594	594
q8	2040	2063	2113	2063
q9	6681	6509	6500	6500
q10	3869	3800	3682	3682
q11	360	242	250	242
q12	392	229	229	229
q13	16999	2974	2995	2974
q14	275	218	225	218
q15	530	483	491	483
q16	495	392	375	375
q17	1024	671	665	665
q18	7972	7516	7437	7437
q19	1621	1488	1447	1447
q20	541	322	328	322
q21	6662	3260	3955	3260
q22	405	346	338	338
Total cold run time: 71471 ms
Total hot run time: 39872 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4339	4207	4315	4207
q2	384	260	278	260
q3	2931	2764	2786	2764
q4	1915	1628	1663	1628
q5	5264	5280	5316	5280
q6	217	129	131	129
q7	2176	1689	1756	1689
q8	3246	3339	3358	3339
q9	8312	8401	8303	8303
q10	3870	3644	3645	3644
q11	582	501	517	501
q12	772	595	586	586
q13	3737	3014	3055	3014
q14	288	258	263	258
q15	537	475	474	474
q16	459	409	424	409
q17	1792	1475	1473	1473
q18	7520	7583	7335	7335
q19	1670	1582	1562	1562
q20	1961	1746	1779	1746
q21	4901	4565	4765	4565
q22	617	553	571	553
Total cold run time: 57490 ms
Total hot run time: 53719 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.48% (9009/24698)
Line Coverage: 28.02% (73887/263664)
Region Coverage: 27.52% (38382/139485)
Branch Coverage: 24.22% (19563/80786)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e6c5ce6a9f953cdc944ed2463814a69ffbfe52f5_e6c5ce6a9f953cdc944ed2463814a69ffbfe52f5/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 172095 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e6c5ce6a9f953cdc944ed2463814a69ffbfe52f5, data reload: false

query1	925	380	376	376
query2	6466	2436	2392	2392
query3	6651	207	209	207
query4	19301	17111	16945	16945
query5	4158	488	477	477
query6	253	184	154	154
query7	4588	302	287	287
query8	356	309	311	309
query9	8543	2455	2444	2444
query10	613	303	289	289
query11	10482	10079	9884	9884
query12	149	88	87	87
query13	1644	371	383	371
query14	9580	6226	6725	6226
query15	222	189	200	189
query16	7560	263	263	263
query17	1399	540	534	534
query18	1767	273	273	273
query19	199	157	157	157
query20	97	85	86	85
query21	209	128	127	127
query22	4209	4009	4040	4009
query23	33467	33064	33378	33064
query24	12366	2831	2810	2810
query25	674	357	358	357
query26	1788	150	149	149
query27	3074	324	312	312
query28	7249	2070	2064	2064
query29	1143	618	605	605
query30	270	148	148	148
query31	964	725	787	725
query32	92	59	54	54
query33	770	287	291	287
query34	952	476	463	463
query35	744	618	650	618
query36	1115	925	932	925
query37	209	73	74	73
query38	2859	2741	2757	2741
query39	860	807	795	795
query40	273	124	123	123
query41	65	54	55	54
query42	121	99	99	99
query43	629	571	538	538
query44	1221	734	745	734
query45	197	165	165	165
query46	1067	727	705	705
query47	1863	1742	1769	1742
query48	367	295	302	295
query49	1185	411	402	402
query50	761	387	388	387
query51	7029	7012	6820	6820
query52	117	94	92	92
query53	366	291	293	291
query54	929	450	442	442
query55	78	80	73	73
query56	285	302	285	285
query57	1142	1033	1058	1033
query58	255	237	263	237
query59	3396	3163	3451	3163
query60	297	278	294	278
query61	96	93	100	93
query62	649	443	451	443
query63	316	298	289	289
query64	9819	2332	1813	1813
query65	3195	3121	3140	3121
query66	1375	347	346	346
query67	15465	14954	14937	14937
query68	4585	533	528	528
query69	558	440	386	386
query70	1115	1153	1142	1142
query71	409	274	290	274
query72	7321	5442	5646	5442
query73	752	328	329	328
query74	5871	5505	5463	5463
query75	3369	2652	2672	2652
query76	2697	930	918	918
query77	652	304	311	304
query78	10650	9734	9867	9734
query79	2741	514	505	505
query80	1954	472	459	459
query81	552	265	225	225
query82	705	116	105	105
query83	334	172	180	172
query84	275	89	89	89
query85	1813	280	271	271
query86	484	311	310	310
query87	3307	3092	3096	3092
query88	4033	2354	2351	2351
query89	481	391	377	377
query90	1803	193	196	193
query91	131	101	99	99
query92	69	50	51	50
query93	3829	519	513	513
query94	1247	192	192	192
query95	402	318	324	318
query96	592	263	269	263
query97	3297	3037	3074	3037
query98	226	198	201	198
query99	1317	830	861	830
Total cold run time: 278096 ms
Total hot run time: 172095 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.95 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e6c5ce6a9f953cdc944ed2463814a69ffbfe52f5, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.06
query4	1.66	0.08	0.08
query5	0.51	0.51	0.50
query6	1.14	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.50	0.50
query10	0.52	0.57	0.55
query11	0.15	0.12	0.12
query12	0.15	0.12	0.12
query13	0.60	0.58	0.60
query14	0.77	0.77	0.77
query15	0.82	0.81	0.79
query16	0.37	0.36	0.36
query17	1.00	1.01	0.97
query18	0.24	0.22	0.23
query19	1.84	1.70	1.83
query20	0.01	0.01	0.02
query21	15.42	0.67	0.68
query22	4.02	6.88	2.18
query23	18.25	1.38	1.29
query24	2.10	0.25	0.23
query25	0.15	0.08	0.08
query26	0.27	0.17	0.17
query27	0.08	0.08	0.08
query28	13.24	1.01	0.99
query29	12.59	3.29	3.30
query30	0.26	0.07	0.06
query31	2.85	0.39	0.39
query32	3.27	0.47	0.47
query33	2.88	2.94	2.89
query34	17.18	4.43	4.40
query35	4.49	4.51	4.52
query36	0.64	0.46	0.45
query37	0.18	0.16	0.16
query38	0.15	0.14	0.14
query39	0.05	0.04	0.04
query40	0.17	0.15	0.15
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.17 s
Total hot run time: 30.95 s

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@Gabriel39
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.46% (9007/24702)
Line Coverage: 28.00% (73871/263782)
Region Coverage: 27.48% (38360/139591)
Branch Coverage: 24.19% (19557/80856)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a7d3bece2d35913ae73c01e8d094557e988584ff_a7d3bece2d35913ae73c01e8d094557e988584ff/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 40926 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a7d3bece2d35913ae73c01e8d094557e988584ff, data reload: false

------ Round 1 ----------------------------------
q1	17616	4333	4319	4319
q2	2023	191	200	191
q3	10446	1155	1093	1093
q4	10191	709	757	709
q5	7498	2651	2563	2563
q6	220	135	134	134
q7	943	611	604	604
q8	9231	2061	2048	2048
q9	8786	6499	6495	6495
q10	8959	3766	3712	3712
q11	451	239	240	239
q12	561	235	217	217
q13	17773	2976	2945	2945
q14	262	212	227	212
q15	508	487	494	487
q16	519	376	377	376
q17	963	626	630	626
q18	8212	7953	7920	7920
q19	7199	1383	1450	1383
q20	708	334	332	332
q21	5341	3976	4050	3976
q22	406	352	345	345
Total cold run time: 118816 ms
Total hot run time: 40926 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4566	4401	4367	4367
q2	366	256	265	256
q3	3086	2913	2880	2880
q4	1980	1700	1692	1692
q5	5479	5513	5468	5468
q6	219	127	124	124
q7	2168	1914	1829	1829
q8	3262	3391	3428	3391
q9	8687	8681	8761	8681
q10	4127	3859	3896	3859
q11	615	497	492	492
q12	789	611	632	611
q13	16109	3041	3127	3041
q14	335	272	274	272
q15	516	489	479	479
q16	480	426	444	426
q17	1809	1485	1495	1485
q18	8044	7791	7675	7675
q19	1754	1633	1661	1633
q20	3253	1842	1878	1842
q21	4901	5031	4868	4868
q22	785	536	566	536
Total cold run time: 73330 ms
Total hot run time: 55907 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173555 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a7d3bece2d35913ae73c01e8d094557e988584ff, data reload: false

query1	812	394	369	369
query2	6395	2369	2295	2295
query3	6413	208	209	208
query4	19250	17500	17359	17359
query5	3768	486	468	468
query6	251	163	170	163
query7	4477	295	299	295
query8	349	281	285	281
query9	8559	2409	2399	2399
query10	574	312	290	290
query11	10595	10174	10063	10063
query12	137	89	89	89
query13	1585	353	354	353
query14	9044	6985	6952	6952
query15	232	192	191	191
query16	7473	255	262	255
query17	1406	541	539	539
query18	1933	272	286	272
query19	198	151	153	151
query20	93	89	85	85
query21	207	130	130	130
query22	4314	4036	4257	4036
query23	33569	33757	33518	33518
query24	11147	2915	2849	2849
query25	643	354	355	354
query26	1505	152	149	149
query27	3056	326	312	312
query28	7253	2066	2058	2058
query29	991	650	621	621
query30	273	155	152	152
query31	958	738	731	731
query32	92	53	56	53
query33	771	293	278	278
query34	916	469	455	455
query35	707	639	607	607
query36	1126	970	951	951
query37	164	69	68	68
query38	2853	2743	2740	2740
query39	849	790	802	790
query40	209	125	126	125
query41	53	50	53	50
query42	120	99	98	98
query43	563	555	536	536
query44	1212	737	734	734
query45	197	170	162	162
query46	1064	699	707	699
query47	1864	1778	1802	1778
query48	376	303	298	298
query49	956	429	419	419
query50	789	392	378	378
query51	6898	6762	6721	6721
query52	108	98	92	92
query53	353	290	287	287
query54	844	457	449	449
query55	75	76	76	76
query56	305	280	283	280
query57	1146	1048	1057	1048
query58	264	234	250	234
query59	3327	3240	3462	3240
query60	310	282	291	282
query61	114	110	109	109
query62	626	443	466	443
query63	322	289	293	289
query64	8988	2209	1778	1778
query65	3183	3101	3118	3101
query66	1455	325	330	325
query67	15481	14987	14949	14949
query68	6938	528	531	528
query69	639	443	387	387
query70	1122	1034	1153	1034
query71	483	276	273	273
query72	7807	5432	6122	5432
query73	810	320	325	320
query74	5851	5437	5477	5437
query75	4156	2682	2680	2680
query76	4040	875	947	875
query77	685	302	296	296
query78	10401	9790	9755	9755
query79	2106	520	520	520
query80	1336	469	458	458
query81	584	222	217	217
query82	750	103	102	102
query83	235	177	167	167
query84	270	83	94	83
query85	1410	315	278	278
query86	453	316	305	305
query87	3278	3086	3115	3086
query88	3851	2367	2375	2367
query89	472	388	384	384
query90	1861	187	189	187
query91	129	99	100	99
query92	69	51	50	50
query93	2081	499	495	495
query94	1181	188	187	187
query95	414	320	310	310
query96	593	269	267	267
query97	3282	3019	3018	3018
query98	210	211	201	201
query99	1314	848	848	848
Total cold run time: 274951 ms
Total hot run time: 173555 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.7 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a7d3bece2d35913ae73c01e8d094557e988584ff, data reload: false

query1	0.04	0.04	0.02
query2	0.08	0.04	0.04
query3	0.24	0.05	0.04
query4	1.66	0.07	0.08
query5	0.50	0.49	0.49
query6	1.12	0.74	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.48
query10	0.55	0.56	0.55
query11	0.17	0.12	0.12
query12	0.14	0.12	0.12
query13	0.60	0.58	0.60
query14	0.80	0.78	0.77
query15	0.83	0.81	0.80
query16	0.35	0.34	0.37
query17	0.97	0.98	1.00
query18	0.22	0.24	0.24
query19	1.90	1.68	1.80
query20	0.02	0.01	0.01
query21	15.45	0.66	0.67
query22	3.77	7.32	1.97
query23	18.24	1.39	1.30
query24	2.16	0.22	0.23
query25	0.15	0.09	0.09
query26	0.26	0.18	0.18
query27	0.09	0.08	0.08
query28	13.23	1.02	0.99
query29	12.57	3.33	3.31
query30	0.26	0.06	0.06
query31	2.85	0.40	0.39
query32	3.25	0.49	0.46
query33	2.86	2.90	2.94
query34	17.10	4.45	4.43
query35	4.46	4.49	4.50
query36	0.65	0.46	0.46
query37	0.20	0.16	0.16
query38	0.15	0.14	0.14
query39	0.05	0.04	0.04
query40	0.17	0.14	0.14
query41	0.10	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.05	0.04
Total cold run time: 108.92 s
Total hot run time: 30.7 s

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@Gabriel39
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39339 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cf878fdc2a12513c48fd73b67db1fd1b8717b48d, data reload: false

------ Round 1 ----------------------------------
q1	17615	4321	4270	4270
q2	2024	187	193	187
q3	10458	1095	1069	1069
q4	10193	766	724	724
q5	7535	2661	2649	2649
q6	218	134	133	133
q7	937	606	591	591
q8	9205	2053	2033	2033
q9	8795	6427	6437	6427
q10	8997	3764	3712	3712
q11	456	239	232	232
q12	468	222	231	222
q13	18011	2976	2954	2954
q14	267	209	226	209
q15	509	465	471	465
q16	511	376	377	376
q17	948	606	611	606
q18	7998	7354	7333	7333
q19	5194	1327	1422	1327
q20	661	325	335	325
q21	5000	3199	3168	3168
q22	395	327	334	327
Total cold run time: 116395 ms
Total hot run time: 39339 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4331	4295	4206	4206
q2	370	259	265	259
q3	2976	2720	2914	2720
q4	2009	1744	1712	1712
q5	5565	5513	5520	5513
q6	227	130	129	129
q7	2194	1803	1905	1803
q8	3290	3422	3367	3367
q9	8640	8623	8779	8623
q10	4166	3885	3846	3846
q11	577	484	492	484
q12	769	613	644	613
q13	17023	3258	3184	3184
q14	313	261	278	261
q15	526	490	488	488
q16	505	430	424	424
q17	1827	1494	1508	1494
q18	7961	7786	7691	7691
q19	1855	1615	1594	1594
q20	3040	1881	1905	1881
q21	5260	4835	4932	4835
q22	634	537	542	537
Total cold run time: 74058 ms
Total hot run time: 55664 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169521 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cf878fdc2a12513c48fd73b67db1fd1b8717b48d, data reload: false

query1	934	382	371	371
query2	6454	2413	2390	2390
query3	6635	208	216	208
query4	20850	17069	17208	17069
query5	3604	474	465	465
query6	258	165	157	157
query7	4585	289	287	287
query8	319	296	280	280
query9	8366	2443	2407	2407
query10	605	288	285	285
query11	10623	9999	10041	9999
query12	122	84	85	84
query13	1629	363	361	361
query14	9782	7473	6460	6460
query15	225	187	187	187
query16	7764	282	256	256
query17	1874	556	512	512
query18	1944	271	269	269
query19	197	152	156	152
query20	96	88	81	81
query21	219	125	130	125
query22	4359	4223	3970	3970
query23	33912	33380	33542	33380
query24	10797	2862	2773	2773
query25	631	376	363	363
query26	1120	160	152	152
query27	2444	325	322	322
query28	7129	2113	2147	2113
query29	907	627	643	627
query30	253	152	154	152
query31	957	760	761	760
query32	95	52	56	52
query33	788	279	292	279
query34	1088	464	480	464
query35	752	670	626	626
query36	1125	956	967	956
query37	141	70	73	70
query38	2929	2830	2797	2797
query39	887	838	831	831
query40	212	126	126	126
query41	53	49	54	49
query42	114	95	103	95
query43	593	562	557	557
query44	1176	721	737	721
query45	190	163	194	163
query46	1058	714	709	709
query47	1868	1779	1823	1779
query48	363	294	301	294
query49	856	406	405	405
query50	760	386	390	386
query51	6825	6851	6824	6824
query52	102	88	93	88
query53	346	285	283	283
query54	898	438	435	435
query55	75	74	75	74
query56	276	259	266	259
query57	1120	1042	1023	1023
query58	256	238	255	238
query59	3438	3196	3117	3117
query60	293	292	271	271
query61	94	95	88	88
query62	619	444	454	444
query63	312	295	292	292
query64	8757	2221	1783	1783
query65	3219	3068	3088	3068
query66	750	331	336	331
query67	15287	14970	14905	14905
query68	4534	529	533	529
query69	529	396	352	352
query70	1191	1155	1094	1094
query71	436	278	271	271
query72	7545	2741	2575	2575
query73	727	316	316	316
query74	5909	5407	5429	5407
query75	3338	2652	2681	2652
query76	2761	944	898	898
query77	442	297	348	297
query78	10202	9915	9784	9784
query79	2119	511	509	509
query80	2237	463	459	459
query81	558	215	222	215
query82	862	106	101	101
query83	296	176	166	166
query84	262	90	88	88
query85	1983	280	265	265
query86	476	329	292	292
query87	3307	3042	3131	3042
query88	3852	2319	2330	2319
query89	472	374	381	374
query90	1756	190	183	183
query91	125	101	100	100
query92	60	47	51	47
query93	2417	498	485	485
query94	1103	186	182	182
query95	399	307	374	307
query96	582	258	263	258
query97	3187	3041	3060	3041
query98	225	203	196	196
query99	1284	818	839	818
Total cold run time: 272485 ms
Total hot run time: 169521 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.23 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cf878fdc2a12513c48fd73b67db1fd1b8717b48d, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.50	0.47	0.49
query6	1.13	0.74	0.73
query7	0.03	0.02	0.01
query8	0.05	0.04	0.05
query9	0.56	0.49	0.49
query10	0.55	0.55	0.54
query11	0.16	0.11	0.12
query12	0.15	0.11	0.12
query13	0.59	0.59	0.59
query14	0.77	0.79	0.75
query15	0.82	0.80	0.81
query16	0.37	0.37	0.35
query17	0.97	0.97	0.95
query18	0.21	0.24	0.26
query19	1.76	1.67	1.75
query20	0.02	0.01	0.01
query21	15.42	0.66	0.67
query22	4.27	7.32	1.62
query23	18.29	1.41	1.27
query24	2.16	0.24	0.22
query25	0.16	0.08	0.09
query26	0.25	0.18	0.18
query27	0.08	0.08	0.08
query28	13.22	1.02	1.01
query29	12.62	3.24	3.22
query30	0.26	0.06	0.07
query31	2.85	0.39	0.39
query32	3.28	0.47	0.47
query33	2.88	2.94	2.92
query34	16.91	4.50	4.47
query35	4.56	4.46	4.46
query36	0.66	0.48	0.50
query37	0.19	0.16	0.15
query38	0.16	0.15	0.15
query39	0.05	0.03	0.03
query40	0.17	0.14	0.13
query41	0.10	0.04	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.31 s
Total hot run time: 30.23 s

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

dataroaring
dataroaring approved these changes Jun 23, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 23, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@Gabriel39 Gabriel39 merged commit 1c435c2 into apache:master Jun 24, 2024
28 of 31 checks passed
dataroaring pushed a commit that referenced this pull request Jun 26, 2024
@Gabriel39 @dataroaring
[Improvement](load) Do no block in group commit sink (#36612)
861a34f 
Group commit sink operator will create an internal loading task before
starting. This is a blocking stop now to create task by RPC which is not
allowed on pipeline engine.
This PR makes this blocking step a dependency.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Mryange Mryange Mryange approved these changes

@zclllyybb zclllyybb zclllyybb approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Gabriel39 @doris-robot @Mryange @zclllyybb @dataroaring