Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[enhancement](stmt-forward) make forwarded stmt be able to be killed #31688

Merged
merged 4 commits into from
Mar 26, 2024
Merged

[enhancement](stmt-forward) make forwarded stmt be able to be killed #31688

merged 4 commits into from
Mar 26, 2024

Conversation

TangSiyang2001
Copy link
Collaborator

@TangSiyang2001 TangSiyang2001 commented Mar 1, 2024 •
edited
Loading

Proposed changes

Forward a cancel request to the corresponding FE when cancelling a forwarded stmt.

Further comments

Test killing by query id.

// follower starts an insert stmt (that would definitely be forwarded)
+------------------+------+------+-----------------+---------------------+----------+------+---------+------+-------+-----------------------------------+-----------------------------------------------+
| CurrentConnected | Id   | User | Host            | LoginTime           | Catalog  | Db   | Command | Time | State | QueryId                           | Info                                          |
+------------------+------+------+-----------------+---------------------+----------+------+---------+------+-------+-----------------------------------+-----------------------------------------------+
|                  |    0 | root | 127.0.0.1:55374 | 2024-03-07 19:58:04 | internal | ssb  | Query   |    6 | OK    | 52835997b107483d-a93b51be74ed2b84 | insert into lineorder select * from lineorder |
| Yes              |    2 | root | 127.0.0.1:55730 | 2024-03-07 19:59:56 | internal |      | Query   |    0 | OK    | f2b002a6dff74713-8540609060ac64b8 | show processlist                              |
+------------------+------+------+-----------------+---------------------+----------+------+---------+------+-------+-----------------------------------+-----------------------------------------------+

// then kill it
MySQL [(none)]> kill query "52835997b107483d-a93b51be74ed2b84";
Query OK, 0 rows affected (0.010 sec)

// observed killed on follower client
MySQL [ssb]> insert into lineorder select * from lineorder;
ERROR 1105 (HY000): errCode = 2, detailMessage = Cancelled

// observed cancelled in master log
2024-03-07 20:00:08,206 WARN (thrift-server-pool-2|267) [OlapInsertExecutor.onFail():184] insert [label_52835997b107483d_a93b51be74ed2b84] with query id 52835997b107483d-a93b51be74ed2b84 failed
org.apache.doris.common.DdlException: errCode = 2, detailMessage = Cancelled
        at org.apache.doris.common.ErrorReport.reportDdlException(ErrorReport.java:68) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.nereids.trees.plans.commands.insert.AbstractInsertExecutor.execImpl(AbstractInsertExecutor.java:135) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.nereids.trees.plans.commands.insert.AbstractInsertExecutor.executeSingleInsert(AbstractInsertExecutor.java:167) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommand.runInternal(InsertIntoTableCommand.java:183) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommand.run(InsertIntoTableCommand.java:99) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:622) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:497) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.proxyExecute(ConnectProcessor.java:601) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.service.FrontendServiceImpl.forward(FrontendServiceImpl.java:1028) ~[doris-fe.jar:1.2-SNAPSHOT]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
        at org.apache.doris.service.FeServer.lambda$start$0(FeServer.java:59) ~[doris-fe.jar:1.2-SNAPSHOT]
        at jdk.proxy2.$Proxy38.forward(Unknown Source) ~[?:?]
        at org.apache.doris.thrift.FrontendService$Processor$forward.getResult(FrontendService.java:3492) ~[fe-common-1.2-SNAPSHOT.jar:1.2-SNAPSHOT]
        at org.apache.doris.thrift.FrontendService$Processor$forward.getResult(FrontendService.java:3472) ~[fe-common-1.2-SNAPSHOT.jar:1.2-SNAPSHOT]
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) ~[libthrift-0.16.0.jar:0.16.0]
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38) ~[libthrift-0.16.0.jar:0.16.0]
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250) ~[libthrift-0.16.0.jar:0.16.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]

Test killing by connection.

// follower
MySQL [(none)]> show processlist;
+------------------+------+------+-----------------+---------------------+----------+------+---------+------+-------+-----------------------------------+-----------------------------------------------+
| CurrentConnected | Id   | User | Host            | LoginTime           | Catalog  | Db   | Command | Time | State | QueryId                           | Info                                          |
+------------------+------+------+-----------------+---------------------+----------+------+---------+------+-------+-----------------------------------+-----------------------------------------------+
|                  |    0 | root | 127.0.0.1:55374 | 2024-03-07 19:58:04 | internal | ssb  | Query   |    3 | OK    | 380ce6fde9704b84-a607ad25184132a2 | insert into lineorder select * from lineorder |
| Yes              |    2 | root | 127.0.0.1:55730 | 2024-03-07 19:59:56 | internal |      | Query   |    0 | OK    | 778d237c3d5d4bff-8444977c1bd754e8 | show processlist                              |
+------------------+------+------+-----------------+---------------------+----------+------+---------+------+-------+-----------------------------------+-----------------------------------------------+
2 rows in set (0.002 sec)

MySQL [(none)]> kill 0;
Query OK, 0 rows affected (0.005 sec)

MySQL [ssb]> insert into lineorder select * from lineorder;
ERROR 2013 (HY000): Lost connection to MySQL server during query

// Master
2024-03-07 20:05:29,222 WARN (thrift-server-pool-2|267) [OlapInsertExecutor.onFail():184] insert [label_380ce6fde9704b84_a607ad25184132a2] with query id 380ce6fde9704b84-a607ad25184132a2 failed
org.apache.doris.common.DdlException: errCode = 2, detailMessage = Cancelled
        at org.apache.doris.common.ErrorReport.reportDdlException(ErrorReport.java:68) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.nereids.trees.plans.commands.insert.AbstractInsertExecutor.execImpl(AbstractInsertExecutor.java:135) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.nereids.trees.plans.commands.insert.AbstractInsertExecutor.executeSingleInsert(AbstractInsertExecutor.java:167) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommand.runInternal(InsertIntoTableCommand.java:183) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.nereids.trees.plans.commands.insert.InsertIntoTableCommand.run(InsertIntoTableCommand.java:99) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:622) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:497) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.proxyExecute(ConnectProcessor.java:601) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.service.FrontendServiceImpl.forward(FrontendServiceImpl.java:1028) ~[doris-fe.jar:1.2-SNAPSHOT]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
        at org.apache.doris.service.FeServer.lambda$start$0(FeServer.java:59) ~[doris-fe.jar:1.2-SNAPSHOT]
        at jdk.proxy2.$Proxy38.forward(Unknown Source) ~[?:?]
        at org.apache.doris.thrift.FrontendService$Processor$forward.getResult(FrontendService.java:3492) ~[fe-common-1.2-SNAPSHOT.jar:1.2-SNAPSHOT]
        at org.apache.doris.thrift.FrontendService$Processor$forward.getResult(FrontendService.java:3472) ~[fe-common-1.2-SNAPSHOT.jar:1.2-SNAPSHOT]
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) ~[libthrift-0.16.0.jar:0.16.0]
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38) ~[libthrift-0.16.0.jar:0.16.0]
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250) ~[libthrift-0.16.0.jar:0.16.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@TangSiyang2001 TangSiyang2001 changed the title [enhancement](stmt-forward) make forwarded stmt cancelable [enhancement](stmt-forward) make forwarded stmt be able to be killed Mar 1, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 1, 2024

clang-tidy review says "All clean, LGTM! 👍"

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@TangSiyang2001
Copy link
Collaborator Author

run p0

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38025 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 932986ae7db37316566374c7c05ac455a54e1b4b, data reload: false

------ Round 1 ----------------------------------
q1	17721	4051	4024	4024
q2	2033	157	144	144
q3	10548	961	922	922
q4	4668	926	942	926
q5	7612	2960	3025	2960
q6	177	124	129	124
q7	1302	840	832	832
q8	9503	1971	2090	1971
q9	7337	6484	6440	6440
q10	8223	2577	2571	2571
q11	420	227	216	216
q12	745	332	325	325
q13	17970	2938	2899	2899
q14	279	253	251	251
q15	494	454	452	452
q16	485	398	396	396
q17	958	848	832	832
q18	6762	5997	5930	5930
q19	1582	1527	1517	1517
q20	544	282	276	276
q21	7538	3730	3724	3724
q22	801	293	313	293
Total cold run time: 107702 ms
Total hot run time: 38025 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4008	3977	3976	3976
q2	324	233	230	230
q3	2958	2949	2888	2888
q4	1817	1821	1791	1791
q5	5241	5250	5256	5250
q6	203	119	118	118
q7	2277	1863	1839	1839
q8	3216	3270	3276	3270
q9	8588	8565	8515	8515
q10	6134	3724	3726	3724
q11	538	450	436	436
q12	688	528	550	528
q13	10415	2781	2786	2781
q14	279	262	257	257
q15	479	443	439	439
q16	465	399	410	399
q17	1699	1674	1655	1655
q18	8038	7584	7246	7246
q19	1669	1648	1625	1625
q20	1955	1774	1731	1731
q21	4940	4781	4753	4753
q22	523	468	461	461
Total cold run time: 66454 ms
Total hot run time: 53912 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 177778 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 932986ae7db37316566374c7c05ac455a54e1b4b, data reload: false

query1	930	357	338	338
query2	7741	2154	2081	2081
query3	6730	215	217	215
query4	26850	21123	20975	20975
query5	4430	469	495	469
query6	269	176	176	176
query7	4628	313	299	299
query8	237	207	163	163
query9	8332	2205	2164	2164
query10	430	239	223	223
query11	15019	14467	14458	14458
query12	135	96	86	86
query13	1637	437	443	437
query14	9492	6991	7160	6991
query15	235	203	184	184
query16	7235	267	268	267
query17	956	576	545	545
query18	1918	295	284	284
query19	218	167	161	161
query20	97	92	89	89
query21	198	135	127	127
query22	4622	4429	4479	4429
query23	31841	30756	30573	30573
query24	12109	3094	3119	3094
query25	714	414	402	402
query26	1823	167	164	164
query27	2945	361	377	361
query28	6764	1828	1784	1784
query29	1276	616	607	607
query30	310	145	148	145
query31	939	735	745	735
query32	106	69	60	60
query33	757	263	261	261
query34	979	484	476	476
query35	941	812	801	801
query36	923	850	840	840
query37	275	71	74	71
query38	3272	3103	3129	3103
query39	1409	1353	1394	1353
query40	302	122	118	118
query41	60	56	55	55
query42	108	102	114	102
query43	454	417	409	409
query44	1088	736	726	726
query45	208	195	195	195
query46	1073	811	789	789
query47	1622	1544	1557	1544
query48	437	366	353	353
query49	1222	350	353	350
query50	789	388	378	378
query51	6723	6605	6467	6467
query52	101	97	96	96
query53	353	288	290	288
query54	338	253	249	249
query55	89	89	86	86
query56	239	314	224	224
query57	1094	991	990	990
query58	260	229	220	220
query59	2642	2460	2255	2255
query60	278	283	261	261
query61	151	118	115	115
query62	646	414	404	404
query63	311	284	287	284
query64	6471	3259	3276	3259
query65	3061	3041	3032	3032
query66	1484	344	346	344
query67	15179	14607	14362	14362
query68	12719	558	586	558
query69	718	382	384	382
query70	1518	1114	1103	1103
query71	599	281	282	281
query72	10063	2671	2532	2532
query73	2373	343	342	342
query74	7125	6599	6779	6599
query75	5907	2637	2645	2637
query76	7151	1097	1143	1097
query77	661	264	272	264
query78	10266	9552	9570	9552
query79	11154	531	516	516
query80	1396	406	444	406
query81	469	212	208	208
query82	246	85	86	85
query83	229	146	150	146
query84	281	78	79	78
query85	1186	342	330	330
query86	366	284	285	284
query87	3460	3188	3161	3161
query88	3472	2384	2364	2364
query89	525	361	369	361
query90	2023	176	179	176
query91	162	129	128	128
query92	60	51	51	51
query93	4474	521	512	512
query94	1297	193	188	188
query95	447	352	347	347
query96	601	270	268	268
query97	4046	3894	3935	3894
query98	222	220	209	209
query99	1087	778	773	773
Total cold run time: 312808 ms
Total hot run time: 177778 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.36 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 932986ae7db37316566374c7c05ac455a54e1b4b, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.02	0.02
query3	0.23	0.06	0.06
query4	1.68	0.10	0.10
query5	0.51	0.51	0.50
query6	1.31	0.66	0.66
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.58	0.51	0.51
query10	0.56	0.56	0.57
query11	0.13	0.10	0.10
query12	0.13	0.10	0.10
query13	0.57	0.58	0.56
query14	0.74	0.77	0.75
query15	0.82	0.80	0.81
query16	0.37	0.38	0.37
query17	1.01	0.95	1.02
query18	0.27	0.25	0.27
query19	1.82	1.73	1.73
query20	0.02	0.02	0.01
query21	15.41	0.66	0.66
query22	2.66	3.43	3.21
query23	17.59	0.96	0.94
query24	2.33	0.57	0.19
query25	0.28	0.05	0.03
query26	0.19	0.14	0.13
query27	0.04	0.03	0.02
query28	12.16	0.87	0.82
query29	12.57	3.14	3.51
query30	0.61	0.55	0.54
query31	2.81	0.34	0.35
query32	3.38	0.43	0.44
query33	2.95	2.89	2.89
query34	15.47	4.32	4.33
query35	4.31	4.33	4.30
query36	1.09	1.02	1.03
query37	0.06	0.05	0.05
query38	0.04	0.02	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.14
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.19 s
Total hot run time: 31.36 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 932986ae7db37316566374c7c05ac455a54e1b4b with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       18.4 seconds inserted 10000000 Rows, about 543K ops/s

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@TangSiyang2001 TangSiyang2001 force-pushed the kill-forwarded-stmt branch 2 times, most recently from 37bb133 to eb0d4d3 Compare March 6, 2024 16:11
@TangSiyang2001
Copy link
Collaborator Author

run buildall

@TangSiyang2001 TangSiyang2001 marked this pull request as ready for review March 7, 2024 12:09
@TangSiyang2001
Copy link
Collaborator Author

run feut

liaoxin01
liaoxin01 previously approved these changes Mar 18, 2024
View reviewed changes
@TangSiyang2001 TangSiyang2001 dismissed stale reviews from liaoxin01 and dataroaring via a0a4c0d March 18, 2024 09:03
@TangSiyang2001
Copy link
Collaborator Author

resolved conflict

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Mar 18, 2024
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@liaoxin01
Copy link
Contributor

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 20, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@TangSiyang2001
Copy link
Collaborator Author

run p0

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 38734 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8ce1be15e8c6b7de2b0d50d544aea36e5e251c06, data reload: false

------ Round 1 ----------------------------------
q1	18021	4526	4301	4301
q2	2276	165	158	158
q3	11929	1157	1244	1157
q4	10442	789	762	762
q5	7704	3139	3136	3136
q6	213	127	125	125
q7	1061	610	592	592
q8	9586	2048	2046	2046
q9	7384	6740	6743	6740
q10	10166	3481	3557	3481
q11	440	217	219	217
q12	396	197	199	197
q13	17801	2867	2881	2867
q14	244	205	208	205
q15	511	461	459	459
q16	495	374	375	374
q17	986	549	570	549
q18	7239	6463	6471	6463
q19	1721	1496	1414	1414
q20	556	266	249	249
q21	3560	2955	2995	2955
q22	353	287	293	287
Total cold run time: 113084 ms
Total hot run time: 38734 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4167	4118	4125	4118
q2	339	246	236	236
q3	3075	2982	2901	2901
q4	1874	1581	1593	1581
q5	5311	5342	5362	5342
q6	198	119	120	119
q7	2281	1829	1874	1829
q8	3214	3315	3357	3315
q9	8756	8715	8700	8700
q10	3807	3794	3731	3731
q11	550	446	440	440
q12	724	525	528	525
q13	16918	2864	2874	2864
q14	283	245	258	245
q15	506	452	464	452
q16	482	441	412	412
q17	1737	1505	1479	1479
q18	7625	7204	7077	7077
q19	1648	1526	1554	1526
q20	1925	1762	1713	1713
q21	4724	4816	4740	4740
q22	572	469	427	427
Total cold run time: 70716 ms
Total hot run time: 53772 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187850 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8ce1be15e8c6b7de2b0d50d544aea36e5e251c06, data reload: false

query1	924	373	360	360
query2	7361	2066	2119	2066
query3	6707	212	211	211
query4	31855	21480	21653	21480
query5	4385	515	410	410
query6	271	176	172	172
query7	4626	292	302	292
query8	237	183	189	183
query9	9272	2270	2248	2248
query10	564	251	256	251
query11	15307	14600	14529	14529
query12	138	93	90	90
query13	1614	398	410	398
query14	12421	11717	11535	11535
query15	312	217	206	206
query16	8239	266	260	260
query17	1979	608	556	556
query18	2117	291	285	285
query19	370	166	156	156
query20	94	89	94	89
query21	203	135	136	135
query22	4974	4771	4793	4771
query23	33422	32835	32769	32769
query24	10909	2897	2884	2884
query25	616	394	385	385
query26	1112	164	164	164
query27	2370	345	355	345
query28	7038	1881	1857	1857
query29	869	651	628	628
query30	302	153	154	153
query31	982	746	759	746
query32	94	61	59	59
query33	788	265	337	265
query34	951	487	481	481
query35	851	633	627	627
query36	1056	910	896	896
query37	122	83	77	77
query38	3615	3451	3470	3451
query39	1488	1442	1402	1402
query40	214	118	117	117
query41	50	50	49	49
query42	104	100	95	95
query43	487	450	457	450
query44	1214	736	732	732
query45	287	262	264	262
query46	1126	720	719	719
query47	1898	1826	1822	1822
query48	454	348	359	348
query49	1102	344	340	340
query50	765	379	376	376
query51	6762	6754	6606	6606
query52	109	95	98	95
query53	351	281	282	281
query54	316	250	247	247
query55	88	76	85	76
query56	253	235	236	235
query57	1186	1131	1138	1131
query58	242	210	215	210
query59	2830	2562	2695	2562
query60	285	243	246	243
query61	97	96	93	93
query62	662	445	455	445
query63	308	277	281	277
query64	5249	4020	3949	3949
query65	3080	3034	3055	3034
query66	865	361	360	360
query67	15194	14688	14649	14649
query68	5424	513	518	513
query69	595	394	361	361
query70	1192	1200	1155	1155
query71	433	281	283	281
query72	6600	2735	2559	2559
query73	692	306	315	306
query74	7014	6660	6608	6608
query75	3709	2965	2959	2959
query76	3979	820	869	820
query77	625	264	268	264
query78	10868	10275	10202	10202
query79	8397	538	536	536
query80	1987	403	406	403
query81	547	214	222	214
query82	1658	203	200	200
query83	326	146	145	145
query84	291	82	73	73
query85	1538	357	370	357
query86	480	317	295	295
query87	3701	3545	3563	3545
query88	5115	2330	2322	2322
query89	543	370	395	370
query90	1958	184	185	184
query91	185	149	148	148
query92	69	52	53	52
query93	6760	510	493	493
query94	1228	189	186	186
query95	442	335	341	335
query96	630	273	276	273
query97	3054	2898	2895	2895
query98	236	218	209	209
query99	1226	915	912	912
Total cold run time: 307635 ms
Total hot run time: 187850 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 8ce1be15e8c6b7de2b0d50d544aea36e5e251c06 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       20.9 seconds inserted 10000000 Rows, about 478K ops/s

dataroaring
dataroaring approved these changes Mar 26, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit e708249 into apache:master Mar 26, 2024
26 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gavinchou gavinchou gavinchou approved these changes

@liaoxin01 liaoxin01 liaoxin01 approved these changes

@Yukang-Lian Yukang-Lian Yukang-Lian approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. meta-change reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@TangSiyang2001 @doris-robot @liaoxin01 @gavinchou @Yukang-Lian @dataroaring