Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improvement](index)add index_id column in show index stmt #30431

Merged
merged 2 commits into from
Jan 30, 2024
Merged

[improvement](index)add index_id column in show index stmt #30431

merged 2 commits into from
Jan 30, 2024

Conversation

qidaye
Copy link
Contributor

@qidaye qidaye commented Jan 26, 2024

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@qidaye
Copy link
Contributor Author

qidaye commented Jan 26, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39604 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d22014a17cbdb00dd7c3f3262c0c9c2dc0215848, data reload: false

------ Round 1 ----------------------------------
q1	17618	5494	5369	5369
q2	2042	141	132	132
q3	10609	953	942	942
q4	5002	777	887	777
q5	7960	3037	3019	3019
q6	191	129	123	123
q7	1137	763	756	756
q8	9317	2101	2088	2088
q9	7956	6617	6479	6479
q10	8112	2448	2427	2427
q11	412	200	200	200
q12	760	271	271	271
q13	17982	3397	3323	3323
q14	291	264	257	257
q15	544	503	499	499
q16	471	403	412	403
q17	957	546	511	511
q18	7711	6809	6918	6809
q19	1588	1421	1431	1421
q20	582	331	328	328
q21	7095	3168	3283	3168
q22	813	302	302	302
Total cold run time: 109150 ms
Total hot run time: 39604 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5551	5380	5366	5366
q2	323	219	213	213
q3	3343	3249	3233	3233
q4	2092	2114	2069	2069
q5	6097	5850	5948	5850
q6	199	116	116	116
q7	2316	1913	1868	1868
q8	3223	3393	3405	3393
q9	9133	8954	8753	8753
q10	5351	3874	3838	3838
q11	561	450	442	442
q12	822	629	662	629
q13	16903	3111	3183	3111
q14	280	258	270	258
q15	562	512	505	505
q16	514	488	449	449
q17	1857	1846	1808	1808
q18	9633	19886	9772	9772
q19	23828	1574	1679	1574
q20	4594	1932	1900	1900
q21	14437	5278	5377	5278
q22	957	564	560	560
Total cold run time: 112576 ms
Total hot run time: 60985 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 175373 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d22014a17cbdb00dd7c3f3262c0c9c2dc0215848, data reload: false

query1	923	325	321	321
query2	6575	1827	1889	1827
query3	6699	201	197	197
query4	34343	22137	22208	22137
query5	4450	440	363	363
query6	255	153	146	146
query7	4603	262	259	259
query8	223	176	174	174
query9	8328	2528	2546	2528
query10	420	229	220	220
query11	16824	15673	15444	15444
query12	115	68	65	65
query13	1684	372	387	372
query14	9767	6754	6824	6754
query15	220	178	178	178
query16	5696	259	249	249
query17	957	467	466	466
query18	1780	260	247	247
query19	176	126	124	124
query20	71	69	72	69
query21	190	129	127	127
query22	4930	4721	4780	4721
query23	31556	30875	30766	30766
query24	12980	2836	2840	2836
query25	583	322	300	300
query26	1941	137	143	137
query27	3457	290	284	284
query28	7774	1827	1825	1825
query29	2268	594	594	594
query30	281	135	137	135
query31	917	719	756	719
query32	73	52	48	48
query33	715	209	210	209
query34	1291	452	457	452
query35	914	789	754	754
query36	1315	1221	1203	1203
query37	175	58	58	58
query38	3347	3224	3184	3184
query39	1318	1245	1247	1245
query40	344	82	85	82
query41	37	39	36	36
query42	89	81	80	80
query43	514	481	484	481
query44	1104	689	697	689
query45	190	174	175	174
query46	1074	667	654	654
query47	1665	1563	1583	1563
query48	381	315	317	315
query49	1222	282	279	279
query50	688	315	310	310
query51	5300	5228	5235	5228
query52	79	73	72	72
query53	325	256	257	256
query54	246	183	174	174
query55	80	76	73	73
query56	183	167	168	167
query57	1000	945	933	933
query58	183	150	152	150
query59	2651	2628	2570	2570
query60	207	180	195	180
query61	86	82	81	81
query62	588	379	378	378
query63	274	254	260	254
query64	6034	1798	1778	1778
query65	3300	3221	3227	3221
query66	1505	318	309	309
query67	15619	15231	14767	14767
query68	14379	518	522	518
query69	604	293	285	285
query70	2247	1373	1393	1373
query71	480	206	215	206
query72	4844	2817	2874	2817
query73	3896	314	310	310
query74	6938	6378	6384	6378
query75	4953	2262	2312	2262
query76	6281	1074	1011	1011
query77	729	226	236	226
query78	9200	8636	8447	8447
query79	2750	510	515	510
query80	575	322	322	322
query81	453	200	203	200
query82	200	82	81	81
query83	161	113	113	113
query84	275	73	66	66
query85	1106	339	339	339
query86	388	379	404	379
query87	3502	3268	3301	3268
query88	2753	2210	2197	2197
query89	421	364	363	363
query90	1964	186	184	184
query91	155	116	123	116
query92	48	44	45	44
query93	1304	459	443	443
query94	1314	157	160	157
query95	494	464	453	453
query96	621	330	331	330
query97	4241	4126	4217	4126
query98	195	183	188	183
query99	1020	675	726	675
Total cold run time: 303593 ms
Total hot run time: 175373 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.75 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d22014a17cbdb00dd7c3f3262c0c9c2dc0215848, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.02	0.02
query3	0.22	0.05	0.06
query4	1.70	0.07	0.07
query5	0.54	0.52	0.53
query6	1.29	0.62	0.63
query7	0.02	0.02	0.01
query8	0.03	0.02	0.02
query9	0.54	0.48	0.50
query10	0.56	0.55	0.55
query11	0.12	0.09	0.08
query12	0.12	0.09	0.09
query13	0.59	0.61	0.60
query14	0.78	0.80	0.79
query15	0.78	0.80	0.77
query16	0.40	0.37	0.37
query17	0.98	0.99	1.01
query18	0.23	0.27	0.24
query19	1.94	1.78	1.77
query20	0.01	0.01	0.02
query21	15.42	0.54	0.60
query22	2.59	1.89	2.49
query23	17.43	0.76	0.85
query24	2.36	1.89	0.90
query25	0.43	0.25	0.14
query26	0.47	0.14	0.14
query27	0.05	0.06	0.05
query28	10.87	0.75	0.78
query29	12.50	3.12	3.07
query30	0.53	0.49	0.49
query31	2.78	0.37	0.35
query32	3.36	0.48	0.48
query33	3.20	3.23	3.20
query34	15.83	4.31	4.32
query35	4.34	4.32	4.28
query36	1.11	1.07	1.07
query37	0.07	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.13	0.13
query41	0.07	0.01	0.02
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.6 s
Total hot run time: 30.75 s

airborne12
airborne12 approved these changes Jan 26, 2024
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit d22014a17cbdb00dd7c3f3262c0c9c2dc0215848 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.4 seconds inserted 10000000 Rows, about 746K ops/s

xiaokang
xiaokang reviewed Jan 26, 2024
View reviewed changes
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change the testcase output

@qidaye
Copy link
Contributor Author

qidaye commented Jan 29, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37418 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ed3abbb045b9c26803e936dbd46688cedc5e10da, data reload: false

------ Round 1 ----------------------------------
q1	17671	5808	5000	5000
q2	2055	155	140	140
q3	10673	906	947	906
q4	4689	673	693	673
q5	7690	2784	2815	2784
q6	188	119	119	119
q7	1119	721	715	715
q8	9420	2011	2006	2006
q9	7164	6312	6272	6272
q10	8079	2457	2400	2400
q11	415	208	196	196
q12	781	273	283	273
q13	18000	3300	3280	3280
q14	279	252	244	244
q15	533	488	484	484
q16	453	408	409	408
q17	913	523	474	474
q18	6992	6071	6151	6071
q19	1543	1311	1397	1311
q20	593	319	321	319
q21	7016	3133	3052	3052
q22	806	305	291	291
Total cold run time: 107072 ms
Total hot run time: 37418 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5193	5065	5206	5065
q2	339	234	244	234
q3	2976	2595	2799	2595
q4	1799	1481	1597	1481
q5	5355	5236	5201	5201
q6	202	117	114	114
q7	2144	1755	1791	1755
q8	3080	3188	3202	3188
q9	8690	8556	13862	8556
q10	5339	3385	3461	3385
q11	565	470	457	457
q12	791	592	600	592
q13	17062	3240	3227	3227
q14	274	262	271	262
q15	544	496	492	492
q16	534	471	455	455
q17	2105	1631	1761	1631
q18	9104	11125	8775	8775
q19	24638	1595	1524	1524
q20	4646	1923	1896	1896
q21	17029	4948	4833	4833
q22	3334	485	483	483
Total cold run time: 115743 ms
Total hot run time: 56201 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172491 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ed3abbb045b9c26803e936dbd46688cedc5e10da, data reload: false

query1	939	330	319	319
query2	6585	1857	1923	1857
query3	6705	201	198	198
query4	32405	21958	22034	21958
query5	4464	411	414	411
query6	240	174	147	147
query7	4605	259	253	253
query8	261	172	172	172
query9	8592	2256	2240	2240
query10	411	207	202	202
query11	18018	15307	15231	15231
query12	120	68	66	66
query13	1676	390	362	362
query14	9196	6455	6393	6393
query15	204	176	174	174
query16	5701	244	244	244
query17	967	464	461	461
query18	1816	246	244	244
query19	175	131	125	125
query20	73	68	69	68
query21	204	131	140	131
query22	4987	4712	4795	4712
query23	31090	30245	30348	30245
query24	12450	2792	2764	2764
query25	591	303	300	300
query26	1967	141	133	133
query27	3171	279	281	279
query28	7659	1831	1802	1802
query29	2135	625	607	607
query30	285	135	142	135
query31	887	699	732	699
query32	71	50	52	50
query33	692	206	212	206
query34	1161	452	457	452
query35	850	779	748	748
query36	1288	1101	1194	1101
query37	93	60	62	60
query38	3248	3197	3138	3138
query39	1297	1251	1246	1246
query40	349	85	84	84
query41	36	35	34	34
query42	90	78	83	78
query43	482	495	465	465
query44	1108	691	688	688
query45	192	173	172	172
query46	1046	645	641	641
query47	1601	1526	1487	1487
query48	380	303	304	303
query49	1198	278	276	276
query50	672	314	305	305
query51	5248	5166	5227	5166
query52	89	81	74	74
query53	329	250	257	250
query54	260	180	176	176
query55	77	71	80	71
query56	163	159	163	159
query57	1006	910	883	883
query58	177	155	155	155
query59	2585	2556	2444	2444
query60	197	179	181	179
query61	86	87	82	82
query62	614	355	348	348
query63	282	256	259	256
query64	5971	1769	1741	1741
query65	3237	3222	3168	3168
query66	1384	321	315	315
query67	14401	14336	14517	14336
query68	3746	480	471	471
query69	426	298	297	297
query70	1553	1458	1480	1458
query71	280	212	208	208
query72	3718	2830	2862	2830
query73	689	316	313	313
query74	6645	6263	6268	6263
query75	2826	2275	2253	2253
query76	2648	1016	1108	1016
query77	357	230	222	222
query78	9031	8912	8523	8523
query79	2487	479	498	479
query80	1978	321	316	316
query81	524	199	196	196
query82	695	82	77	77
query83	240	115	110	110
query84	280	72	70	70
query85	2212	344	334	334
query86	510	413	355	355
query87	3409	3307	3297	3297
query88	3887	2163	2168	2163
query89	406	345	365	345
query90	1932	180	179	179
query91	141	116	115	115
query92	55	45	43	43
query93	2274	431	416	416
query94	1376	158	156	156
query95	487	460	450	450
query96	604	313	317	313
query97	4296	4105	4161	4105
query98	221	192	177	177
query99	1068	672	677	672
Total cold run time: 281540 ms
Total hot run time: 172491 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.02 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ed3abbb045b9c26803e936dbd46688cedc5e10da, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.02	0.02
query3	0.23	0.06	0.06
query4	1.67	0.10	0.10
query5	0.54	0.52	0.52
query6	1.19	0.63	0.62
query7	0.01	0.01	0.02
query8	0.04	0.02	0.02
query9	0.52	0.49	0.47
query10	0.56	0.56	0.56
query11	0.12	0.08	0.09
query12	0.11	0.10	0.09
query13	0.60	0.60	0.61
query14	0.77	0.80	0.80
query15	0.79	0.77	0.77
query16	0.40	0.38	0.38
query17	1.01	1.01	1.04
query18	0.22	0.27	0.26
query19	1.89	1.80	1.81
query20	0.02	0.01	0.01
query21	15.41	0.59	0.58
query22	2.58	2.52	1.97
query23	16.96	0.78	0.82
query24	2.63	1.60	0.91
query25	0.31	0.19	0.12
query26	0.62	0.14	0.14
query27	0.06	0.05	0.04
query28	11.02	0.85	0.83
query29	12.60	3.14	3.07
query30	0.57	0.51	0.55
query31	2.78	0.34	0.35
query32	3.40	0.48	0.47
query33	3.23	3.15	3.20
query34	15.80	4.28	4.31
query35	4.41	4.48	4.31
query36	1.12	1.06	1.06
query37	0.06	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.02	0.01
query40	0.16	0.12	0.12
query41	0.07	0.02	0.01
query42	0.02	0.01	0.02
query43	0.03	0.02	0.02
Total cold run time: 104.7 s
Total hot run time: 31.02 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ed3abbb045b9c26803e936dbd46688cedc5e10da with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.6 seconds inserted 10000000 Rows, about 735K ops/s

xiaokang
xiaokang approved these changes Jan 29, 2024
View reviewed changes
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 29, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@qidaye qidaye merged commit 98ac9dd into apache:master Jan 30, 2024
26 checks passed
@qidaye qidaye deleted the show_index_id branch January 30, 2024 02:17
qidaye added a commit to qidaye/incubator-doris that referenced this pull request Jan 30, 2024
@qidaye
[improvement](index)add index_id column in show index stmt (apache#30431
cd8cd92 
)
@qidaye qidaye mentioned this pull request Jan 30, 2024
Merged
yiguolei pushed a commit that referenced this pull request Jan 30, 2024
@qidaye @Doris-Extras
[improvement](index)add index_id column in show index stmt (#30431)
b712f0b
xiaokang pushed a commit that referenced this pull request Jan 31, 2024
@qidaye
[improvement](index)add index_id column in show index stmt (#30431) (#…
3468b32 
…30544)
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@qidaye
[improvement](index)add index_id column in show index stmt (apache#30431
33e1ed8 
) (apache#30544)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xiaokang xiaokang xiaokang approved these changes

@airborne12 airborne12 airborne12 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.5-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@qidaye @doris-robot @xiaokang @airborne12