Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Improment](executor)Add downgrade scan thread num by column num #35351

Merged
merged 1 commit into from
May 27, 2024
Merged

[Improment](executor)Add downgrade scan thread num by column num #35351

merged 1 commit into from
May 27, 2024

Conversation

wangbo
Copy link
Contributor

@wangbo wangbo commented May 24, 2024

Proposed changes

refer #35340

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@wangbo
Copy link
Contributor Author

wangbo commented May 24, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@wangbo
Copy link
Contributor Author

wangbo commented May 24, 2024

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 24, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 40498 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit da083b2c762f1d60fc17d36a5ee9b471225bcb77, data reload: false

------ Round 1 ----------------------------------
q1	17813	5610	4381	4381
q2	2666	188	204	188
q3	11184	1187	1175	1175
q4	10298	864	847	847
q5	7845	2747	2814	2747
q6	224	140	137	137
q7	977	623	619	619
q8	9456	2142	2126	2126
q9	9052	6545	6524	6524
q10	8962	3692	3691	3691
q11	468	242	232	232
q12	432	225	222	222
q13	17764	2991	2964	2964
q14	253	215	226	215
q15	507	474	460	460
q16	532	378	376	376
q17	999	674	739	674
q18	8173	7586	7437	7437
q19	6373	1536	1514	1514
q20	691	321	312	312
q21	5006	3381	3883	3381
q22	343	276	280	276
Total cold run time: 120018 ms
Total hot run time: 40498 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4361	4247	4234	4234
q2	385	272	271	271
q3	3025	2734	2746	2734
q4	1881	1597	1624	1597
q5	5248	5293	5269	5269
q6	211	126	128	126
q7	2169	1695	1751	1695
q8	3199	3348	3315	3315
q9	8379	8353	8397	8353
q10	3903	3681	3676	3676
q11	581	475	482	475
q12	746	609	595	595
q13	16990	3014	3017	3014
q14	278	263	272	263
q15	509	475	476	475
q16	462	410	413	410
q17	1785	1481	1517	1481
q18	7653	7438	7481	7438
q19	1683	1537	1575	1537
q20	1990	1798	1779	1779
q21	4905	4695	4765	4695
q22	597	507	516	507
Total cold run time: 70940 ms
Total hot run time: 53939 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172016 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit da083b2c762f1d60fc17d36a5ee9b471225bcb77, data reload: false

query1	929	380	365	365
query2	6442	2463	2316	2316
query3	6648	205	209	205
query4	19223	17306	17379	17306
query5	4136	417	421	417
query6	245	155	149	149
query7	4591	298	284	284
query8	265	185	179	179
query9	8603	2430	2398	2398
query10	449	279	254	254
query11	10586	10074	10035	10035
query12	132	90	85	85
query13	1631	358	364	358
query14	9343	6907	7611	6907
query15	214	169	169	169
query16	7974	265	262	262
query17	1838	522	524	522
query18	2025	286	268	268
query19	202	153	159	153
query20	95	87	86	86
query21	202	137	152	137
query22	4346	3944	3894	3894
query23	33402	33245	32820	32820
query24	12040	2945	2796	2796
query25	650	370	359	359
query26	1743	158	156	156
query27	2922	319	322	319
query28	7300	2083	2095	2083
query29	1042	617	595	595
query30	282	169	176	169
query31	979	752	731	731
query32	97	52	52	52
query33	762	283	282	282
query34	965	479	496	479
query35	757	601	606	601
query36	1095	914	916	914
query37	201	71	73	71
query38	2925	2768	2761	2761
query39	855	803	788	788
query40	275	124	123	123
query41	47	43	44	43
query42	104	100	98	98
query43	597	539	548	539
query44	1294	737	749	737
query45	179	171	166	166
query46	1064	746	733	733
query47	1835	1752	1779	1752
query48	378	304	299	299
query49	1190	378	385	378
query50	772	395	398	395
query51	6766	6703	6753	6703
query52	109	93	87	87
query53	349	288	290	288
query54	966	432	422	422
query55	70	72	70	70
query56	266	242	246	242
query57	1145	1071	1100	1071
query58	236	214	232	214
query59	3392	3164	3119	3119
query60	281	264	254	254
query61	93	96	104	96
query62	664	466	448	448
query63	318	291	293	291
query64	9804	2196	1766	1766
query65	3179	3116	3107	3107
query66	1394	323	335	323
query67	15376	15036	14682	14682
query68	4591	545	554	545
query69	454	280	274	274
query70	1101	1127	1148	1127
query71	404	278	264	264
query72	7435	5357	5393	5357
query73	724	323	316	316
query74	5988	5668	5640	5640
query75	3308	2676	2632	2632
query76	2610	1034	997	997
query77	412	266	271	266
query78	10215	9701	9754	9701
query79	2594	526	524	524
query80	2018	461	447	447
query81	527	250	253	250
query82	760	95	94	94
query83	315	172	174	172
query84	264	87	87	87
query85	1893	289	269	269
query86	484	331	290	290
query87	3274	3188	3126	3126
query88	4090	2399	2352	2352
query89	485	399	393	393
query90	1975	186	187	186
query91	121	93	98	93
query92	56	50	48	48
query93	3319	531	514	514
query94	1275	184	185	184
query95	404	315	316	315
query96	606	270	271	270
query97	3207	3043	3035	3035
query98	244	232	221	221
query99	1238	856	866	856
Total cold run time: 276670 ms
Total hot run time: 172016 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.13 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit da083b2c762f1d60fc17d36a5ee9b471225bcb77, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.04	0.05
query4	1.68	0.08	0.08
query5	0.50	0.49	0.55
query6	1.12	0.75	0.72
query7	0.02	0.01	0.01
query8	0.05	0.03	0.04
query9	0.54	0.48	0.47
query10	0.54	0.54	0.53
query11	0.16	0.12	0.11
query12	0.14	0.11	0.12
query13	0.59	0.58	0.60
query14	0.76	0.78	0.77
query15	0.82	0.82	0.81
query16	0.36	0.36	0.35
query17	1.01	0.96	1.02
query18	0.21	0.25	0.23
query19	1.90	1.71	1.73
query20	0.01	0.01	0.01
query21	15.49	0.67	0.65
query22	3.93	8.49	1.61
query23	18.31	1.39	1.24
query24	1.89	0.22	0.21
query25	0.16	0.08	0.09
query26	0.27	0.17	0.17
query27	0.08	0.08	0.08
query28	13.36	1.01	1.00
query29	13.24	3.34	3.31
query30	0.24	0.06	0.06
query31	2.86	0.38	0.38
query32	3.30	0.47	0.47
query33	2.92	2.91	2.90
query34	16.96	4.40	4.37
query35	4.47	4.48	4.54
query36	0.65	0.46	0.46
query37	0.17	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.03
query40	0.16	0.14	0.15
query41	0.10	0.04	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.03
Total cold run time: 109.59 s
Total hot run time: 30.13 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.65% (9019/25297)
Line Coverage: 27.31% (74578/273098)
Region Coverage: 26.54% (38598/145459)
Branch Coverage: 23.40% (19696/84158)
Coverage Report: http://coverage.selectdb-in.cc/coverage/da083b2c762f1d60fc17d36a5ee9b471225bcb77_da083b2c762f1d60fc17d36a5ee9b471225bcb77/report/index.html

xinyiZzz
xinyiZzz approved these changes May 27, 2024
View reviewed changes
Copy link
Contributor

@xinyiZzz xinyiZzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wangbo wangbo merged commit ec365c4 into apache:master May 27, 2024
26 of 28 checks passed
yiguolei pushed a commit that referenced this pull request May 27, 2024
@wangbo @Doris-Extras
Add downgrade scan thread num by column num (#35351)
c44affb
seawinde pushed a commit to seawinde/doris that referenced this pull request May 27, 2024
@wangbo @seawinde
Add downgrade scan thread num by column num (apache#35351)
a029ba4
@wangbo wangbo mentioned this pull request May 28, 2024
Merged
yiguolei pushed a commit that referenced this pull request May 28, 2024
@wangbo
[Fix]Add downgrade scan thread num by column num (#35485)
6d99953 
pick #35351
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yiguolei yiguolei yiguolei approved these changes

@xinyiZzz xinyiZzz xinyiZzz approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.0-merged meta-change reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@wangbo @doris-robot @yiguolei @xinyiZzz @dataroaring