Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improvement](mtmv) Remove useless code and add some doc #32047

Merged
merged 1 commit into from
Mar 13, 2024

Conversation

seawinde
Copy link
Contributor

Proposed changes

  1. Use visitor to remove the result sink in mv plan
  2. Remove useless code
  3. Add more doc

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@seawinde
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37884 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7877cbb04993fd83459ef160822b98c561b59fc8, data reload: false

------ Round 1 ----------------------------------
q1	17625	4034	4145	4034
q2	2016	149	143	143
q3	10557	944	919	919
q4	4657	934	945	934
q5	7610	2824	2965	2824
q6	180	124	124	124
q7	1287	806	819	806
q8	9490	2025	2047	2025
q9	7319	6464	6435	6435
q10	8222	2550	2548	2548
q11	423	224	229	224
q12	735	316	305	305
q13	17978	2959	2966	2959
q14	275	257	252	252
q15	495	459	466	459
q16	484	385	397	385
q17	951	879	851	851
q18	6721	5953	5849	5849
q19	1576	1521	1506	1506
q20	561	285	287	285
q21	7590	3735	3720	3720
q22	817	320	297	297
Total cold run time: 107569 ms
Total hot run time: 37884 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4025	3965	4011	3965
q2	320	233	242	233
q3	2943	2935	2906	2906
q4	1845	1840	1808	1808
q5	5221	5234	5215	5215
q6	204	118	119	118
q7	2228	1827	1818	1818
q8	3208	3271	3250	3250
q9	8534	8535	8530	8530
q10	6143	3714	3733	3714
q11	536	446	455	446
q12	703	547	537	537
q13	12571	2861	2837	2837
q14	275	245	265	245
q15	483	447	461	447
q16	469	421	402	402
q17	1690	1648	1654	1648
q18	7736	7513	7286	7286
q19	1806	1608	1593	1593
q20	1900	1683	1705	1683
q21	4881	4925	5267	4925
q22	539	439	467	439
Total cold run time: 68260 ms
Total hot run time: 54045 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186442 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7877cbb04993fd83459ef160822b98c561b59fc8, data reload: false

query1	924	374	357	357
query2	7393	1965	1963	1963
query3	6712	213	213	213
query4	24642	20901	21057	20901
query5	4286	845	451	451
query6	278	178	176	176
query7	4630	312	305	305
query8	235	168	177	168
query9	8460	2201	2176	2176
query10	464	248	247	247
query11	14940	14629	14653	14629
query12	131	86	88	86
query13	1662	434	445	434
query14	10629	9776	10089	9776
query15	230	198	192	192
query16	7751	278	272	272
query17	1885	559	553	553
query18	1962	287	274	274
query19	194	157	157	157
query20	95	87	91	87
query21	199	134	127	127
query22	4583	4460	4408	4408
query23	31534	30866	31090	30866
query24	10113	3043	2988	2988
query25	662	389	393	389
query26	1088	175	185	175
query27	2271	367	357	357
query28	6280	1802	1779	1779
query29	1067	621	613	613
query30	308	147	151	147
query31	927	734	753	734
query32	108	63	57	57
query33	781	259	258	258
query34	994	476	483	476
query35	852	643	647	643
query36	922	790	801	790
query37	125	75	79	75
query38	3235	3203	3137	3137
query39	1415	1372	1366	1366
query40	217	116	114	114
query41	66	64	61	61
query42	105	104	104	104
query43	457	418	401	401
query44	1087	727	709	709
query45	282	252	267	252
query46	1044	789	750	750
query47	1643	1547	1545	1545
query48	436	344	359	344
query49	1137	338	341	338
query50	781	383	388	383
query51	6746	6686	6628	6628
query52	114	100	97	97
query53	357	283	287	283
query54	308	246	243	243
query55	88	92	84	84
query56	246	227	248	227
query57	1062	992	996	992
query58	253	222	219	219
query59	2649	2368	2395	2368
query60	267	289	252	252
query61	115	111	109	109
query62	665	403	417	403
query63	311	281	282	281
query64	5195	3441	3259	3259
query65	3023	3002	2981	2981
query66	855	334	329	329
query67	14741	14265	14354	14265
query68	6164	548	566	548
query69	616	390	398	390
query70	1190	1146	1117	1117
query71	495	288	274	274
query72	6463	2652	2495	2495
query73	761	335	320	320
query74	6869	6542	6524	6524
query75	9072	8058	8114	8058
query76	4198	1148	1144	1144
query77	646	260	258	258
query78	10323	9648	9589	9589
query79	9076	522	532	522
query80	1741	443	373	373
query81	539	212	218	212
query82	820	196	208	196
query83	216	140	145	140
query84	286	80	78	78
query85	1496	335	338	335
query86	493	278	325	278
query87	3418	3235	3280	3235
query88	4236	2298	2275	2275
query89	552	361	373	361
query90	1983	175	178	175
query91	164	130	136	130
query92	60	51	50	50
query93	6968	531	513	513
query94	1132	192	193	192
query95	455	350	346	346
query96	609	269	258	258
query97	4436	4291	4296	4291
query98	225	200	205	200
query99	1170	742	760	742
Total cold run time: 296089 ms
Total hot run time: 186442 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.79 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7877cbb04993fd83459ef160822b98c561b59fc8, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.06	0.06
query4	1.68	0.10	0.10
query5	0.49	0.49	0.50
query6	1.31	0.67	0.65
query7	0.02	0.01	0.02
query8	0.04	0.02	0.02
query9	0.56	0.52	0.51
query10	0.58	0.58	0.57
query11	0.13	0.10	0.10
query12	0.13	0.10	0.10
query13	0.58	0.57	0.56
query14	0.74	0.75	0.77
query15	0.82	0.81	0.80
query16	0.37	0.36	0.38
query17	0.99	0.96	1.02
query18	0.26	0.28	0.25
query19	1.80	1.78	1.72
query20	0.02	0.01	0.01
query21	15.40	0.63	0.63
query22	2.82	3.67	2.57
query23	17.46	1.00	0.89
query24	2.37	0.33	0.58
query25	0.23	0.06	0.04
query26	0.26	0.14	0.13
query27	0.04	0.03	0.04
query28	11.98	0.85	0.83
query29	12.56	3.29	3.22
query30	0.60	0.58	0.61
query31	2.86	0.36	0.32
query32	3.94	0.45	0.44
query33	2.84	2.81	2.85
query34	15.51	4.28	4.28
query35	4.29	4.29	4.29
query36	1.08	1.02	1.02
query37	0.06	0.06	0.06
query38	0.04	0.03	0.03
query39	0.02	0.01	0.02
query40	0.16	0.13	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.02	0.03	0.03
Total cold run time: 105.49 s
Total hot run time: 30.79 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 13, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

Comment on lines +65 to +73
Plan mvPlan = originPlan.accept(new DefaultPlanRewriter<Object>() {
@Override
public Plan visit(Plan plan, Object context) {
if (plan instanceof LogicalResultSink) {
return ((LogicalResultSink<Plan>) plan).child();
}
return super.visit(plan, context);
}
}, null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor:
alternative 1:

new DefaultPlanRewriter<Object>() {
            @Override
            public Plan visitLogicalResultSink(LogicalResultSink sink, Object context) {
                return plan.child().accept(this, context);
            }
        }

alternative 2:

Plan mvPlan = originPlan.rewriteUp(p -> p instanceof LogicalResultSink ? p.child(0) : p)

@morrySnow morrySnow merged commit 9f628f3 into apache:master Mar 13, 2024
27 of 31 checks passed
yiguolei pushed a commit that referenced this pull request Mar 15, 2024
1. Use visitor to remove the result sink in mv plan
2. Remove useless code
3. Add more doc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants