Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[opt](exec) opt const expr exec in set operator #34823

Merged
merged 2 commits into from
May 16, 2024
Merged

Conversation

HappenLee
Copy link
Contributor

Proposed changes

before:

(select 0 from lineorder) intersect (select null); 

cost: 14s

after:

(select 0 from lineorder) intersect (select null);

cost: 4s

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@HappenLee
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40873 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 15c3255f147a1076b4d309aa5f72173d71124255, data reload: false

------ Round 1 ----------------------------------
q1	17698	4443	4260	4260
q2	2025	187	188	187
q3	10557	1284	1247	1247
q4	10786	838	860	838
q5	7776	2686	2759	2686
q6	217	130	129	129
q7	1010	584	575	575
q8	9480	2139	2081	2081
q9	9290	6708	6637	6637
q10	9064	3674	3759	3674
q11	467	251	234	234
q12	413	214	216	214
q13	17803	2998	2934	2934
q14	264	217	215	215
q15	514	476	468	468
q16	515	388	380	380
q17	963	678	663	663
q18	8067	7462	7453	7453
q19	2704	1573	1452	1452
q20	646	302	291	291
q21	5112	3976	4049	3976
q22	359	291	279	279
Total cold run time: 115730 ms
Total hot run time: 40873 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4289	4190	4150	4150
q2	385	264	268	264
q3	2949	2780	2717	2717
q4	1913	1584	1526	1526
q5	5268	5253	5328	5253
q6	221	120	120	120
q7	2268	1887	1855	1855
q8	3210	3346	3347	3346
q9	8383	8338	8342	8338
q10	3870	3699	3645	3645
q11	563	470	505	470
q12	753	579	605	579
q13	16287	2982	2941	2941
q14	289	265	268	265
q15	522	476	472	472
q16	487	419	404	404
q17	1778	1494	1466	1466
q18	7640	7678	7450	7450
q19	1642	1522	1541	1522
q20	1979	1783	1773	1773
q21	5018	4912	4852	4852
q22	594	506	495	495
Total cold run time: 70308 ms
Total hot run time: 53903 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.67% (8982/25180)
Line Coverage: 27.33% (74272/271761)
Region Coverage: 26.56% (38375/144468)
Branch Coverage: 23.38% (19575/83708)
Coverage Report: http://coverage.selectdb-in.cc/coverage/15c3255f147a1076b4d309aa5f72173d71124255_15c3255f147a1076b4d309aa5f72173d71124255/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 187838 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 15c3255f147a1076b4d309aa5f72173d71124255, data reload: false

query1	952	411	396	396
query2	6795	2368	2352	2352
query3	6654	213	208	208
query4	22700	21111	21591	21111
query5	4197	408	425	408
query6	251	176	177	176
query7	4599	298	300	298
query8	237	188	188	188
query9	8720	2436	2410	2410
query10	441	247	249	247
query11	14694	14297	14163	14163
query12	136	91	88	88
query13	1655	383	372	372
query14	10307	8349	7776	7776
query15	273	175	174	174
query16	8082	264	259	259
query17	1857	546	542	542
query18	1921	277	270	270
query19	203	145	149	145
query20	93	84	85	84
query21	192	145	129	129
query22	5112	5046	4979	4979
query23	34460	33485	33472	33472
query24	12099	2983	2917	2917
query25	634	369	389	369
query26	1684	159	154	154
query27	2938	315	329	315
query28	7305	2050	2047	2047
query29	985	612	585	585
query30	326	169	179	169
query31	962	743	738	738
query32	91	52	53	52
query33	745	242	243	242
query34	1081	469	488	469
query35	795	693	659	659
query36	1075	886	921	886
query37	276	70	72	70
query38	2891	2801	2797	2797
query39	1609	1588	1568	1568
query40	304	125	121	121
query41	45	44	42	42
query42	103	100	97	97
query43	633	576	559	559
query44	1182	717	731	717
query45	262	251	264	251
query46	1079	729	700	700
query47	1991	1888	1879	1879
query48	370	303	305	303
query49	1181	390	392	390
query50	765	381	393	381
query51	6856	6764	6798	6764
query52	100	93	84	84
query53	350	286	279	279
query54	916	433	440	433
query55	71	72	72	72
query56	236	216	227	216
query57	1255	1139	1170	1139
query58	230	206	205	205
query59	3458	3120	3291	3120
query60	265	246	246	246
query61	109	109	119	109
query62	669	505	476	476
query63	306	281	287	281
query64	9821	7519	7415	7415
query65	3168	3078	3099	3078
query66	1351	349	338	338
query67	15480	14961	15122	14961
query68	6139	543	535	535
query69	524	303	307	303
query70	1162	1125	1137	1125
query71	468	267	269	267
query72	7463	2556	2387	2387
query73	725	325	327	325
query74	6660	6199	6271	6199
query75	4048	2634	2659	2634
query76	4386	974	928	928
query77	612	266	260	260
query78	10617	10210	10186	10186
query79	1943	504	511	504
query80	1295	426	425	425
query81	509	245	240	240
query82	708	98	97	97
query83	191	174	165	165
query84	269	86	84	84
query85	1410	273	258	258
query86	401	305	304	304
query87	3319	3155	3157	3155
query88	4007	2501	2435	2435
query89	487	376	387	376
query90	1979	183	184	183
query91	123	95	95	95
query92	57	50	49	49
query93	3145	507	494	494
query94	1121	182	180	180
query95	380	305	301	301
query96	599	275	269	269
query97	3194	3062	3021	3021
query98	240	225	209	209
query99	1265	921	905	905
Total cold run time: 292029 ms
Total hot run time: 187838 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.5 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 15c3255f147a1076b4d309aa5f72173d71124255, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.07	0.06
query4	1.65	0.10	0.09
query5	0.50	0.51	0.51
query6	1.14	0.74	0.72
query7	0.02	0.01	0.01
query8	0.05	0.05	0.04
query9	0.53	0.48	0.50
query10	0.55	0.54	0.54
query11	0.16	0.12	0.12
query12	0.14	0.11	0.11
query13	0.59	0.59	0.60
query14	0.79	0.80	0.76
query15	0.84	0.82	0.82
query16	0.36	0.34	0.37
query17	0.95	1.02	0.97
query18	0.23	0.27	0.24
query19	1.73	1.74	1.71
query20	0.02	0.01	0.01
query21	15.43	0.71	0.68
query22	4.14	7.50	2.02
query23	18.27	1.41	1.29
query24	1.99	0.23	0.22
query25	0.17	0.08	0.08
query26	0.26	0.16	0.17
query27	0.08	0.08	0.07
query28	13.26	1.00	0.98
query29	12.66	3.30	3.23
query30	0.26	0.09	0.08
query31	2.83	0.38	0.37
query32	3.31	0.46	0.47
query33	2.86	2.79	2.88
query34	17.08	4.41	4.40
query35	4.54	4.52	4.44
query36	0.64	0.45	0.48
query37	0.18	0.15	0.15
query38	0.16	0.14	0.14
query39	0.04	0.03	0.04
query40	0.17	0.14	0.17
query41	0.10	0.04	0.06
query42	0.06	0.05	0.06
query43	0.05	0.04	0.04
Total cold run time: 109.15 s
Total hot run time: 30.5 s

@HappenLee
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.70% (8998/25202)
Line Coverage: 27.38% (74462/272006)
Region Coverage: 26.61% (38487/144652)
Branch Coverage: 23.43% (19644/83832)
Coverage Report: http://coverage.selectdb-in.cc/coverage/254bb119cc2b364f7a36295a058c147d2d23661c_254bb119cc2b364f7a36295a058c147d2d23661c/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 182712 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 254bb119cc2b364f7a36295a058c147d2d23661c, data reload: false

query1	913	384	369	369
query2	7416	2413	2380	2380
query3	6662	221	222	221
query4	23301	21837	22160	21837
query5	3826	442	433	433
query6	285	180	177	177
query7	4732	304	307	304
query8	244	196	193	193
query9	8545	2482	2486	2482
query10	408	266	265	265
query11	15214	14884	14702	14702
query12	126	90	94	90
query13	1555	379	374	374
query14	10088	7630	8288	7630
query15	250	166	169	166
query16	8194	251	257	251
query17	1696	539	542	539
query18	2094	276	273	273
query19	199	144	147	144
query20	89	82	89	82
query21	192	130	122	122
query22	5050	4818	4848	4818
query23	34598	33711	33551	33551
query24	10676	2842	2970	2842
query25	604	383	367	367
query26	1353	156	157	156
query27	2923	323	315	315
query28	7510	2050	2036	2036
query29	866	622	607	607
query30	278	181	171	171
query31	963	744	734	734
query32	90	50	55	50
query33	733	246	247	246
query34	1051	488	484	484
query35	798	677	674	674
query36	1083	932	884	884
query37	133	68	68	68
query38	2823	2783	2776	2776
query39	1591	1544	1552	1544
query40	204	124	125	124
query41	47	45	43	43
query42	101	95	93	93
query43	600	539	563	539
query44	1204	738	734	734
query45	270	261	248	248
query46	1081	755	758	755
query47	1964	1862	1849	1849
query48	400	305	296	296
query49	874	416	403	403
query50	758	393	410	393
query51	6821	6782	6851	6782
query52	104	91	91	91
query53	354	296	286	286
query54	857	421	421	421
query55	77	74	75	74
query56	243	217	223	217
query57	1215	1158	1182	1158
query58	214	196	201	196
query59	3550	3256	3168	3168
query60	252	235	237	235
query61	91	89	87	87
query62	619	453	459	453
query63	309	289	290	289
query64	4467	1785	1724	1724
query65	3176	3025	3060	3025
query66	763	327	345	327
query67	15378	14878	14905	14878
query68	4606	540	540	540
query69	445	316	309	309
query70	1175	1107	1164	1107
query71	374	274	277	274
query72	7259	2536	2381	2381
query73	727	322	326	322
query74	6479	6102	6156	6102
query75	3301	2679	2616	2616
query76	2676	991	852	852
query77	384	312	271	271
query78	10570	10083	10087	10083
query79	2453	526	514	514
query80	1069	441	454	441
query81	548	241	240	240
query82	741	101	92	92
query83	235	167	165	165
query84	236	92	82	82
query85	1323	270	261	261
query86	448	311	304	304
query87	3262	3120	3132	3120
query88	4062	2354	2353	2353
query89	466	396	390	390
query90	1820	188	193	188
query91	127	98	100	98
query92	68	48	48	48
query93	1627	491	494	491
query94	947	183	184	183
query95	393	301	307	301
query96	580	264	266	264
query97	3152	2955	3016	2955
query98	236	220	217	217
query99	1109	903	917	903
Total cold run time: 278043 ms
Total hot run time: 182712 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.43 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 254bb119cc2b364f7a36295a058c147d2d23661c, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.05
query4	1.68	0.08	0.07
query5	0.50	0.50	0.50
query6	1.12	0.72	0.73
query7	0.01	0.01	0.01
query8	0.05	0.04	0.05
query9	0.54	0.50	0.49
query10	0.54	0.54	0.54
query11	0.16	0.12	0.11
query12	0.14	0.12	0.11
query13	0.59	0.59	0.61
query14	0.78	0.77	0.77
query15	0.83	0.81	0.81
query16	0.37	0.35	0.35
query17	0.95	1.03	1.03
query18	0.24	0.23	0.26
query19	1.88	1.77	1.75
query20	0.01	0.01	0.01
query21	15.74	0.65	0.66
query22	4.32	7.35	1.72
query23	18.25	1.37	1.33
query24	1.57	0.31	0.21
query25	0.13	0.08	0.09
query26	0.28	0.17	0.17
query27	0.08	0.07	0.07
query28	13.36	1.01	0.99
query29	13.40	3.32	3.27
query30	0.24	0.05	0.07
query31	2.87	0.40	0.38
query32	3.27	0.48	0.48
query33	2.84	2.87	2.84
query34	17.17	4.47	4.41
query35	4.54	4.50	4.56
query36	0.68	0.49	0.45
query37	0.17	0.15	0.15
query38	0.16	0.15	0.14
query39	0.05	0.03	0.04
query40	0.16	0.14	0.15
query41	0.09	0.05	0.04
query42	0.05	0.06	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.21 s
Total hot run time: 30.43 s

@HappenLee
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.70% (8997/25199)
Line Coverage: 27.37% (74447/271976)
Region Coverage: 26.61% (38486/144634)
Branch Coverage: 23.42% (19633/83818)
Coverage Report: http://coverage.selectdb-in.cc/coverage/be9e1441843676d0c80dcec9ddb0ead265c81e7a_be9e1441843676d0c80dcec9ddb0ead265c81e7a/report/index.html

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 16, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@HappenLee HappenLee merged commit 7fa7efb into apache:master May 16, 2024
26 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants