Skip to content

branch-3.0: [fix](mtmv) Fix mv rewrite fail when base table add column #44867#44955

Merged
morrySnow merged 1 commit intobranch-3.0from
auto-pick-44867-branch-3.0
Dec 6, 2024
Merged

branch-3.0: [fix](mtmv) Fix mv rewrite fail when base table add column #44867#44955
morrySnow merged 1 commit intobranch-3.0from
auto-pick-44867-branch-3.0

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Dec 3, 2024

Cherry-picked from #44867

### What problem does this PR solve?

Problem Summary:

Fix mv rewrite fail when base table add column
Such as mv def is as following:

            select
            o_orderstatus,
            l_suppkey,
            sum_merge(sum_state(o_shippriority)),
            group_concat_merge(group_concat_state(o_orderstatus)),
            avg_merge(avg_state(l_linenumber)),
            max_by_merge(max_by_state(O_COMMENT,o_totalprice)),
            count_merge(count_state(l_orderkey)),
            multi_distinct_count_merge(multi_distinct_count_state(l_shipmode))
            from lineitem
            left join orders 
            on l_orderkey = o_orderkey and l_shipdate = o_orderdate
            group by
            o_orderstatus,
            l_suppkey
            order by o_orderstatus;

if add column to table orders, would cause rewrite fail

### Release note

Fix mv rewrite fail when base table add column
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Dec 3, 2024
@dataroaring dataroaring reopened this Dec 3, 2024
@doris-robot
Copy link

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40764 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b2560c93d4b29cfa584b0bbb33e8345a5b135b0e, data reload: false

------ Round 1 ----------------------------------
q1	17566	7390	8074	7390
q2	2053	175	192	175
q3	10669	1908	1142	1142
q4	10565	834	730	730
q5	7729	2834	2828	2828
q6	234	148	147	147
q7	967	615	610	610
q8	9599	1962	2061	1962
q9	7885	6422	6384	6384
q10	6999	2320	2338	2320
q11	462	269	258	258
q12	406	224	207	207
q13	17763	2944	3006	2944
q14	232	205	198	198
q15	559	520	513	513
q16	692	612	598	598
q17	981	549	540	540
q18	7149	6459	6550	6459
q19	3909	1063	988	988
q20	502	202	200	200
q21	3910	3215	3209	3209
q22	1072	962	976	962
Total cold run time: 111903 ms
Total hot run time: 40764 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7340	7263	7284	7263
q2	342	240	237	237
q3	3027	2839	2925	2839
q4	2055	1751	1702	1702
q5	5626	5667	5718	5667
q6	224	146	147	146
q7	2212	1748	1787	1748
q8	3288	3376	3473	3376
q9	8853	8804	8770	8770
q10	3547	3511	3483	3483
q11	584	507	504	504
q12	788	586	599	586
q13	16480	3121	3110	3110
q14	312	282	296	282
q15	576	517	522	517
q16	711	656	670	656
q17	1814	1624	1607	1607
q18	7972	7929	7455	7455
q19	2551	1516	1577	1516
q20	2039	1845	1841	1841
q21	9020	5135	5281	5135
q22	1087	1019	1033	1019
Total cold run time: 80448 ms
Total hot run time: 59459 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194136 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b2560c93d4b29cfa584b0bbb33e8345a5b135b0e, data reload: false

query1	1226	928	926	926
query2	6252	2090	2044	2044
query3	10871	4168	4103	4103
query4	66679	28129	23569	23569
query5	5762	456	442	442
query6	485	188	178	178
query7	6085	324	313	313
query8	327	232	230	230
query9	9393	2717	2704	2704
query10	528	274	263	263
query11	17859	15208	15856	15208
query12	159	99	107	99
query13	1583	465	419	419
query14	11557	6755	7311	6755
query15	213	187	177	177
query16	7352	490	472	472
query17	1316	581	602	581
query18	1899	335	337	335
query19	214	163	160	160
query20	124	112	110	110
query21	208	104	108	104
query22	4622	4608	4414	4414
query23	34672	33956	33978	33956
query24	5839	2957	2952	2952
query25	547	431	416	416
query26	684	185	171	171
query27	2062	303	305	303
query28	4518	2614	2527	2527
query29	706	448	433	433
query30	239	164	164	164
query31	1004	811	856	811
query32	70	54	53	53
query33	461	283	271	271
query34	910	519	538	519
query35	858	731	724	724
query36	1092	957	922	922
query37	118	66	80	66
query38	4067	4003	3992	3992
query39	1505	1499	1439	1439
query40	202	99	99	99
query41	47	48	47	47
query42	115	100	95	95
query43	534	492	480	480
query44	1180	800	810	800
query45	181	165	167	165
query46	1159	757	746	746
query47	1985	1897	1864	1864
query48	462	361	384	361
query49	714	388	380	380
query50	843	416	432	416
query51	7258	7191	7332	7191
query52	99	92	88	88
query53	261	186	183	183
query54	558	458	448	448
query55	74	72	73	72
query56	248	229	227	227
query57	1188	1103	1091	1091
query58	200	202	192	192
query59	3093	2937	2806	2806
query60	273	254	257	254
query61	132	105	102	102
query62	761	647	662	647
query63	204	191	192	191
query64	1764	650	634	634
query65	3246	3149	3166	3149
query66	744	317	308	308
query67	16000	15301	15415	15301
query68	4573	557	537	537
query69	426	268	254	254
query70	1160	1075	1099	1075
query71	392	253	255	253
query72	6548	2432	3846	2432
query73	747	341	337	337
query74	10302	8835	8875	8835
query75	3316	2616	2644	2616
query76	2844	1106	1085	1085
query77	463	258	265	258
query78	10722	9691	9491	9491
query79	8764	582	572	572
query80	1941	425	410	410
query81	542	247	239	239
query82	1404	119	118	118
query83	253	142	139	139
query84	297	86	75	75
query85	1745	297	286	286
query86	479	255	298	255
query87	4607	4285	4235	4235
query88	5756	2402	2402	2402
query89	548	287	289	287
query90	2071	185	174	174
query91	192	145	143	143
query92	59	46	46	46
query93	6612	524	534	524
query94	905	296	257	257
query95	348	237	245	237
query96	623	280	291	280
query97	3283	3135	3142	3135
query98	222	205	207	205
query99	1860	1283	1282	1282
Total cold run time: 341595 ms
Total hot run time: 194136 ms

@morrySnow morrySnow merged commit 0734cc3 into branch-3.0 Dec 6, 2024
@github-actions github-actions bot deleted the auto-pick-44867-branch-3.0 branch December 6, 2024 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants