[fix](index compaction) Support merge null_bitmap during index compaction #30352

qidaye · 2024-01-25T04:17:24Z

Proposed changes

null_bitmap file is not considered in index compaction process. This will lead wrong query result when doc is contain NULL values.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

qidaye · 2024-01-25T04:17:59Z

run buildall

github-actions · 2024-01-25T04:24:07Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-01-25T04:49:32Z

TPC-H: Total hot run time: 50059 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1265a89924ddd8bd97b892943416b4355f79e91e, data reload: false

------ Round 1 ----------------------------------
q1	17967	4443	4353	4353
q2	2025	150	143	143
q3	10278	1919	1910	1910
q4	10097	1226	1316	1226
q5	8552	3974	4017	3974
q6	229	122	120	120
q7	2028	1598	1578	1578
q8	9313	2735	2745	2735
q9	11157	10557	10418	10418
q10	8683	3531	3529	3529
q11	416	233	235	233
q12	457	301	299	299
q13	18358	3955	4035	3955
q14	347	326	330	326
q15	500	446	457	446
q16	709	597	589	589
q17	1138	975	966	966
q18	7331	7070	6928	6928
q19	1687	1561	1559	1559
q20	550	303	302	302
q21	4446	4080	4121	4080
q22	498	396	390	390
Total cold run time: 116766 ms
Total hot run time: 50059 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4362	4286	4282	4282
q2	324	230	222	222
q3	4203	4171	4166	4166
q4	2779	2762	2799	2762
q5	7284	7240	7191	7191
q6	240	120	120	120
q7	3225	2861	2840	2840
q8	4373	4496	4466	4466
q9	17184	17069	16963	16963
q10	4250	4298	4298	4298
q11	749	676	692	676
q12	1028	851	852	851
q13	6804	3708	3764	3708
q14	450	424	414	414
q15	508	445	456	445
q16	765	701	716	701
q17	3921	3874	3889	3874
q18	8838	8725	8745	8725
q19	1722	1699	1649	1649
q20	2374	2130	2138	2130
q21	8526	8468	8435	8435
q22	1041	935	953	935
Total cold run time: 84950 ms
Total hot run time: 79853 ms

doris-robot · 2024-01-25T05:04:02Z

TPC-DS: Total hot run time: 241169 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1265a89924ddd8bd97b892943416b4355f79e91e, data reload: false

query1	909	391	388	388
query2	6531	2141	2062	2062
query3	6913	201	197	197
query4	21236	17979	17899	17899
query5	19722	6288	6214	6214
query6	281	216	238	216
query7	4152	315	309	309
query8	274	247	227	227
query9	3127	2717	2670	2670
query10	411	285	303	285
query11	11270	10734	10674	10674
query12	120	70	75	70
query13	5664	635	648	635
query14	17492	13344	13656	13344
query15	381	234	239	234
query16	6449	267	265	265
query17	3175	863	887	863
query18	2274	413	406	406
query19	211	149	157	149
query20	76	74	75	74
query21	185	99	94	94
query22	5347	5149	5145	5145
query23	32555	31971	31931	31931
query24	7025	6553	6497	6497
query25	528	432	450	432
query26	535	166	163	163
query27	1870	296	299	296
query28	6123	2255	2229	2229
query29	2963	2715	2761	2715
query30	237	158	162	158
query31	890	769	746	746
query32	63	60	56	56
query33	391	264	250	250
query34	851	473	498	473
query35	1114	886	909	886
query36	1804	1396	1521	1396
query37	96	58	62	58
query38	3107	2927	2889	2889
query39	1359	1320	1333	1320
query40	210	96	99	96
query41	35	32	31	31
query42	86	84	86	84
query43	567	615	587	587
query44	1116	724	728	724
query45	243	227	229	227
query46	1230	982	956	956
query47	1804	1642	1838	1642
query48	1001	705	682	682
query49	598	388	374	374
query50	862	607	582	582
query51	5687	5525	5541	5525
query52	88	87	84	84
query53	443	332	312	312
query54	2410	2242	2257	2242
query55	80	75	74	74
query56	225	215	203	203
query57	1179	1130	1093	1093
query58	212	203	185	185
query59	3430	3229	3244	3229
query60	214	187	196	187
query61	86	82	88	82
query62	778	453	468	453
query63	474	341	340	340
query64	2061	1361	1281	1281
query65	3659	3604	3639	3604
query66	827	369	367	367
query67	16469	16803	15469	15469
query68	7656	640	661	640
query69	568	330	364	330
query70	1973	1786	1660	1660
query71	392	294	311	294
query72	4600	3357	3363	3357
query73	714	318	315	315
query74	6353	5881	5809	5809
query75	4395	3480	3528	3480
query76	4652	1178	1204	1178
query77	528	253	269	253
query78	32165	50480	53478	50480
query79	14932	602	625	602
query80	4285	383	382	382
query81	566	235	227	227
query82	1401	103	107	103
query83	324	140	134	134
query84	261	67	68	67
query85	1695	277	275	275
query86	451	368	404	368
query87	3236	3063	3001	3001
query88	7035	2323	2296	2296
query89	394	274	290	274
query90	2503	206	207	206
query91	148	114	115	114
query92	67	52	50	50
query93	5155	607	615	607
query94	1668	208	210	208
query95	1090	1055	1068	1055
query96	639	332	324	324
query97	6434	6389	6409	6389
query98	187	173	170	170
query99	3666	967	891	891
Total cold run time: 343918 ms
Total hot run time: 241169 ms

doris-robot · 2024-01-25T05:09:28Z

ClickBench: Total hot run time: 30.6 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1265a89924ddd8bd97b892943416b4355f79e91e, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.24	0.05	0.04
query4	1.82	0.07	0.07
query5	0.53	0.52	0.52
query6	1.26	0.62	0.62
query7	0.02	0.01	0.01
query8	0.03	0.03	0.02
query9	0.50	0.50	0.49
query10	0.54	0.53	0.54
query11	0.11	0.09	0.08
query12	0.11	0.09	0.09
query13	0.62	0.62	0.61
query14	0.78	0.77	0.78
query15	0.78	0.76	0.75
query16	0.36	0.40	0.35
query17	1.01	1.00	1.00
query18	0.21	0.25	0.26
query19	1.92	1.84	1.85
query20	0.02	0.01	0.02
query21	15.49	0.56	0.57
query22	1.84	1.93	1.32
query23	17.45	0.95	0.95
query24	6.67	4.32	0.89
query25	2.38	0.16	0.16
query26	0.15	0.14	0.14
query27	0.15	0.16	0.14
query28	4.32	0.71	0.73
query29	12.78	2.42	2.30
query30	0.60	0.54	0.50
query31	2.82	0.39	0.38
query32	3.41	0.50	0.49
query33	3.08	3.09	3.10
query34	15.26	4.79	4.80
query35	4.85	4.83	4.85
query36	1.03	1.01	1.01
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.01	0.02	0.02
query40	0.16	0.13	0.14
query41	0.07	0.01	0.02
query42	0.01	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 103.58 s
Total hot run time: 30.6 s

doris-robot · 2024-01-25T05:12:40Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1265a89924ddd8bd97b892943416b4355f79e91e with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

doris-robot · 2024-01-25T05:14:51Z

TeamCity be ut coverage result:
Function Coverage: 37.83% (8033/21232)
Line Coverage: 29.54% (65453/221579)
Region Coverage: 28.95% (33607/116071)
Branch Coverage: 24.82% (17245/69484)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1265a89924ddd8bd97b892943416b4355f79e91e_1265a89924ddd8bd97b892943416b4355f79e91e/report/index.html

qidaye · 2024-01-25T05:34:52Z

run buildall

github-actions · 2024-01-25T05:41:09Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-01-25T05:56:26Z

TeamCity be ut coverage result:
Function Coverage: 37.83% (8032/21232)
Line Coverage: 29.53% (65445/221588)
Region Coverage: 28.95% (33606/116076)
Branch Coverage: 24.82% (17245/69488)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bc90a78782f95aec934baf3fdf9a5b0011e976f9_bc90a78782f95aec934baf3fdf9a5b0011e976f9/report/index.html

doris-robot · 2024-01-25T06:03:46Z

TPC-H: Total hot run time: 49824 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc90a78782f95aec934baf3fdf9a5b0011e976f9, data reload: false

------ Round 1 ----------------------------------
q1	17611	4433	4402	4402
q2	2056	146	138	138
q3	10450	1884	1893	1884
q4	10392	1269	1327	1269
q5	8369	4302	3968	3968
q6	231	122	122	122
q7	2011	1620	1583	1583
q8	9533	2704	2710	2704
q9	13161	10492	10386	10386
q10	8654	3525	3480	3480
q11	407	235	247	235
q12	461	298	308	298
q13	18360	3984	4075	3984
q14	351	332	339	332
q15	497	454	463	454
q16	679	598	591	591
q17	1123	918	942	918
q18	7227	6792	6915	6792
q19	1686	1567	1482	1482
q20	527	308	315	308
q21	4497	4107	4100	4100
q22	498	399	394	394
Total cold run time: 118781 ms
Total hot run time: 49824 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4282	4315	4303	4303
q2	321	226	221	221
q3	4149	4193	4171	4171
q4	2763	2740	2761	2740
q5	7306	7252	7270	7252
q6	233	116	118	116
q7	3275	2896	2856	2856
q8	4408	4540	4524	4524
q9	17288	17131	17058	17058
q10	4278	4397	4436	4397
q11	795	722	722	722
q12	1074	922	932	922
q13	6452	3872	3926	3872
q14	477	444	454	444
q15	508	474	470	470
q16	807	765	799	765
q17	3847	3927	3879	3879
q18	8887	8842	8833	8833
q19	1752	1682	1672	1672
q20	2434	2132	2137	2132
q21	8601	8549	8554	8549
q22	1077	931	961	931
Total cold run time: 85014 ms
Total hot run time: 80829 ms

doris-robot · 2024-01-25T06:18:18Z

TPC-DS: Total hot run time: 239109 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc90a78782f95aec934baf3fdf9a5b0011e976f9, data reload: false

query1	897	385	378	378
query2	6542	2207	2070	2070
query3	6916	195	194	194
query4	21425	17994	18045	17994
query5	19735	6272	6265	6265
query6	295	216	229	216
query7	4182	300	301	300
query8	273	258	239	239
query9	3156	2720	2630	2630
query10	427	277	300	277
query11	11370	10694	10719	10694
query12	122	76	77	76
query13	5651	637	629	629
query14	17877	13306	13214	13214
query15	378	232	247	232
query16	6431	260	263	260
query17	3031	878	880	878
query18	2247	413	404	404
query19	205	147	148	147
query20	78	74	76	74
query21	185	103	97	97
query22	5213	5264	5115	5115
query23	32427	31895	31952	31895
query24	6908	6552	6480	6480
query25	529	421	449	421
query26	524	166	161	161
query27	1816	301	298	298
query28	6106	2227	2218	2218
query29	3000	2692	2670	2670
query30	247	157	164	157
query31	896	705	745	705
query32	65	60	59	59
query33	382	249	242	242
query34	826	478	491	478
query35	1091	941	879	879
query36	1507	1740	1568	1568
query37	94	64	62	62
query38	3087	2933	2940	2933
query39	1375	1307	1306	1306
query40	208	95	97	95
query41	33	32	33	32
query42	85	94	90	90
query43	680	679	572	572
query44	1124	717	712	712
query45	245	229	235	229
query46	1230	955	963	955
query47	1882	1845	2084	1845
query48	987	683	647	647
query49	610	353	379	353
query50	855	610	600	600
query51	5636	5501	5547	5501
query52	95	79	85	79
query53	444	326	307	307
query54	2461	2230	2197	2197
query55	79	76	83	76
query56	228	205	200	200
query57	1138	1025	1097	1025
query58	222	209	190	190
query59	3441	3351	3001	3001
query60	223	202	199	199
query61	84	83	87	83
query62	776	473	504	473
query63	478	329	327	327
query64	2013	1303	1306	1303
query65	3696	3645	3618	3618
query66	810	364	359	359
query67	15727	15396	15177	15177
query68	8860	665	673	665
query69	570	337	344	337
query70	1916	1639	1849	1639
query71	413	298	307	298
query72	4616	3327	3334	3327
query73	730	320	314	314
query74	6273	5950	5838	5838
query75	4912	3545	3576	3545
query76	5467	1151	1181	1151
query77	882	250	244	244
query78	32388	53652	48883	48883
query79	15573	622	611	611
query80	4698	391	377	377
query81	565	227	230	227
query82	1573	95	95	95
query83	333	129	128	128
query84	259	69	69	69
query85	1829	280	278	278
query86	447	393	393	393
query87	3226	3003	3006	3003
query88	7057	2302	2315	2302
query89	453	282	285	282
query90	2503	203	210	203
query91	151	125	117	117
query92	63	55	52	52
query93	6463	559	573	559
query94	1670	198	206	198
query95	1123	1082	1075	1075
query96	658	320	323	320
query97	6488	6370	6336	6336
query98	198	172	176	172
query99	3688	911	1002	911
Total cold run time: 349051 ms
Total hot run time: 239109 ms

doris-robot · 2024-01-25T06:23:44Z

ClickBench: Total hot run time: 30.69 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bc90a78782f95aec934baf3fdf9a5b0011e976f9, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.24	0.04	0.05
query4	1.83	0.06	0.07
query5	0.52	0.52	0.51
query6	1.24	0.62	0.61
query7	0.01	0.01	0.01
query8	0.04	0.02	0.02
query9	0.52	0.49	0.49
query10	0.55	0.54	0.53
query11	0.12	0.08	0.09
query12	0.11	0.09	0.09
query13	0.62	0.62	0.60
query14	0.77	0.79	0.77
query15	0.77	0.75	0.76
query16	0.39	0.36	0.37
query17	1.02	1.00	1.00
query18	0.24	0.25	0.24
query19	1.94	1.85	1.89
query20	0.01	0.01	0.01
query21	15.51	0.57	0.57
query22	1.80	2.16	1.20
query23	16.97	0.91	0.87
query24	4.11	3.35	1.42
query25	2.12	0.16	0.14
query26	0.15	0.14	0.15
query27	0.14	0.15	0.14
query28	7.22	0.72	0.70
query29	12.75	2.14	2.27
query30	0.54	0.51	0.51
query31	2.82	0.39	0.38
query32	3.40	0.50	0.48
query33	3.10	3.04	3.11
query34	15.26	4.81	4.80
query35	4.84	4.83	4.84
query36	1.05	1.01	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.01	0.02	0.02
query43	0.03	0.01	0.02
Total cold run time: 103.18 s
Total hot run time: 30.69 s

doris-robot · 2024-01-25T06:26:56Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit bc90a78782f95aec934baf3fdf9a5b0011e976f9 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

…tion Spport merge null_bitmap during index compaction. We read the source indices null_bitmap files and write them to new ones according to the translation vector doc id mapping. fix cl_test compile error

qidaye · 2024-01-25T06:31:06Z

run buildall

github-actions · 2024-01-25T06:37:52Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-01-25T06:55:21Z

TPC-H: Total hot run time: 49945 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 09065f82c1cfade627ec29116b4f69e85441f359, data reload: false

------ Round 1 ----------------------------------
q1	17532	4362	4314	4314
q2	2032	154	141	141
q3	10269	1916	1963	1916
q4	10110	1249	1419	1249
q5	8550	3960	4005	3960
q6	234	123	126	123
q7	2045	1602	1581	1581
q8	9265	2728	2739	2728
q9	10841	11224	10460	10460
q10	8656	3542	3524	3524
q11	432	240	233	233
q12	480	298	311	298
q13	18373	3959	4055	3959
q14	352	328	320	320
q15	514	456	457	456
q16	702	589	592	589
q17	1124	986	1001	986
q18	7327	6830	6865	6830
q19	1688	1594	1488	1488
q20	535	303	307	303
q21	4443	4101	4146	4101
q22	486	386	406	386
Total cold run time: 115990 ms
Total hot run time: 49945 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4322	4298	4314	4298
q2	318	226	232	226
q3	4222	4176	4198	4176
q4	2762	2755	2749	2749
q5	7274	7260	7179	7179
q6	236	124	122	122
q7	3255	2876	2828	2828
q8	4422	4510	4530	4510
q9	17145	17051	17007	17007
q10	4243	4264	4311	4264
q11	758	658	675	658
q12	1040	847	855	847
q13	6704	3752	3748	3748
q14	464	433	421	421
q15	494	449	447	447
q16	784	706	723	706
q17	3898	3852	3820	3820
q18	8856	8832	8973	8832
q19	1705	1705	1657	1657
q20	2426	2139	2111	2111
q21	8571	8565	8393	8393
q22	1073	951	961	951
Total cold run time: 84972 ms
Total hot run time: 79950 ms

doris-robot · 2024-01-25T07:02:15Z

TeamCity be ut coverage result:
Function Coverage: 37.84% (8034/21232)
Line Coverage: 29.55% (65470/221588)
Region Coverage: 28.96% (33620/116076)
Branch Coverage: 24.83% (17254/69488)
Coverage Report: http://coverage.selectdb-in.cc/coverage/09065f82c1cfade627ec29116b4f69e85441f359_09065f82c1cfade627ec29116b4f69e85441f359/report/index.html

doris-robot · 2024-01-25T07:09:29Z

TPC-DS: Total hot run time: 235540 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 09065f82c1cfade627ec29116b4f69e85441f359, data reload: false

query1	919	388	379	379
query2	6512	2203	1943	1943
query3	6908	203	197	197
query4	20183	18068	17880	17880
query5	19719	6265	6227	6227
query6	276	212	223	212
query7	4144	292	291	291
query8	262	252	247	247
query9	3141	2719	2659	2659
query10	403	286	285	285
query11	11374	10763	10714	10714
query12	122	75	72	72
query13	5657	665	636	636
query14	17658	13266	13480	13266
query15	400	231	246	231
query16	6444	266	267	266
query17	3218	853	877	853
query18	2266	400	408	400
query19	207	144	152	144
query20	74	76	75	75
query21	187	97	91	91
query22	5340	5199	5098	5098
query23	32415	31793	31992	31793
query24	6917	6560	6476	6476
query25	510	431	426	426
query26	521	159	157	157
query27	1875	291	293	291
query28	6124	2241	2224	2224
query29	2818	2754	2741	2741
query30	237	162	160	160
query31	889	725	791	725
query32	65	59	57	57
query33	393	249	252	249
query34	849	461	484	461
query35	1114	891	909	891
query36	1474	1555	1461	1461
query37	87	58	56	56
query38	3096	2917	2890	2890
query39	1365	1325	1321	1321
query40	199	98	91	91
query41	37	35	31	31
query42	88	87	90	87
query43	651	598	576	576
query44	1116	729	720	720
query45	241	227	230	227
query46	1234	993	968	968
query47	1775	1717	1754	1717
query48	1013	693	685	685
query49	621	379	368	368
query50	868	593	595	593
query51	5631	5531	5471	5471
query52	87	77	79	77
query53	450	331	321	321
query54	2448	2226	2279	2226
query55	82	78	72	72
query56	211	199	200	199
query57	1104	1100	1004	1004
query58	198	190	208	190
query59	3512	3089	3155	3089
query60	204	186	192	186
query61	83	84	80	80
query62	805	521	489	489
query63	479	341	336	336
query64	1998	1240	1258	1240
query65	3644	3636	3637	3636
query66	828	371	358	358
query67	17435	15777	15024	15024
query68	5155	634	653	634
query69	536	350	342	342
query70	1768	1625	2047	1625
query71	374	302	314	302
query72	4246	3270	3311	3270
query73	706	317	319	317
query74	6386	5898	5922	5898
query75	4079	3525	3450	3450
query76	2978	1218	1219	1218
query77	421	250	246	246
query78	30815	49004	46012	46012
query79	14364	616	614	614
query80	4582	375	374	374
query81	561	233	228	228
query82	467	101	92	92
query83	332	134	128	128
query84	257	66	67	66
query85	2533	272	271	271
query86	487	448	387	387
query87	3288	2996	3027	2996
query88	6185	2331	2307	2307
query89	370	302	272	272
query90	2484	207	209	207
query91	162	117	112	112
query92	66	52	51	51
query93	2737	590	584	584
query94	1862	208	207	207
query95	1097	1061	1052	1052
query96	629	332	324	324
query97	6504	6304	6403	6304
query98	185	178	166	166
query99	3891	878	941	878
Total cold run time: 333712 ms
Total hot run time: 235540 ms

doris-robot · 2024-01-25T07:14:57Z

ClickBench: Total hot run time: 30.82 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 09065f82c1cfade627ec29116b4f69e85441f359, data reload: false

query1	0.02	0.02	0.01
query2	0.05	0.03	0.02
query3	0.25	0.05	0.04
query4	1.82	0.07	0.07
query5	0.53	0.52	0.52
query6	1.26	0.61	0.61
query7	0.01	0.01	0.01
query8	0.04	0.02	0.02
query9	0.53	0.48	0.47
query10	0.54	0.51	0.54
query11	0.12	0.09	0.10
query12	0.11	0.09	0.10
query13	0.61	0.61	0.60
query14	0.78	0.78	0.79
query15	0.79	0.75	0.77
query16	0.36	0.36	0.37
query17	1.04	1.02	1.00
query18	0.21	0.28	0.28
query19	1.95	1.77	1.85
query20	0.01	0.01	0.01
query21	15.48	0.62	0.57
query22	1.69	1.68	2.18
query23	17.12	1.10	0.84
query24	5.68	1.02	2.90
query25	2.28	0.16	0.14
query26	0.14	0.12	0.15
query27	0.14	0.15	0.14
query28	6.15	0.73	0.71
query29	12.62	2.24	2.37
query30	0.57	0.53	0.52
query31	2.82	0.38	0.37
query32	3.44	0.50	0.50
query33	3.09	3.05	3.06
query34	15.28	4.79	4.81
query35	4.85	4.87	4.84
query36	1.08	1.02	1.02
query37	0.05	0.05	0.05
query38	0.04	0.02	0.02
query39	0.01	0.02	0.01
query40	0.16	0.15	0.14
query41	0.06	0.02	0.02
query42	0.02	0.01	0.01
query43	0.02	0.02	0.01
Total cold run time: 103.82 s
Total hot run time: 30.82 s

doris-robot · 2024-01-25T07:18:09Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 09065f82c1cfade627ec29116b4f69e85441f359 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

qidaye · 2024-01-25T07:22:51Z

run p0

qidaye · 2024-01-25T07:59:51Z

run p0

…tion apache#30326 (apache#30352)

github-actions bot added the kind/test label Jan 25, 2024

qidaye force-pushed the pick_merge_null_bitmap branch from 1265a89 to bc90a78 Compare January 25, 2024 05:34

[fix](index compaction) Support merge null_bitmap during index compac…

09065f8

…tion Spport merge null_bitmap during index compaction. We read the source indices null_bitmap files and write them to new ones according to the translation vector doc id mapping. fix cl_test compile error

qidaye force-pushed the pick_merge_null_bitmap branch from bc90a78 to 09065f8 Compare January 25, 2024 06:30

xiaokang merged commit 4b7a1ac into apache:branch-2.0 Jan 31, 2024
22 of 25 checks passed

qidaye deleted the pick_merge_null_bitmap branch February 2, 2024 06:32

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024

[fix](index compaction) Support merge null_bitmap during index compac…

c980a49

…tion apache#30326 (apache#30352)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](index compaction) Support merge null_bitmap during index compaction #30352

[fix](index compaction) Support merge null_bitmap during index compaction #30352

qidaye commented Jan 25, 2024

qidaye commented Jan 25, 2024

github-actions bot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

qidaye commented Jan 25, 2024

github-actions bot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

qidaye commented Jan 25, 2024

github-actions bot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

qidaye commented Jan 25, 2024

qidaye commented Jan 25, 2024

[fix](index compaction) Support merge null_bitmap during index compaction #30352

[fix](index compaction) Support merge null_bitmap during index compaction #30352

Conversation

qidaye commented Jan 25, 2024

Proposed changes

Further comments

qidaye commented Jan 25, 2024

github-actions bot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

qidaye commented Jan 25, 2024

github-actions bot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

qidaye commented Jan 25, 2024

github-actions bot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

doris-robot commented Jan 25, 2024

qidaye commented Jan 25, 2024

qidaye commented Jan 25, 2024