B-Tree keys histogram is empty or invalid in some rare cases

[kunga]

In some databases we've seen a big shards that don't split because of an invalid keys histogram

https://st.yandex-team.ru/YDBOPS-10140

[kunga]

Gathered logs shows that in a strange way an end node event goes before corresponding begin event, so we do not load and split it:

added slice [0, 18275268) data size (3.43GiB - 0B) => 3.5GiB

...

slicing node Part: [72075186767136724:1716:7548798:1:12288:8388608:0] PageId: 526222 Level: 3 BeginRowId: 0 EndRowId: 18275268 BeginDataSize: 0 EndDataSize: 3680386500 BeginKey: 8 EndKey: 8 State: 0 => take

...

iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 0 closedDataSize: 0 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 22 currentKeyPointer: Part: [72075186767136724:1717:106220:1:12288:671:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 450 BeginDataSize: 0 EndDataSize: 90708 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8

iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 1 currentKeyPointer: Part: [72075186767136724:1717:106220:1:12288:671:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 450 BeginDataSize: 0 EndDataSize: 90708 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8

starting for b-tree index
adding part [72075186767136724:1717:96171:1:12288:263276:0] data size (74.1MiB in total)
adding group {0,0} PageId: 10950 RowCount: 405378 DataSize: 76416997 GroupDataSize: 99288 ErasedRowCount: 392100 LevelCount: 2 IndexSize: 1037342
added slice [0, 405378) data size (72.9MiB - 0B) => 72.9MiB
adding group {0,1} PageId: 10951 RowCount: 3094 DataSize: 99288 ErasedRowCount: 0 LevelCount: 1 IndexSize: 609
added slice [0, 405378) data size (97KiB - 0B) => 73MiB
adding part [72075186767136724:1717:104741:1:12288:1432:0] data size (333KiB in total)
adding group {0,0} PageId: 50 RowCount: 1620 DataSize: 329892 GroupDataSize: 3412 ErasedRowCount: 1567 LevelCount: 1 IndexSize: 4316
added slice [0, 1620) data size (322KiB - 0B) => 73.3MiB
adding group {0,1} PageId: 46 RowCount: 106 DataSize: 3412 ErasedRowCount: 0 LevelCount: 0 IndexSize: 0
added slice [0, 1620) data size (3.33KiB - 0B) => 73.3MiB
adding part [72075186767136724:1717:106719:1:12288:1005:0] data size (197KiB in total)
adding group {0,0} PageId: 31 RowCount: 917 DataSize: 188102 GroupDataSize: 8200 ErasedRowCount: 785 LevelCount: 1 IndexSize: 2530
added slice [0, 917) data size (184KiB - 0B) => 73.5MiB
adding group {0,1} PageId: 32 RowCount: 255 DataSize: 8200 ErasedRowCount: 0 LevelCount: 1 IndexSize: 81
added slice [0, 917) data size (8.01KiB - 0B) => 73.5MiB
adding part [72075186767136724:1717:105250:1:12288:1173:0] data size (255KiB in total)
adding group {0,0} PageId: 39 RowCount: 1221 DataSize: 247679 GroupDataSize: 6740 ErasedRowCount: 1108 LevelCount: 1 IndexSize: 3403
added slice [0, 1221) data size (242KiB - 0B) => 73.7MiB
adding group {0,1} PageId: 35 RowCount: 210 DataSize: 6740 ErasedRowCount: 0 LevelCount: 0 IndexSize: 0
added slice [0, 1221) data size (6.58KiB - 0B) => 73.7MiB
adding part [72075186767136724:1717:105736:1:12288:660:0] data size (103KiB in total)
adding group {0,0} PageId: 17 RowCount: 448 DataSize: 89330 GroupDataSize: 11464 ErasedRowCount: 258 LevelCount: 1 IndexSize: 1208
added slice [0, 448) data size (87.2KiB - 0B) => 73.8MiB
adding group {0,1} PageId: 18 RowCount: 357 DataSize: 11464 ErasedRowCount: 0 LevelCount: 1 IndexSize: 81
added slice [0, 448) data size (11.2KiB - 0B) => 73.8MiB
adding part [72075186767136724:1717:100216:1:12288:8911:0] data size (2.43MiB in total)
adding group {0,0} PageId: 361 RowCount: 12573 DataSize: 2478138 GroupDataSize: 26096 ErasedRowCount: 12072 LevelCount: 2 IndexSize: 33570
added slice [0, 12573) data size (2.36MiB - 0B) => 76.2MiB
adding group {0,1} PageId: 362 RowCount: 813 DataSize: 26096 ErasedRowCount: 0 LevelCount: 1 IndexSize: 169
added slice [0, 12573) data size (25.5KiB - 0B) => 76.2MiB
adding part [72075186767136724:1717:104231:1:12288:5693:0] data size (1.51MiB in total)
adding group {0,0} PageId: 226 RowCount: 7142 DataSize: 1484404 GroupDataSize: 68808 ErasedRowCount: 6063 LevelCount: 2 IndexSize: 20433
added slice [0, 7142) data size (1.42MiB - 0B) => 77.6MiB
adding group {0,1} PageId: 227 RowCount: 2144 DataSize: 68808 ErasedRowCount: 0 LevelCount: 1 IndexSize: 433
added slice [0, 7142) data size (67.2KiB - 0B) => 77.7MiB
adding part [72075186767136724:1717:106220:1:12288:671:0] data size (104KiB in total)
adding group {0,0} PageId: 17 RowCount: 450 DataSize: 90708 GroupDataSize: 11848 ErasedRowCount: 256 LevelCount: 1 IndexSize: 1232
added slice [0, 450) data size (88.6KiB - 0B) => 77.8MiB
adding group {0,1} PageId: 18 RowCount: 369 DataSize: 11848 ErasedRowCount: 0 LevelCount: 1 IndexSize: 81
added slice [0, 450) data size (11.6KiB - 0B) => 77.8MiB
adding part [72075186767136724:1716:7548798:1:12288:8388608:0] data size (3.48GiB in total)
adding group {0,0} PageId: 526222 RowCount: 18275268 DataSize: 3680386500 GroupDataSize: 29124 ErasedRowCount: 28958 LevelCount: 3 IndexSize: 53484057
added slice [0, 18275268) data size (3.43GiB - 0B) => 3.5GiB
adding group {0,1} PageId: 526223 RowCount: 907 DataSize: 29124 ErasedRowCount: 0 LevelCount: 1 IndexSize: 213
added slice [0, 18275268) data size (28.4KiB - 0B) => 3.5GiB
adding part [72075186767136724:1717:107217:1:12288:931:0] data size (180KiB in total)
adding group {0,0} PageId: 28 RowCount: 802 DataSize: 168875 GroupDataSize: 9480 ErasedRowCount: 653 LevelCount: 1 IndexSize: 2252
added slice [0, 802) data size (165KiB - 0B) => 3.5GiB
adding group {0,1} PageId: 29 RowCount: 295 DataSize: 9480 ErasedRowCount: 0 LevelCount: 1 IndexSize: 81
added slice [0, 802) data size (9.26KiB - 0B) => 3.5GiB
adding part [72075186767136724:1717:107720:1:12288:1197:0] data size (254KiB in total)
adding group {0,0} PageId: 39 RowCount: 1183 DataSize: 245937 GroupDataSize: 7304 ErasedRowCount: 1066 LevelCount: 1 IndexSize: 3313
added slice [0, 1183) data size (240KiB - 0B) => 3.5GiB
adding group {0,1} PageId: 40 RowCount: 227 DataSize: 7304 ErasedRowCount: 0 LevelCount: 1 IndexSize: 81
added slice [0, 1183) data size (7.13KiB - 0B) => 3.5GiB
building histogram with row resolution 1870700, data size resolution 359MiB
slicing node Part: [72075186767136724:1717:96171:1:12288:263276:0] PageId: 10950 Level: 2 BeginRowId: 0 EndRowId: 405378 BeginDataSize: 0 EndDataSize: 76416997 BeginKey: 8 EndKey: 8 State: 0 => take
slicing node Part: [72075186767136724:1717:104741:1:12288:1432:0] PageId: 50 Level: 1 BeginRowId: 0 EndRowId: 1620 BeginDataSize: 0 EndDataSize: 329892 BeginKey: 8 EndKey: 8 State: 0 => take
slicing node Part: [72075186767136724:1717:106719:1:12288:1005:0] PageId: 31 Level: 1 BeginRowId: 0 EndRowId: 917 BeginDataSize: 0 EndDataSize: 188102 BeginKey: 8 EndKey: 8 State: 0 => take
slicing node Part: [72075186767136724:1717:105250:1:12288:1173:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1221 BeginDataSize: 0 EndDataSize: 247679 BeginKey: 8 EndKey: 8 State: 0 => take
slicing node Part: [72075186767136724:1717:105736:1:12288:660:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 448 BeginDataSize: 0 EndDataSize: 89330 BeginKey: 8 EndKey: 8 State: 0 => take
slicing node Part: [72075186767136724:1717:100216:1:12288:8911:0] PageId: 361 Level: 2 BeginRowId: 0 EndRowId: 12573 BeginDataSize: 0 EndDataSize: 2478138 BeginKey: 8 EndKey: 8 State: 0 => take
slicing node Part: [72075186767136724:1717:104231:1:12288:5693:0] PageId: 226 Level: 2 BeginRowId: 0 EndRowId: 7142 BeginDataSize: 0 EndDataSize: 1484404 BeginKey: 8 EndKey: 8 State: 0 => take
slicing node Part: [72075186767136724:1717:106220:1:12288:671:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 450 BeginDataSize: 0 EndDataSize: 90708 BeginKey: 8 EndKey: 8 State: 0 => take
slicing node Part: [72075186767136724:1716:7548798:1:12288:8388608:0] PageId: 526222 Level: 3 BeginRowId: 0 EndRowId: 18275268 BeginDataSize: 0 EndDataSize: 3680386500 BeginKey: 8 EndKey: 8 State: 0 => take
slicing node Part: [72075186767136724:1717:107217:1:12288:931:0] PageId: 28 Level: 1 BeginRowId: 0 EndRowId: 802 BeginDataSize: 0 EndDataSize: 168875 BeginKey: 8 EndKey: 8 State: 0 => take
slicing node Part: [72075186767136724:1717:107720:1:12288:1197:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1183 BeginDataSize: 0 EndDataSize: 245937 BeginKey: 8 EndKey: 8 State: 0 => take
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 0 closedDataSize: 0 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 22 currentKeyPointer: Part: [72075186767136724:1717:106220:1:12288:671:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 450 BeginDataSize: 0 EndDataSize: 90708 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
processing event Part: [72075186767136724:1717:106220:1:12288:671:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 450 BeginDataSize: 0 EndDataSize: 90708 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 450 closedDataSize: 90708 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 21 currentKeyPointer: Part: [72075186767136724:1717:106220:1:12288:671:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 450 BeginDataSize: 0 EndDataSize: 90708 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 0 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 450 closedDataSize: 90708 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 21 currentKeyPointer: Part: [72075186767136724:1717:105250:1:12288:1173:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1221 BeginDataSize: 0 EndDataSize: 247679 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
processing event Part: [72075186767136724:1717:105250:1:12288:1173:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1221 BeginDataSize: 0 EndDataSize: 247679 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 1671 closedDataSize: 338387 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 20 currentKeyPointer: Part: [72075186767136724:1717:105250:1:12288:1173:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1221 BeginDataSize: 0 EndDataSize: 247679 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 0 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 1671 closedDataSize: 338387 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 20 currentKeyPointer: Part: [72075186767136724:1717:107720:1:12288:1197:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1183 BeginDataSize: 0 EndDataSize: 245937 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
processing event Part: [72075186767136724:1717:107720:1:12288:1197:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1183 BeginDataSize: 0 EndDataSize: 245937 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 2854 closedDataSize: 5843 24 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 19 currentKeyPointer: Part: [72075186767136724:1717:107720:1:12288:1197:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1183 BeginDataSize: 0 EndDataSize: 245937 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 0 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 2854 closedDataSize: 584324 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 19 currentKeyPointer: Part: [72075186767136724:1717:105736:1:12288:660:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 448 BeginDataSize: 0 EndDataSize: 89330 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
processing event Part: [72075186767136724:1717:105736:1:12288:660:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 448 BeginDataSize: 0 EndDataSize: 89330 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 3302 closedDataSize: 673654 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 18 currentKeyPointer: Part: [72075186767136724:1717:105736:1:12288:660:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 448 BeginDataSize: 0 EndDataSize: 89330 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 0 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 3302 closedDataSize: 673654 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 18 currentKeyPointer: Part: [72075186767136724:1717:104741:1:12288:1432:0] PageId: 50 Level: 1 BeginRowId: 0 EndRowId: 1620 BeginDataSize: 0 EndDataSize: 329892 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
processing event Part: [72075186767136724:1717:104741:1:12288:1432:0] PageId: 50 Level: 1 BeginRowId: 0 EndRowId: 1620 BeginDataSize: 0 EndDataSize: 329892 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 4922 closedDataSize: 1003546 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 17 currentKeyPointer: Part: [72075186767136724:1717:104741:1:12288:1432:0] PageId: 50 Level: 1 BeginRowId: 0 EndRowId: 1620 BeginDataSize: 0 EndDataSize: 329892 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 0 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 4922 closedDataSize: 1003546 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 17 currentKeyPointer: Part: [72075186767136724:1717:107217:1:12288:931:0] PageId: 28 Level: 1 BeginRowId: 0 EndRowId: 802 BeginDataSize: 0 EndDataSize: 168875 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
processing event Part: [72075186767136724:1717:107217:1:12288:931:0] PageId: 28 Level: 1 BeginRowId: 0 EndRowId: 802 BeginDataSize: 0 EndDataSize: 168875 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 5724 closedDataSize: 1172421 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 16 currentKeyPointer: Part: [72075186767136724:1717:107217:1:12288:931:0] PageId: 28 Level: 1 BeginRowId: 0 EndRowId: 802 BeginDataSize: 0 EndDataSize: 168875 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 0 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 5724 closedDataSize: 1172421 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 16 currentKeyPointer: Part: [72075186767136724:1717:106719:1:12288:1005:0] PageId: 31 Level: 1 BeginRowId: 0 EndRowId: 917 BeginDataSize: 0 EndDataSize: 188102 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
processing event Part: [72075186767136724:1717:106719:1:12288:1005:0] PageId: 31 Level: 1 BeginRowId: 0 EndRowId: 917 BeginDataSize: 0 EndDataSize: 188102 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 6641 closedDataSize: 1360523 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 15 currentKeyPointer: Part: [72075186767136724:1717:106719:1:12288:1005:0] PageId: 31 Level: 1 BeginRowId: 0 EndRowId: 917 BeginDataSize: 0 EndDataSize: 188102 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 0 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 6641 closedDataSize: 1360523 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 15 currentKeyPointer: Part: [72075186767136724:1717:104231:1:12288:5693:0] PageId: 226 Level: 2 BeginRowId: 0 EndRowId: 7142 BeginDataSize: 0 EndDataSize: 1484404 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
processing event Part: [72075186767136724:1717:104231:1:12288:5693:0] PageId: 226 Level: 2 BeginRowId: 0 EndRowId: 7142 BeginDataSize: 0 EndDataSize: 1484404 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 13783 closedDataSize: 2844927 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 14 currentKeyPointer: Part: [72075186767136724:1717:104231:1:12288:5693:0] PageId: 226 Level: 2 BeginRowId: 0 EndRowId: 7142 BeginDataSize: 0 EndDataSize: 1484404 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 0 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 13783 closedDataSize: 2844927 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 14 currentKeyPointer: Part: [72075186767136724:1717:100216:1:12288:8911:0] PageId: 361 Level: 2 BeginRowId: 0 EndRowId: 12573 BeginDataSize: 0 EndDataSize: 2478138 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
processing event Part: [72075186767136724:1717:100216:1:12288:8911:0] PageId: 361 Level: 2 BeginRowId: 0 EndRowId: 12573 BeginDataSize: 0 EndDataSize: 2478138 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 26356 closedDataSize: 5323065 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 13 currentKeyPointer: Part: [72075186767136724:1717:100216:1:12288:8911:0] PageId: 361 Level: 2 BeginRowId: 0 EndRowId: 12573 BeginDataSize: 0 EndDataSize: 2478138 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 0 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 26356 closedDataSize: 5323065 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 13 currentKeyPointer: Part: [72075186767136724:1717:96171:1:12288:263276:0] PageId: 10950 Level: 2 BeginRowId: 0 EndRowId: 405378 BeginDataSize: 0 EndDataSize: 76416997 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
processing event Part: [72075186767136724:1717:96171:1:12288:263276:0] PageId: 10950 Level: 2 BeginRowId: 0 EndRowId: 405378 BeginDataSize: 0 EndDataSize: 76416997 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 431734 closedDataSize: 81740062 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 12 currentKeyPointer: Part: [72075186767136724:1717:96171:1:12288:263276:0] PageId: 10950 Level: 2 BeginRowId: 0 EndRowId: 405378 BeginDataSize: 0 EndDataSize: 76416997 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 0 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 431734 closedDataSize: 81740062 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 12 currentKeyPointer: Part: [72075186767136724:1716:7548798:1:12288:8388608:0] PageId: 526222 Level: 3 BeginRowId: 0 EndRowId: 18275268 BeginDataSize: 0 EndDataSize: 3680386500 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
processing event Part: [72075186767136724:1716:7548798:1:12288:8388608:0] PageId: 526222 Level: 3 BeginRowId: 0 EndRowId: 18275268 BeginDataSize: 0 EndDataSize: 3680386500 BeginKey: 8 EndKey: 8 State: 0 IsBegin: 0 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 0 nextHistogramRowCount: 1870700 nextHistogramDataSize: 376240832 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 11 currentKeyPointer: Part: [72075186767136724:1716:7548798:1:12288:8388608:0] PageId: 526222 Level: 3 BeginRowId: 0 EndRowId: 18275268 BeginDataSize: 0 EndDataSize: 3680386500 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 0 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 11 currentKeyPointer: Part: [72075186767136724:1716:7548798:1:12288:8388608:0] PageId: 526222 Level: 3 BeginRowId: 0 EndRowId: 18275268 BeginDataSize: 0 EndDataSize: 3680386500 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
processing event Part: [72075186767136724:1716:7548798:1:12288:8388608:0] PageId: 526222 Level: 3 BeginRowId: 0 EndRowId: 18275268 BeginDataSize: 0 EndDataSize: 3680386500 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 10 currentKeyPointer: Part: [72075186767136724:1716:7548798:1:12288:8388608:0] PageId: 526222 Level: 3 BeginRowId: 0 EndRowId: 18275268 BeginDataSize: 0 EndDataSize: 3680386500 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 10 currentKeyPointer: Part: [72075186767136724:1717:96171:1:12288:263276:0] PageId: 10950 Level: 2 BeginRowId: 0 EndRowId: 405378 BeginDataSize: 0 EndDataSize: 76416997 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
processing event Part: [72075186767136724:1717:96171:1:12288:263276:0] PageId: 10950 Level: 2 BeginRowId: 0 EndRowId: 405378 BeginDataSize: 0 EndDataSize: 76416997 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 9 currentKeyPointer: Part: [72075186767136724:1717:96171:1:12288:263276:0] PageId: 10950 Level: 2 BeginRowId: 0 EndRowId: 405378 BeginDataSize: 0 EndDataSize: 76416997 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 9 currentKeyPointer: Part: [72075186767136724:1717:100216:1:12288:8911:0] PageId: 361 Level: 2 BeginRowId: 0 EndRowId: 12573 BeginDataSize: 0 EndDataSize: 2478138 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
processing event Part: [72075186767136724:1717:100216:1:12288:8911:0] PageId: 361 Level: 2 BeginRowId: 0 EndRowId: 12573 BeginDataSize: 0 EndDataSize: 2478138 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 8 currentKeyPointer: Part: [72075186767136724:1717:100216:1:12288:8911:0] PageId: 361 Level: 2 BeginRowId: 0 EndRowId: 12573 BeginDataSize: 0 EndDataSize: 2478138 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 8 currentKeyPointer: Part: [72075186767136724:1717:107217:1:12288:931:0] PageId: 28 Level: 1 BeginRowId: 0 EndRowId: 802 BeginDataSize: 0 EndDataSize: 168875 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
processing event Part: [72075186767136724:1717:107217:1:12288:931:0] PageId: 28 Level: 1 BeginRowId: 0 EndRowId: 802 BeginDataSize: 0 EndDataSize: 168875 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 7 currentKeyPointer: Part: [72075186767136724:1717:107217:1:12288:931:0] PageId: 28 Level: 1 BeginRowId: 0 EndRowId: 802 BeginDataSize: 0 EndDataSize: 168875 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 7 currentKeyPointer: Part: [72075186767136724:1717:104741:1:12288:1432:0] PageId: 50 Level: 1 BeginRowId: 0 EndRowId: 1620 BeginDataSize: 0 EndDataSize: 329892 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
processing event Part: [72075186767136724:1717:104741:1:12288:1432:0] PageId: 50 Level: 1 BeginRowId: 0 EndRowId: 1620 BeginDataSize: 0 EndDataSize: 329892 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 6 currentKeyPointer: Part: [72075186767136724:1717:104741:1:12288:1432:0] PageId: 50 Level: 1 BeginRowId: 0 EndRowId: 1620 BeginDataSize: 0 EndDataSize: 329892 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 6 currentKeyPointer: Part: [72075186767136724:1717:104231:1:12288:5693:0] PageId: 226 Level: 2 BeginRowId: 0 EndRowId: 7142 BeginDataSize: 0 EndDataSize: 1484404 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
processing event Part: [72075186767136724:1717:104231:1:12288:5693:0] PageId: 226 Level: 2 BeginRowId: 0 EndRowId: 7142 BeginDataSize: 0 EndDataSize: 1484404 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 5 currentKeyPointer: Part: [72075186767136724:1717:104231:1:12288:5693:0] PageId: 226 Level: 2 BeginRowId: 0 EndRowId: 7142 BeginDataSize: 0 EndDataSize: 1484404 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 5 currentKeyPointer: Part: [72075186767136724:1717:107720:1:12288:1197:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1183 BeginDataSize: 0 EndDataSize: 245937 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
processing event Part: [72075186767136724:1717:107720:1:12288:1197:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1183 BeginDataSize: 0 EndDataSize: 245937 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 4 currentKeyPointer: Part: [72075186767136724:1717:107720:1:12288:1197:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1183 BeginDataSize: 0 EndDataSize: 245937 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 4 currentKeyPointer: Part: [72075186767136724:1717:105736:1:12288:660:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 448 BeginDataSize: 0 EndDataSize: 89330 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
processing event Part: [72075186767136724:1717:105736:1:12288:660:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 448 BeginDataSize: 0 EndDataSize: 89330 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 3 currentKeyPointer: Part: [72075186767136724:1717:105736:1:12288:660:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 448 BeginDataSize: 0 EndDataSize: 89330 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 3 currentKeyPointer: Part: [72075186767136724:1717:105250:1:12288:1173:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1221 BeginDataSize: 0 EndDataSize: 247679 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
processing event Part: [72075186767136724:1717:105250:1:12288:1173:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1221 BeginDataSize: 0 EndDataSize: 247679 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 2 currentKeyPointer: Part: [72075186767136724:1717:105250:1:12288:1173:0] PageId: 39 Level: 1 BeginRowId: 0 EndRowId: 1221 BeginDataSize: 0 EndDataSize: 247679 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 2 currentKeyPointer: Part: [72075186767136724:1717:106719:1:12288:1005:0] PageId: 31 Level: 1 BeginRowId: 0 EndRowId: 917 BeginDataSize: 0 EndDataSize: 188102 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
processing event Part: [72075186767136724:1717:106719:1:12288:1005:0] PageId: 31 Level: 1 BeginRowId: 0 EndRowId: 917 BeginDataSize: 0 EndDataSize: 188102 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 1 currentKeyPointer: Part: [72075186767136724:1717:106719:1:12288:1005:0] PageId: 31 Level: 1 BeginRowId: 0 EndRowId: 917 BeginDataSize: 0 EndDataSize: 188102 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
iterating stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 1 currentKeyPointer: Part: [72075186767136724:1717:106220:1:12288:671:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 450 BeginDataSize: 0 EndDataSize: 90708 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
processing event Part: [72075186767136724:1717:106220:1:12288:671:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 450 BeginDataSize: 0 EndDataSize: 90708 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
checking stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 0 currentKeyPointer: Part: [72075186767136724:1717:106220:1:12288:671:0] PageId: 17 Level: 1 BeginRowId: 0 EndRowId: 450 BeginDataSize: 0 EndDataSize: 90708 BeginKey: 8 EndKey: 8 State: 2 IsBegin: 1 Key: 8
finished stats.RowCountHistogram: 0 stats.DataSizeHistogram: 1 nextHistogramRowCount: 1870700 nextHistogramDataSize: 18446744073709551615 closedRowCount: 18707002 closedDataSize: 3762126562 openedRowCount: 0 openedDataSize: 0 openedSortedByRowCount: 0 openedSortedByDataSize: 0 FutureEvents: 0
finished for b-tree index ready: 1 stats: RowCount: 18707002 DataSize: 3762408326 IndexSize: 54595485 RowCountHistogram: 0 DataSizeHistogram: 1

This implies that the node BeginKey and EndKey are the same, but we should have taken them from the slice bounds and the slice contain more than 3GB of data 🤔 🤔

[kunga]

Fixed in #15720