Skip to content

Fix compacted pages offload race #15377

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 16 commits into from
Mar 11, 2025
Merged

Fix compacted pages offload race #15377

merged 16 commits into from
Mar 11, 2025

Conversation

kunga
Copy link
Member

@kunga kunga commented Mar 5, 2025 •
edited
Loading

Changelog entry

...

Changelog category

  • Not for changelog (changelog entry is not required)

Description for reviewers

Instead of pinning all compacted pages while they are being transfered to executor, unload them immediately

Fixes the ambiguous behaviour of compacting more data than the shared cache can fit in limit has

Fixes a case when compacted pages are offloaded from Shared Cache, but SharedBody stuck in Private Cache by sending a touch event and replying withTEvUpdated.DroppedPages

Also a few improvements:

  • Apply config limit immediately
  • Verify no flat index pages are loaded
  • Do not touch index pages in loader after compaction as these pages may be offloaded concurrently

To sum up, the compaction process works as follows:

  • Writes compacted pages
  • Makes TPage handle for them
  • Sends TEvSaveCompactedPages with TPage inside to Shared Cache
  • Builds page collections with Sticky (loaded and used TSharedPageRef) and Regular (unloaded and unused TSharedPageRef)
  • Sends resulting page collections to Executor

Shared Cache works as follows:

  • Accepts TEvSaveCompactedPages and puts them into cache structure (without an owner)
  • May offload some unloaded pages according to the limit (without any notifications)
  • On TEvTouch of unknown pages replies with TEvUpdated.DroppedPages with these pages (instead of notifications)

Executor works as follows:

  • Accepts NOps::TEvResult with resulting page collections
  • Adds compacted page collections to Private Cache
  • Sends TEvTouch for all provided pages to ensure that they are still exist in Shared Cache

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 5, 2025 •
edited
Loading

🟢 2025-03-11 11:25:48 UTC The validation of the Pull Request description is successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 5, 2025 •
edited
Loading

⚪ 2025-03-05 16:39:39 UTC Pre-commit check linux-x86_64-relwithdebinfo for d4d9201 has started.
⚪ 2025-03-05 16:39:54 UTC Artifacts will be uploaded here
⚪ 2025-03-05 16:42:38 UTC ya make is running...
🟡 2025-03-05 17:40:11 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26470 23870 0 5 2470 125

⚪ 2025-03-05 17:42:26 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-05 17:53:53 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
181 (only retried tests) 76 0 0 0 105

🟢 2025-03-05 17:54:03 UTC Build successful.
🟢 2025-03-05 17:54:21 UTC ydbd size 2.1 GiB changed* by +6.6 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 796e7a6 merge: d4d9201 diff diff %
ydbd size 2 294 065 160 Bytes 2 294 071 904 Bytes +6.6 KiB +0.000%
ydbd stripped size 480 395 264 Bytes 480 396 416 Bytes +1.1 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 5, 2025 •
edited
Loading

⚪ 2025-03-05 16:39:40 UTC Pre-commit check linux-x86_64-release-asan for d4d9201 has started.
⚪ 2025-03-05 16:39:54 UTC Artifacts will be uploaded here
⚪ 2025-03-05 16:42:45 UTC ya make is running...
🟡 2025-03-05 17:52:34 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12105 11741 0 253 73 38

⚪ 2025-03-05 17:54:02 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-05 18:12:17 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
534 (only retried tests) 261 0 163 76 34

⚪ 2025-03-05 18:12:30 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-05 18:28:57 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
410 (only retried tests) 146 0 177 53 34

🟢 2025-03-05 18:29:09 UTC Build successful.
🟢 2025-03-05 18:29:43 UTC ydbd size 3.7 GiB changed* by -96 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: 1067562 merge: d4d9201 diff diff %
ydbd size 3 997 184 136 Bytes 3 997 184 040 Bytes -96 Bytes -0.000%
ydbd stripped size 1 390 284 008 Bytes 1 390 283 944 Bytes -64 Bytes -0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@kunga kunga marked this pull request as ready for review March 6, 2025 08:43
@kunga kunga requested a review from snaury March 6, 2025 08:43
@kunga kunga self-assigned this Mar 7, 2025
snaury
snaury previously approved these changes Mar 7, 2025
View reviewed changes
@kunga kunga marked this pull request as draft March 7, 2025 13:57
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 7, 2025 •
edited
Loading

⚪ 2025-03-07 13:59:37 UTC Pre-commit check linux-x86_64-release-asan for a439d63 has started.
⚪ 2025-03-07 13:59:41 UTC Artifacts will be uploaded here
⚪ 2025-03-07 14:02:34 UTC ya make is running...
🟡 2025-03-07 15:11:47 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12137 11940 0 112 49 36

⚪ 2025-03-07 15:12:55 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-07 15:29:03 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
268 (only retried tests) 226 0 5 6 31

⚪ 2025-03-07 15:29:15 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-07 15:41:29 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
74 (only retried tests) 42 0 2 0 30

🟢 2025-03-07 15:41:43 UTC Build successful.
🟢 2025-03-07 15:42:16 UTC ydbd size 3.7 GiB changed* by +992 Bytes, which is < 100.0 KiB vs main: OK

ydbd size dash main: e4821cc merge: a439d63 diff diff %
ydbd size 3 996 097 640 Bytes 3 996 098 632 Bytes +992 Bytes +0.000%
ydbd stripped size 1 389 147 400 Bytes 1 389 148 040 Bytes +640 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 7, 2025 •
edited
Loading

⚪ 2025-03-07 14:01:22 UTC Pre-commit check linux-x86_64-relwithdebinfo for a439d63 has started.
⚪ 2025-03-07 14:01:26 UTC Artifacts will be uploaded here
⚪ 2025-03-07 14:04:14 UTC ya make is running...
🟡 2025-03-07 15:11:17 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26504 23890 0 32 2466 116

⚪ 2025-03-07 15:13:44 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-07 15:25:36 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
205 (only retried tests) 87 0 10 1 107

⚪ 2025-03-07 15:25:45 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-03-07 15:38:53 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
162 (only retried tests) 48 0 8 0 106

🟢 2025-03-07 15:39:03 UTC Build successful.
🟢 2025-03-07 15:39:28 UTC ydbd size 2.1 GiB changed* by -440 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: e4821cc merge: a439d63 diff diff %
ydbd size 2 294 621 968 Bytes 2 294 621 528 Bytes -440 Bytes -0.000%
ydbd stripped size 480 593 312 Bytes 480 593 568 Bytes +256 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@kunga kunga changed the title Fix flapping Shared Cache tests Unload compacted data pages immediately Mar 7, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 7, 2025 •
edited
Loading

⚪ 2025-03-07 15:48:22 UTC Pre-commit check linux-x86_64-relwithdebinfo for 22ef25a has started.
⚪ 2025-03-07 15:48:36 UTC Artifacts will be uploaded here
⚫ 2025-03-07 15:51:08 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 7, 2025 •
edited
Loading

⚪ 2025-03-07 15:50:05 UTC Pre-commit check linux-x86_64-release-asan for 22ef25a has started.
⚪ 2025-03-07 15:50:19 UTC Artifacts will be uploaded here
⚫ 2025-03-07 15:51:08 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 7, 2025 •
edited
Loading

⚪ 2025-03-07 15:54:42 UTC Pre-commit check linux-x86_64-relwithdebinfo for f5a1c5d has started.
⚪ 2025-03-07 15:54:57 UTC Artifacts will be uploaded here
⚪ 2025-03-07 15:57:51 UTC ya make is running...
🟡 2025-03-07 17:00:36 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26519 23730 0 29 2703 57

⚪ 2025-03-07 17:03:14 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-07 17:15:00 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
392 (only retried tests) 266 0 8 9 109

⚪ 2025-03-07 17:15:14 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-03-07 17:21:57 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
165 (only retried tests) 53 0 5 0 107

🟢 2025-03-07 17:22:07 UTC Build successful.
🟢 2025-03-07 17:22:40 UTC ydbd size 2.1 GiB changed* by -464 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: 7c72123 merge: f5a1c5d diff diff %
ydbd size 2 294 624 448 Bytes 2 294 623 984 Bytes -464 Bytes -0.000%
ydbd stripped size 480 593 824 Bytes 480 594 080 Bytes +256 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 11, 2025 •
edited
Loading

⚪ 2025-03-11 10:45:21 UTC Pre-commit check linux-x86_64-release-asan for 492dfc9 has started.
⚪ 2025-03-11 10:45:35 UTC Artifacts will be uploaded here
⚪ 2025-03-11 10:48:18 UTC ya make is running...
🟡 2025-03-11 11:53:48 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12193 12031 0 117 8 37

⚪ 2025-03-11 11:55:35 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-11 12:10:04 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
251 (only retried tests) 166 0 51 3 31

⚪ 2025-03-11 12:10:19 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-11 12:22:55 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
164 (only retried tests) 103 0 33 0 28

🟢 2025-03-11 12:23:05 UTC Build successful.
🟢 2025-03-11 12:23:38 UTC ydbd size 3.7 GiB changed* by +70.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4880f2d merge: 492dfc9 diff diff %
ydbd size 3 998 881 392 Bytes 3 998 953 168 Bytes +70.1 KiB +0.002%
ydbd stripped size 1 389 593 896 Bytes 1 389 616 840 Bytes +22.4 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 11, 2025 •
edited
Loading

⚪ 2025-03-11 10:45:35 UTC Pre-commit check linux-x86_64-relwithdebinfo for 492dfc9 has started.
⚪ 2025-03-11 10:45:50 UTC Artifacts will be uploaded here
⚪ 2025-03-11 10:48:44 UTC ya make is running...
🟡 2025-03-11 11:47:53 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26569 23978 0 3 2472 116

⚪ 2025-03-11 12:20:31 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-11 12:33:48 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
170 (only retried tests) 66 0 0 0 104

🟢 2025-03-11 12:33:55 UTC Build successful.
🟢 2025-03-11 12:34:19 UTC ydbd size 2.1 GiB changed* by +26.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4880f2d merge: 492dfc9 diff diff %
ydbd size 2 296 125 704 Bytes 2 296 152 408 Bytes +26.1 KiB +0.001%
ydbd stripped size 480 730 208 Bytes 480 732 832 Bytes +2.6 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@kunga kunga changed the title Unload compacted data pages immediately Fix compacted pages offload race Mar 11, 2025
@kunga kunga mentioned this pull request Mar 11, 2025
Open
30 tasks
@kunga kunga marked this pull request as ready for review March 11, 2025 13:03
@kunga kunga requested a review from snaury March 11, 2025 13:04
snaury
snaury reviewed Mar 11, 2025
View reviewed changes
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 11, 2025 •
edited
Loading

⚪ 2025-03-11 13:46:37 UTC Pre-commit check linux-x86_64-relwithdebinfo for 3c46dd1 has started.
⚪ 2025-03-11 13:46:52 UTC Artifacts will be uploaded here
⚪ 2025-03-11 13:49:51 UTC ya make is running...
🟡 2025-03-11 15:21:47 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
26572 23989 0 1 2471 111

⚪ 2025-03-11 15:24:59 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-11 15:38:17 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
159 (only retried tests) 52 0 0 0 107

🟢 2025-03-11 15:38:28 UTC Build successful.
🟢 2025-03-11 15:38:54 UTC ydbd size 2.1 GiB changed* by +26.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4e131ff merge: 3c46dd1 diff diff %
ydbd size 2 295 985 800 Bytes 2 296 012 472 Bytes +26.0 KiB +0.001%
ydbd stripped size 480 696 096 Bytes 480 698 656 Bytes +2.5 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 11, 2025 •
edited
Loading

⚪ 2025-03-11 13:47:12 UTC Pre-commit check linux-x86_64-release-asan for 3c46dd1 has started.
⚪ 2025-03-11 13:47:42 UTC Artifacts will be uploaded here
⚪ 2025-03-11 13:51:00 UTC ya make is running...
🟡 2025-03-11 15:32:03 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12193 12013 0 133 11 36

⚪ 2025-03-11 15:33:35 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-11 15:48:16 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
276 (only retried tests) 176 0 62 4 34

⚪ 2025-03-11 15:48:29 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-11 16:02:34 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
178 (only retried tests) 104 0 42 0 32

🟢 2025-03-11 16:02:41 UTC Build successful.
🟢 2025-03-11 16:03:14 UTC ydbd size 3.7 GiB changed* by -226.6 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: 5ed8c5f merge: 3c46dd1 diff diff %
ydbd size 3 999 006 576 Bytes 3 998 774 584 Bytes -226.6 KiB -0.006%
ydbd stripped size 1 389 625 224 Bytes 1 389 606 632 Bytes -18.2 KiB -0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@kunga kunga enabled auto-merge (squash) March 11, 2025 14:04
@kunga kunga merged commit 46be087 into ydb-platform:main Mar 11, 2025
12 checks passed
@serbel324 serbel324 mentioned this pull request Mar 13, 2025
Merged
@alexnick88 alexnick88 mentioned this pull request Mar 19, 2025
Closed
adameat pushed a commit to adameat/ydb that referenced this pull request Mar 20, 2025
@kunga @adameat
Fix compacted pages offload race (ydb-platform#15377)
c434a6f
blinkov pushed a commit that referenced this pull request Mar 21, 2025
@kunga @blinkov
Fix compacted pages offload race (#15377)
cf86700
lberserq pushed a commit to lberserq/ydb that referenced this pull request Mar 28, 2025
@kunga
Fix compacted pages offload race (ydb-platform#15377)
f05511f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@snaury snaury snaury approved these changes

Assignees

@kunga kunga

Labels
not-for-changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Test TSharedPageCache::Compaction_BTreeIndex is failing
2 participants
@kunga @snaury