YDB-7262 Add default permille ICB configuration for vdisk garbage compaction threshold #6819

SammyVimes · 2024-07-18T11:28:40Z

Changelog category

New feature

github-actions · 2024-07-18T11:30:51Z

⚪ 2024-07-18 11:30:50 UTC Pre-commit check for 9a71d6a has started.
⚪ 2024-07-18 11:33:29 UTC Build+Tests linux-x86_64-release-asan is running...
🔴 2024-07-18 12:12:53 UTC Build failed. see the logs.
🔴 2024-07-18 12:12:57 UTC Test run completed, no test results found for commit 3d7a124.
🟢 2024-07-18 12:13:25 UTC ydbd size 5.2 GiB changed* by +7.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `302e204`	merge: `9a71d6a`	diff	diff %
ydbd size	5 546 914 912 Bytes	5 546 922 208 Bytes	+7.1 KiB	+0.000%
ydbd stripped size	1 195 315 792 Bytes	1 195 314 352 Bytes	-1.4 KiB	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-18T11:32:03Z

⚪ 2024-07-18 11:32:02 UTC Pre-commit check for 9a71d6a has started.
⚪ 2024-07-18 11:34:46 UTC Build+Tests linux-x86_64-release-clang14 is running...
🔴 2024-07-18 11:52:14 UTC Build failed. see the logs.

github-actions · 2024-07-18T11:32:24Z

⚪ 2024-07-18 11:32:23 UTC Pre-commit check for 9a71d6a has started.
⚪ 2024-07-18 11:35:11 UTC Build+Tests linux-x86_64-relwithdebinfo is running...
🔴 2024-07-18 12:29:47 UTC Build failed. see the logs.
🔴 2024-07-18 12:29:50 UTC Test run completed, no test results found for commit 3d7a124.
🟢 2024-07-18 12:30:36 UTC ydbd size 8.0 GiB changed* by +17.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `302e204`	merge: `9a71d6a`	diff	diff %
ydbd size	8 642 536 216 Bytes	8 642 553 896 Bytes	+17.3 KiB	+0.000%
ydbd stripped size	470 525 536 Bytes	470 524 960 Bytes	-576 Bytes	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-26T17:09:20Z

⚪ 2024-07-26 17:09:19 UTC Pre-commit check for da4c907 has started.
⚪ 2024-07-26 17:12:23 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-07-26 17:29:12 UTC Build failed. see the logs.

github-actions · 2024-07-26T17:09:36Z

⚪ 2024-07-26 17:09:35 UTC Pre-commit check for da4c907 has started.
⚪ 2024-07-26 17:12:32 UTC Check linux-x86_64-release-asan is running...
🔴 2024-07-26 17:54:46 UTC Build failed. see the logs.
🟡 2024-07-26 17:55:16 UTC ydbd size 5.2 GiB changed* by +116.6 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `28816eb`	merge: `da4c907`	diff	diff %
ydbd size	5 565 688 688 Bytes	5 565 808 120 Bytes	+116.6 KiB	+0.002%
ydbd stripped size	1 198 433 328 Bytes	1 198 455 280 Bytes	+21.4 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-26T17:11:57Z

⚪ 2024-07-26 17:11:57 UTC Pre-commit check for da4c907 has started.
⚪ 2024-07-26 17:14:59 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-07-26 18:04:43 UTC Build failed. see the logs.
🟡 2024-07-26 18:05:25 UTC ydbd size 8.1 GiB changed* by +397.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `9785f6a`	merge: `da4c907`	diff	diff %
ydbd size	8 668 833 544 Bytes	8 669 241 008 Bytes	+397.9 KiB	+0.005%
ydbd stripped size	471 575 616 Bytes	471 588 608 Bytes	+12.7 KiB	+0.003%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

SammyVimes · 2024-07-29T14:05:25Z

ydb/core/blobstorage/vdisk/defrag/defrag_quantum.cpp


-                auto hugeStat = GetHugeStat();
-                Y_ABORT_UNLESS(hugeStat.LockedChunks.size() < 100);
+                // auto hugeStat = GetHugeStat();


Вот тут я не знаю, как правильно сделать, просто убирать verify выглядит не хорошо

Сделать DEBUG VERIFY :)

github-actions · 2024-07-29T14:18:29Z

⚪ 2024-07-29 14:18:29 UTC Pre-commit check for 61d9e1d has started.
⚪ 2024-07-29 14:21:24 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-07-29 14:38:14 UTC Build failed. see the logs.

github-actions · 2024-07-29T14:19:30Z

⚪ 2024-07-29 14:19:29 UTC Pre-commit check for 61d9e1d has started.
⚪ 2024-07-29 14:22:29 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-07-29 14:33:10 UTC Build failed. see the logs.

github-actions · 2024-07-29T14:19:49Z

⚪ 2024-07-29 14:19:49 UTC Pre-commit check for 61d9e1d has started.
⚪ 2024-07-29 14:22:49 UTC Check linux-x86_64-release-asan is running...
🔴 2024-07-29 14:34:53 UTC Build failed. see the logs.

github-actions · 2024-07-29T14:57:47Z

⚪ 2024-07-29 14:57:47 UTC Pre-commit check for ef834b2 has started.
⚪ 2024-07-29 15:00:51 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-07-29 15:08:19 UTC Build successful.

github-actions · 2024-07-29T14:58:15Z

⚪ 2024-07-29 14:58:15 UTC Pre-commit check for ef834b2 has started.
⚪ 2024-07-29 15:01:17 UTC Check linux-x86_64-release-asan is running...
🔴 2024-07-29 17:22:30 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10907	10316	0	90	327	174

🟢 2024-07-29 17:23:33 UTC Build successful.
🟢 2024-07-29 17:24:40 UTC ydbd size 5.2 GiB changed* by -12.9 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `33bdce5`	merge: `ef834b2`	diff	diff %
ydbd size	5 570 084 520 Bytes	5 570 071 336 Bytes	-12.9 KiB	-0.000%
ydbd stripped size	1 199 054 416 Bytes	1 199 050 640 Bytes	-3.7 KiB	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-29T15:00:04Z

⚪ 2024-07-29 15:00:04 UTC Pre-commit check for ef834b2 has started.
⚪ 2024-07-29 15:02:57 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-07-29 16:39:15 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15202	13865	0	1	1325	11

⚪ 2024-07-29 16:40:43 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟢 2024-07-29 16:48:40 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10 (only retried tests)	2	0	0	0	8

🟢 2024-07-29 16:49:56 UTC Build successful.
🟢 2024-07-29 16:50:35 UTC ydbd size 8.1 GiB changed* by -17.9 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `33bdce5`	merge: `ef834b2`	diff	diff %
ydbd size	8 676 558 128 Bytes	8 676 539 760 Bytes	-17.9 KiB	-0.000%
ydbd stripped size	471 820 640 Bytes	471 819 680 Bytes	-960 Bytes	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

alexvru · 2024-07-30T12:43:56Z

ydb/core/blobstorage/vdisk/defrag/defrag_quantum.cpp


-                auto hugeStat = GetHugeStat();
-                Y_ABORT_UNLESS(hugeStat.LockedChunks.size() < 100);
+                // auto hugeStat = GetHugeStat();


Сделать DEBUG VERIFY :)

github-actions · 2024-07-30T14:29:26Z

⚪ 2024-07-30 14:29:26 UTC Pre-commit check for 32ef2e9 has started.
⚪ 2024-07-30 14:32:19 UTC Check linux-x86_64-release-asan is running...
🔴 2024-07-30 16:44:10 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10914	10333	0	85	324	172

🟢 2024-07-30 16:45:14 UTC Build successful.
🟢 2024-07-30 16:45:44 UTC ydbd size 5.2 GiB changed* by +6.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `ee419db`	merge: `32ef2e9`	diff	diff %
ydbd size	5 571 118 576 Bytes	5 571 125 440 Bytes	+6.7 KiB	+0.000%
ydbd stripped size	1 199 220 976 Bytes	1 199 221 424 Bytes	+448 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-30T14:30:55Z

⚪ 2024-07-30 14:30:54 UTC Pre-commit check for 32ef2e9 has started.
⚪ 2024-07-30 14:33:54 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-07-30 14:52:34 UTC Build successful.

github-actions · 2024-07-30T14:30:58Z

⚪ 2024-07-30 14:30:57 UTC Pre-commit check for 32ef2e9 has started.
⚪ 2024-07-30 14:33:57 UTC Check linux-x86_64-relwithdebinfo is running...
🟢 2024-07-30 16:23:34 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15218	13874	0	0	1327	17

🟢 2024-07-30 16:24:59 UTC Build successful.
🟢 2024-07-30 16:25:43 UTC ydbd size 8.1 GiB changed* by +7.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `ee419db`	merge: `32ef2e9`	diff	diff %
ydbd size	8 678 462 728 Bytes	8 678 470 240 Bytes	+7.3 KiB	+0.000%
ydbd stripped size	471 881 056 Bytes	471 881 824 Bytes	+768 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

…paction threshold (ydb-platform#6819) Сloses ydb-platform#7262 (cherry picked from commit 18c3f7a)

Add default permille for vdisk garbage compaction threshold

3d7a124

github-actions bot added the new-feature label Jul 18, 2024

SammyVimes changed the title ~~Add default permille for vdisk garbage compaction threshold~~ Add default permille ICB configuration for vdisk garbage compaction threshold Jul 18, 2024

va-kuznecov mentioned this pull request Jul 24, 2024

Add ICB to control VDisk huge garbage threshold #6638

Closed

alexv-smirnov added the rebase-and-check Rebase PR with the current base branch and check label Jul 26, 2024

github-actions bot removed the rebase-and-check Rebase PR with the current base branch and check label Jul 26, 2024

SammyVimes commented Jul 29, 2024

View reviewed changes

SammyVimes requested a review from alexvru July 29, 2024 14:05

Fix build

1ed42e8

Fix build again

383f91c

github-actions bot added new-feature and removed new-feature labels Jul 30, 2024

alexvru previously approved these changes Jul 30, 2024

View reviewed changes

Replace abort unless with debug abort unless

f1d5957

SammyVimes dismissed alexvru’s stale review via f1d5957 July 30, 2024 14:27

va-kuznecov approved these changes Jul 30, 2024

View reviewed changes

SammyVimes changed the title ~~Add default permille ICB configuration for vdisk garbage compaction threshold~~ YDB-7262 Add default permille ICB configuration for vdisk garbage compaction threshold Jul 30, 2024

github-actions bot added new-feature and removed new-feature labels Jul 30, 2024

SammyVimes enabled auto-merge (squash) July 30, 2024 16:46

SammyVimes merged commit 18c3f7a into ydb-platform:main Jul 30, 2024

SammyVimes added a commit to SammyVimes/ydb that referenced this pull request Jul 31, 2024

YDB-7262 Add default permille ICB configuration for vdisk garbage com…

7674ba0

…paction threshold (ydb-platform#6819) Сloses ydb-platform#7262 (cherry picked from commit 18c3f7a)

SammyVimes added a commit to SammyVimes/ydb that referenced this pull request Jul 31, 2024

YDB-7262 Add default permille ICB configuration for vdisk garbage com…

8692bcf

…paction threshold (ydb-platform#6819) Сloses ydb-platform#7262 (cherry picked from commit 18c3f7a)

StekPerepolnen mentioned this pull request Aug 8, 2024

tenantinfo handler added DatabaseStorage #7554

Merged

YDB-7262 Add default permille ICB configuration for vdisk garbage compaction threshold #6819

YDB-7262 Add default permille ICB configuration for vdisk garbage compaction threshold #6819

Uh oh!

Conversation

SammyVimes commented Jul 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category

Uh oh!

github-actions bot commented Jul 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SammyVimes Jul 29, 2024

Choose a reason for hiding this comment

Uh oh!

alexvru Jul 30, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexvru Jul 30, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

SammyVimes commented Jul 18, 2024 •

edited

Loading

github-actions bot commented Jul 18, 2024 •

edited

Loading

github-actions bot commented Jul 18, 2024 •

edited

Loading

github-actions bot commented Jul 18, 2024 •

edited

Loading

github-actions bot commented Jul 26, 2024 •

edited

Loading

github-actions bot commented Jul 26, 2024 •

edited

Loading

github-actions bot commented Jul 26, 2024 •

edited

Loading

github-actions bot commented Jul 29, 2024 •

edited

Loading

github-actions bot commented Jul 29, 2024 •

edited

Loading

github-actions bot commented Jul 29, 2024 •

edited

Loading

github-actions bot commented Jul 29, 2024 •

edited

Loading

github-actions bot commented Jul 29, 2024 •

edited

Loading

github-actions bot commented Jul 29, 2024 •

edited

Loading

github-actions bot commented Jul 30, 2024 •

edited

Loading

github-actions bot commented Jul 30, 2024 •

edited

Loading

github-actions bot commented Jul 30, 2024 •

edited

Loading