Skip to content

KIKIMR-22120: Take partitioning description of copied table in s3 export #13067

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

pixcc
Copy link
Member

@pixcc pixcc commented Dec 27, 2024

Changelog entry

Fixes creating unrestorable S3 backups when splitting and merging the table

Changelog category

  • Bugfix

@pixcc pixcc requested a review from CyberROFL December 27, 2024 08:10
@pixcc pixcc self-assigned this Dec 27, 2024
@github-actions github-actions bot added bugfix and removed bugfix labels Dec 27, 2024
Copy link

github-actions bot commented Dec 27, 2024

2024-12-27 08:13:43 UTC Pre-commit check linux-x86_64-relwithdebinfo for 61c6358 has started.
2024-12-27 08:13:55 UTC Artifacts will be uploaded here
2024-12-27 08:16:45 UTC ya make is running...
🟡 2024-12-27 08:58:57 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19652 18102 0 9 1417 124

2024-12-27 09:00:35 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-27 09:13:09 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
198 (only retried tests) 76 0 2 0 120

2024-12-27 09:13:17 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-12-27 09:25:02 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
178 (only retried tests) 58 0 1 0 119

🟢 2024-12-27 09:25:09 UTC Build successful.
🟢 2024-12-27 09:25:33 UTC ydbd size 2.1 GiB changed* by +6.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4081b1f merge: 61c6358 diff diff %
ydbd size 2 241 452 872 Bytes 2 241 459 408 Bytes +6.4 KiB +0.000%
ydbd stripped size 471 939 088 Bytes 471 939 728 Bytes +640 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Dec 27, 2024

2024-12-27 08:13:49 UTC Pre-commit check linux-x86_64-release-asan for 61c6358 has started.
2024-12-27 08:14:05 UTC Artifacts will be uploaded here
2024-12-27 08:16:55 UTC ya make is running...
🟡 2024-12-27 09:09:46 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11569 11501 0 38 5 25

2024-12-27 09:10:50 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-27 09:22:46 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
116 (only retried tests) 84 0 7 0 25

2024-12-27 09:22:54 UTC ya make is running... (failed tests rerun, try 3)
🟡 2024-12-27 09:28:01 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
62 (only retried tests) 36 0 2 0 24

🟢 2024-12-27 09:28:08 UTC Build successful.
🟢 2024-12-27 09:28:36 UTC ydbd size 3.6 GiB changed* by +6.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4081b1f merge: 61c6358 diff diff %
ydbd size 3 869 161 408 Bytes 3 869 167 904 Bytes +6.3 KiB +0.000%
ydbd stripped size 1 349 101 808 Bytes 1 349 103 728 Bytes +1.9 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

// Add delay after copying tables
bool dropNotification = false;
THolder<IEventHandle> delayed;
auto prevObserver = runtime.SetObserverFunc([&](TAutoPtr<IEventHandle>& ev) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Есть TBlockEvent, возможно, его будет удобнее использовать.

Comment on lines +1240 to +1246
if (!delayed) {
TDispatchOptions opts;
opts.FinalEvents.emplace_back([&delayed](IEventHandle&) -> bool {
return bool(delayed);
});
runtime.DispatchEvents(opts);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Есть хелпер runtime.WaitFor.

@maximyurchuk maximyurchuk merged commit d996c04 into ydb-platform:main Dec 27, 2024
10 of 14 checks passed
pixcc added a commit to pixcc/ydb that referenced this pull request Jan 9, 2025
pixcc added a commit to pixcc/ydb that referenced this pull request Jan 9, 2025
pixcc added a commit that referenced this pull request Jan 9, 2025
pixcc added a commit that referenced this pull request Jan 9, 2025
zinal pushed a commit to zinal/ydb that referenced this pull request Jan 12, 2025
zinal pushed a commit to zinal/ydb that referenced this pull request Feb 9, 2025
zinal pushed a commit to zinal/ydb that referenced this pull request Feb 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

KIKIMR-22120: Take partitioning description of copied table in s3 export
3 participants