Skip to content

Add datashard interface for data cleanup operation #12896

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Dec 25, 2024
Merged

Add datashard interface for data cleanup operation #12896

merged 2 commits into from
Dec 25, 2024

Conversation

@lex007in
Copy link
Collaborator

@lex007in lex007in commented Dec 23, 2024

Changelog entry

...

Changelog category

  • Not for changelog (changelog entry is not required)

Additional information

WIP

@github-actions
Copy link

github-actions bot commented Dec 23, 2024 •
edited
Loading

⚪ 2024-12-23 19:39:58 UTC Pre-commit check linux-x86_64-relwithdebinfo for a3773e9 has started.
⚪ 2024-12-23 19:40:10 UTC Artifacts will be uploaded here
⚪ 2024-12-23 19:43:20 UTC ya make is running...
🔴 2024-12-23 19:45:17 UTC Build failed, see the logs. Also see fail summary

@github-actions
Copy link

github-actions bot commented Dec 23, 2024 •
edited
Loading

⚪ 2024-12-23 19:40:02 UTC Pre-commit check linux-x86_64-release-asan for a3773e9 has started.
⚪ 2024-12-23 19:40:13 UTC Artifacts will be uploaded here
⚪ 2024-12-23 19:43:24 UTC ya make is running...
🔴 2024-12-23 19:45:26 UTC Build failed, see the logs. Also see fail summary

@lex007in lex007in self-assigned this Dec 24, 2024
@lex007in lex007in force-pushed the datashard_persist_del branch from bd93fae to a8b8018 Compare December 24, 2024 14:38
@lex007in lex007in changed the title Add datashard interface for delete persistence Add datashard interface for data cleanup operation Dec 24, 2024
@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 14:39:56 UTC Pre-commit check linux-x86_64-relwithdebinfo for c1c0602 has started.
⚪ 2024-12-24 14:40:53 UTC Artifacts will be uploaded here
⚪ 2024-12-24 14:44:14 UTC ya make is running...
⚫ 2024-12-24 14:48:17 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 14:41:09 UTC Pre-commit check linux-x86_64-release-asan for c1c0602 has started.
⚪ 2024-12-24 14:41:49 UTC Artifacts will be uploaded here
⚪ 2024-12-24 14:45:32 UTC ya make is running...
⚫ 2024-12-24 14:48:17 UTC Check cancelled

@lex007in lex007in force-pushed the datashard_persist_del branch from a8b8018 to 4ab3b6e Compare December 24, 2024 14:47
@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 14:50:27 UTC Pre-commit check linux-x86_64-relwithdebinfo for f0a3f3d has started.
⚪ 2024-12-24 14:50:44 UTC Artifacts will be uploaded here
⚫ 2024-12-24 14:52:22 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 14:50:43 UTC Pre-commit check linux-x86_64-release-asan for f0a3f3d has started.
⚪ 2024-12-24 14:50:54 UTC Artifacts will be uploaded here
⚫ 2024-12-24 14:52:22 UTC Check cancelled

@lex007in lex007in force-pushed the datashard_persist_del branch from 4ab3b6e to 745515b Compare December 24, 2024 14:52
@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 14:53:42 UTC Pre-commit check linux-x86_64-release-asan for 39ac247 has started.
⚪ 2024-12-24 14:53:46 UTC Artifacts will be uploaded here
⚫ 2024-12-24 14:56:19 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 14:53:57 UTC Pre-commit check linux-x86_64-relwithdebinfo for 39ac247 has started.
⚪ 2024-12-24 14:54:03 UTC Artifacts will be uploaded here
⚫ 2024-12-24 14:56:19 UTC Check cancelled

@lex007in lex007in force-pushed the datashard_persist_del branch from 745515b to cec442e Compare December 24, 2024 14:56
@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 14:57:45 UTC Pre-commit check linux-x86_64-release-asan for b37ef32 has started.
⚪ 2024-12-24 14:57:48 UTC Artifacts will be uploaded here
⚪ 2024-12-24 15:00:57 UTC ya make is running...
⚫ 2024-12-24 16:11:50 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 15:00:23 UTC Pre-commit check linux-x86_64-relwithdebinfo for b37ef32 has started.
⚪ 2024-12-24 15:00:36 UTC Artifacts will be uploaded here
⚪ 2024-12-24 15:04:03 UTC ya make is running...
⚫ 2024-12-24 16:11:50 UTC Check cancelled

@lex007in lex007in force-pushed the datashard_persist_del branch from cec442e to 51593ef Compare December 24, 2024 16:11
@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 16:13:35 UTC Pre-commit check linux-x86_64-relwithdebinfo for 93b5a4c has started.
⚪ 2024-12-24 16:13:49 UTC Artifacts will be uploaded here
⚪ 2024-12-24 16:17:08 UTC ya make is running...
⚫ 2024-12-24 16:18:25 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 16:13:41 UTC Pre-commit check linux-x86_64-release-asan for 93b5a4c has started.
⚪ 2024-12-24 16:13:53 UTC Artifacts will be uploaded here
⚪ 2024-12-24 16:17:21 UTC ya make is running...
⚫ 2024-12-24 16:18:25 UTC Check cancelled

@lex007in lex007in force-pushed the datashard_persist_del branch from 51593ef to e61122d Compare December 24, 2024 16:18
@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 16:20:01 UTC Pre-commit check linux-x86_64-relwithdebinfo for 825d1c2 has started.
⚪ 2024-12-24 16:20:05 UTC Artifacts will be uploaded here
⚪ 2024-12-24 16:23:22 UTC ya make is running...
🟡 2024-12-24 17:55:31 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
31042 28038 0 6 2861 137

⚪ 2024-12-24 17:58:29 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-12-24 18:10:50 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
222 (only retried tests) 103 0 0 1 118

🟢 2024-12-24 18:10:59 UTC Build successful.
🟢 2024-12-24 18:11:18 UTC ydbd size 2.1 GiB changed* by +62.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 9b24da3 merge: 825d1c2 diff diff %
ydbd size 2 235 399 752 Bytes 2 235 463 976 Bytes +62.7 KiB +0.003%
ydbd stripped size 478 085 712 Bytes 478 090 704 Bytes +4.9 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Dec 24, 2024 •
edited
Loading

⚪ 2024-12-24 16:20:02 UTC Pre-commit check linux-x86_64-release-asan for 825d1c2 has started.
⚪ 2024-12-24 16:20:07 UTC Artifacts will be uploaded here
⚪ 2024-12-24 16:23:29 UTC ya make is running...
🟡 2024-12-24 18:03:51 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14064 13949 0 38 20 57

🟢 2024-12-24 18:05:13 UTC Build successful.
🟢 2024-12-24 18:05:39 UTC ydbd size 3.6 GiB changed* by +98.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 9b24da3 merge: 825d1c2 diff diff %
ydbd size 3 866 401 808 Bytes 3 866 503 096 Bytes +98.9 KiB +0.003%
ydbd stripped size 1 351 020 416 Bytes 1 351 043 520 Bytes +22.6 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

azevaykin
azevaykin reviewed Dec 25, 2024
View reviewed changes
@github-actions
Copy link

github-actions bot commented Dec 25, 2024 •
edited
Loading

⚪ 2024-12-25 13:22:35 UTC Pre-commit check linux-x86_64-release-asan for 23a24c8 has started.
⚪ 2024-12-25 13:23:10 UTC Artifacts will be uploaded here
⚪ 2024-12-25 13:26:59 UTC ya make is running...
🟡 2024-12-25 15:03:25 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14100 14010 0 18 14 58

⚪ 2024-12-25 15:04:43 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-25 15:27:31 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
149 (only retried tests) 95 0 3 1 50

⚪ 2024-12-25 15:27:43 UTC ya make is running... (failed tests rerun, try 3)
🟡 2024-12-25 15:40:14 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
101 (only retried tests) 48 0 3 0 50

🟢 2024-12-25 15:40:22 UTC Build successful.
🟡 2024-12-25 15:40:52 UTC ydbd size 3.6 GiB changed* by +151.1 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 46419b6 merge: 23a24c8 diff diff %
ydbd size 3 864 473 640 Bytes 3 864 628 400 Bytes +151.1 KiB +0.004%
ydbd stripped size 1 347 318 064 Bytes 1 347 362 320 Bytes +43.2 KiB +0.003%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Dec 25, 2024 •
edited
Loading

⚪ 2024-12-25 13:22:46 UTC Pre-commit check linux-x86_64-relwithdebinfo for 23a24c8 has started.
⚪ 2024-12-25 13:23:34 UTC Artifacts will be uploaded here
⚪ 2024-12-25 13:27:19 UTC ya make is running...
🟡 2024-12-25 14:58:34 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
31083 28124 0 1 2835 123

⚪ 2024-12-25 15:01:45 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-12-25 15:13:38 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
178 (only retried tests) 59 0 1 1 117

⚪ 2024-12-25 15:13:47 UTC ya make is running... (failed tests rerun, try 3)
🟢 2024-12-25 15:30:11 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
170 (only retried tests) 53 0 0 0 117

🟢 2024-12-25 15:30:21 UTC Build successful.
🟢 2024-12-25 15:30:44 UTC ydbd size 2.1 GiB changed* by +95.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 46419b6 merge: 23a24c8 diff diff %
ydbd size 2 225 994 160 Bytes 2 226 091 408 Bytes +95.0 KiB +0.004%
ydbd stripped size 467 352 912 Bytes 467 361 936 Bytes +8.8 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@lex007in lex007in marked this pull request as ready for review December 25, 2024 13:26
@lex007in lex007in merged commit 2227b46 into ydb-platform:main Dec 25, 2024
12 checks passed
@lex007in lex007in deleted the datashard_persist_del branch December 25, 2024 15:58
@GrigoriyPA GrigoriyPA mentioned this pull request Dec 27, 2024
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Jan 5, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@azevaykin azevaykin azevaykin approved these changes

Assignees

@lex007in lex007in

Labels

not-for-changelog

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@lex007in @azevaykin