Skip to content

Conversation

@yumkam
Copy link
Collaborator

@yumkam yumkam commented Nov 22, 2024

Changelog entry

...

Changelog category

  • Not for changelog (changelog entry is not required)

Additional information

Otherwise checkpoint is removed from Pending and second checkpoint can be sent before previous completed in actors, leading to

Query failed with code INTERNAL_ERROR at 2024-11-02T21:25:04.429491Z
Error
contrib/ydb/library/yql/dq/actors/compute/dq_compute_actor_impl.h:788  Pause(): requirement !PendingCheckpoint failed (appeared 1 time at 2024-11-02T21:25:04.367533Z)

@yumkam yumkam changed the title checkpointing vs abort: wait for completion before aborting [WIP] checkpointing vs abort: wait for completion before aborting Nov 22, 2024
@yumkam yumkam changed the title [WIP] checkpointing vs abort: wait for completion before aborting [early WIP] checkpointing vs abort: wait for completion before aborting Nov 22, 2024
@github-actions
Copy link

github-actions bot commented Nov 22, 2024

2024-11-22 16:34:23 UTC Pre-commit check linux-x86_64-relwithdebinfo for 43f2cb5 has started.
2024-11-22 16:34:34 UTC Artifacts will be uploaded here
2024-11-22 16:37:38 UTC ya make is running...
🟡 2024-11-22 17:32:23 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40134 34423 0 3 5595 113

2024-11-22 17:36:01 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-11-22 17:47:44 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
117 (only retried tests) 8 0 1 0 108

2024-11-22 17:47:53 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-11-22 17:59:09 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
109 (only retried tests) 0 0 1 0 108

🟢 2024-11-22 17:59:15 UTC Build successful.
🟢 2024-11-22 17:59:35 UTC ydbd size 2.5 GiB changed* by +800 Bytes, which is < 100.0 KiB vs main: OK

ydbd size dash main: 198242d merge: 43f2cb5 diff diff %
ydbd size 2 691 739 688 Bytes 2 691 740 488 Bytes +800 Bytes +0.000%
ydbd stripped size 482 183 088 Bytes 482 183 408 Bytes +320 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Nov 22, 2024

2024-11-22 16:34:40 UTC Pre-commit check linux-x86_64-release-asan for 43f2cb5 has started.
2024-11-22 16:34:52 UTC Artifacts will be uploaded here
2024-11-22 16:38:00 UTC ya make is running...
🟡 2024-11-22 17:41:30 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
8908 8839 0 13 7 49

🟢 2024-11-22 17:42:18 UTC Build successful.
🟢 2024-11-22 17:42:44 UTC ydbd size 4.9 GiB changed* by +512 Bytes, which is < 100.0 KiB vs main: OK

ydbd size dash main: 198242d merge: 43f2cb5 diff diff %
ydbd size 5 285 188 648 Bytes 5 285 189 160 Bytes +512 Bytes +0.000%
ydbd stripped size 1 359 580 208 Bytes 1 359 580 528 Bytes +320 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

++*Metrics.StorageError;
}
if (checkpoint.GotAllAcknowledges()) {
if (checkpoint.GetStats().Aborted) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А эта checkpoint структура постоянно пересоздается? Не нужно обнулить здесь Aborted?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Вместе с чекпоинтом, не переиспользуется

@yumkam yumkam changed the title [early WIP] checkpointing vs abort: wait for completion before aborting checkpointing vs abort: wait for completion before aborting Nov 25, 2024
@github-actions
Copy link

github-actions bot commented Nov 25, 2024

2024-11-25 08:49:01 UTC Pre-commit check linux-x86_64-relwithdebinfo for a4ce97d has started.
2024-11-25 08:49:56 UTC Artifacts will be uploaded here
2024-11-25 08:53:27 UTC ya make is running...
🟡 2024-11-25 09:51:53 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
24465 21623 0 2 2725 115

2024-11-25 09:53:55 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-11-25 10:06:00 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
122 (only retried tests) 12 0 1 0 109

2024-11-25 10:06:09 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-11-25 10:17:53 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
110 (only retried tests) 1 0 1 0 108

🟢 2024-11-25 10:18:00 UTC Build successful.
🟢 2024-11-25 10:18:24 UTC ydbd size 2.5 GiB changed* by +4.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: b07b2ad merge: a4ce97d diff diff %
ydbd size 2 690 049 216 Bytes 2 690 053 304 Bytes +4.0 KiB +0.000%
ydbd stripped size 482 184 656 Bytes 482 185 488 Bytes +832 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Nov 25, 2024

2024-11-25 08:49:51 UTC Pre-commit check linux-x86_64-release-asan for a4ce97d has started.
2024-11-25 08:50:02 UTC Artifacts will be uploaded here
2024-11-25 08:52:56 UTC ya make is running...
🟡 2024-11-25 10:15:43 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
8908 8843 0 14 5 46

🟢 2024-11-25 10:16:27 UTC Build successful.
🟢 2024-11-25 10:17:12 UTC ydbd size 4.9 GiB changed* by +7.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: b07b2ad merge: a4ce97d diff diff %
ydbd size 5 284 305 560 Bytes 5 284 312 904 Bytes +7.2 KiB +0.000%
ydbd stripped size 1 359 965 488 Bytes 1 359 967 088 Bytes +1.6 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Nov 25, 2024

2024-11-25 13:47:04 UTC Pre-commit check linux-x86_64-release-asan for ee33bc8 has started.
2024-11-25 13:47:14 UTC Artifacts will be uploaded here
2024-11-25 13:50:10 UTC ya make is running...
🟡 2024-11-25 14:51:28 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
8906 8841 0 10 8 47

🟢 2024-11-25 14:52:13 UTC Build successful.
🟢 2024-11-25 14:52:41 UTC ydbd size 4.9 GiB changed* by +7.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: a458b35 merge: ee33bc8 diff diff %
ydbd size 5 284 389 392 Bytes 5 284 396 680 Bytes +7.1 KiB +0.000%
ydbd stripped size 1 359 969 968 Bytes 1 359 971 504 Bytes +1.5 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Nov 25, 2024

2024-11-25 13:47:24 UTC Pre-commit check linux-x86_64-relwithdebinfo for ee33bc8 has started.
2024-11-25 13:47:35 UTC Artifacts will be uploaded here
2024-11-25 13:50:41 UTC ya make is running...
🟡 2024-11-25 14:49:37 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
24464 21629 0 3 2720 112

2024-11-25 14:51:35 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-11-25 15:03:32 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
116 (only retried tests) 9 0 0 0 107

🟢 2024-11-25 15:03:38 UTC Build successful.
🟢 2024-11-25 15:04:01 UTC ydbd size 2.5 GiB changed* by +4.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: a458b35 merge: ee33bc8 diff diff %
ydbd size 2 690 088 296 Bytes 2 690 092 376 Bytes +4.0 KiB +0.000%
ydbd stripped size 482 185 808 Bytes 482 186 640 Bytes +832 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@yumkam yumkam marked this pull request as ready for review November 25, 2024 14:01
@yumkam yumkam requested a review from a team as a code owner November 25, 2024 14:01
@yumkam yumkam requested a review from dorooleg November 25, 2024 14:20
@yumkam yumkam merged commit ec75229 into ydb-platform:main Nov 25, 2024
10 checks passed
yumkam added a commit to yumkam/ydb that referenced this pull request Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants