Skip to content

Testing the full export/import pipeline of changefeeds #15588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Mar 12, 2025
Merged

Testing the full export/import pipeline of changefeeds #15588

merged 4 commits into from
Mar 12, 2025

Conversation

stanislav-shchetinin
Copy link
Collaborator

Changelog category

  • Not for changelog (changelog entry is not required)

@stanislav-shchetinin stanislav-shchetinin requested a review from a team as a code owner March 11, 2025 14:39
@stanislav-shchetinin stanislav-shchetinin linked an issue Mar 11, 2025 that may be closed by this pull request
Changefeeds: export-import tests #14132
Closed
5 tasks
@github-actions GitHub Actions
Copy link

🟢 2025-03-11 14:40:14 UTC The validation of the Pull Request description is successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 11, 2025 •
edited
Loading

⚪ 2025-03-11 14:41:24 UTC Pre-commit check linux-x86_64-release-asan for f93ae16 has started.
⚪ 2025-03-11 14:41:39 UTC Artifacts will be uploaded here
⚪ 2025-03-11 14:43:50 UTC ya make is running...
🟢 2025-03-11 15:06:08 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
0 0 0 0 0 0

🟢 2025-03-11 15:06:14 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 11, 2025 •
edited
Loading

⚪ 2025-03-11 14:42:18 UTC Pre-commit check linux-x86_64-relwithdebinfo for f93ae16 has started.
⚪ 2025-03-11 14:42:33 UTC Artifacts will be uploaded here
⚪ 2025-03-11 14:44:50 UTC ya make is running...
🟢 2025-03-11 15:11:15 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
220 220 0 0 0 0

🟢 2025-03-11 15:11:26 UTC Build successful.

@stanislav-shchetinin stanislav-shchetinin marked this pull request as draft March 11, 2025 14:57
CyberROFL
CyberROFL previously approved these changes Mar 11, 2025
View reviewed changes
@stanislav-shchetinin stanislav-shchetinin marked this pull request as ready for review March 12, 2025 16:22
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 12, 2025 •
edited
Loading

⚪ 2025-03-12 16:23:10 UTC Pre-commit check linux-x86_64-release-asan for 32877a1 has started.
⚪ 2025-03-12 16:23:26 UTC Artifacts will be uploaded here
⚪ 2025-03-12 16:26:20 UTC ya make is running...
🟡 2025-03-12 17:32:50 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11951 11879 0 27 11 34

⚪ 2025-03-12 17:34:27 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-12 17:46:51 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
115 (only retried tests) 82 0 0 1 32

🟢 2025-03-12 17:47:01 UTC Build successful.
🟡 2025-03-12 17:47:26 UTC ydbd size 3.8 GiB changed* by +637.1 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 8d225b6 merge: 32877a1 diff diff %
ydbd size 4 044 282 688 Bytes 4 044 935 128 Bytes +637.1 KiB +0.016%
ydbd stripped size 1 399 489 544 Bytes 1 399 611 208 Bytes +118.8 KiB +0.009%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 12, 2025 •
edited
Loading

⚪ 2025-03-12 16:23:19 UTC Pre-commit check linux-x86_64-relwithdebinfo for 32877a1 has started.
⚪ 2025-03-12 16:23:38 UTC Artifacts will be uploaded here
⚪ 2025-03-12 16:26:24 UTC ya make is running...
🟡 2025-03-12 17:17:36 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19430 18094 0 2 1220 114

⚪ 2025-03-12 17:19:50 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-12 17:30:59 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
157 (only retried tests) 56 0 0 0 101

🟢 2025-03-12 17:31:09 UTC Build successful.
🟢 2025-03-12 17:31:31 UTC ydbd size 2.1 GiB changed* by +144 Bytes, which is < 100.0 KiB vs main: OK

ydbd size dash main: 96f1f7b merge: 32877a1 diff diff %
ydbd size 2 298 155 960 Bytes 2 298 156 104 Bytes +144 Bytes +0.000%
ydbd stripped size 481 074 976 Bytes 481 075 040 Bytes +64 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

CyberROFL
CyberROFL reviewed Mar 12, 2025
View reviewed changes
ydb/core/tx/schemeshard/schemeshard_import__create.cpp
@@ -537,8 +537,11 @@ struct TSchemeShard::TImport::TTxProgress: public TSchemeShard::TXxport::TTxBase
<< ", txId# " << txId);

Y_ABORT_UNLESS(item.WaitTxId == InvalidTxId);

auto propose = CreateChangefeedPropose(Self, txId, item);
Y_ABORT_UNLESS(propose);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Сделай, пожалуйста, тест, в котором CreateChangefeedPropose вернет ошибку (например, из-за неправильного формата changefeed-а) и реализуй тут отмену операции (посмотри по коду как и где вызывается Cancel/CancelAndPersist).

@stanislav-shchetinin stanislav-shchetinin merged commit 731ba2d into ydb-platform:main Mar 12, 2025
12 checks passed
@stanislav-shchetinin stanislav-shchetinin mentioned this pull request Mar 12, 2025
Closed
@serbel324 serbel324 mentioned this pull request Mar 13, 2025
Merged
stanislav-shchetinin added a commit to stanislav-shchetinin/ydb that referenced this pull request Mar 14, 2025
@stanislav-shchetinin
Testing the full export/import pipeline of changefeeds (ydb-platform#…
3130be3 
…15588)
@stanislav-shchetinin stanislav-shchetinin mentioned this pull request Mar 14, 2025
Closed
stanislav-shchetinin added a commit to stanislav-shchetinin/ydb that referenced this pull request Mar 17, 2025
@stanislav-shchetinin
Testing the full export/import pipeline of changefeeds (ydb-platform#…
cee456b 
…15588)
@stanislav-shchetinin stanislav-shchetinin mentioned this pull request Mar 17, 2025
Merged
adameat pushed a commit to adameat/ydb that referenced this pull request Mar 20, 2025
@stanislav-shchetinin @adameat
Testing the full export/import pipeline of changefeeds (ydb-platform#…
4110f7c 
…15588)
blinkov pushed a commit that referenced this pull request Mar 21, 2025
@stanislav-shchetinin @blinkov
Testing the full export/import pipeline of changefeeds (#15588)
db2f4a3
Emgariko pushed a commit to Emgariko/ydb that referenced this pull request Mar 24, 2025
@stanislav-shchetinin @Emgariko
Testing the full export/import pipeline of changefeeds (ydb-platform#…
9870648 
…15588)
lberserq pushed a commit to lberserq/ydb that referenced this pull request Mar 28, 2025
@stanislav-shchetinin
Testing the full export/import pipeline of changefeeds (ydb-platform#…
8aa096a 
…15588)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CyberROFL CyberROFL CyberROFL approved these changes

Assignees
No one assigned
Labels
not-for-changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Changefeeds: export-import tests
2 participants
@stanislav-shchetinin @CyberROFL