Skip to content

Avoid unnecessary ReadStep coordinator subscriptions #13585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 20, 2025

Conversation

snaury
Copy link
Member

@snaury snaury commented Jan 20, 2025

Changelog entry

...

Changelog category

  • Not for changelog (changelog entry is not required)

Additional information

We use ReadStep subscriptions to restore the maximum version that could have been used in snapshot reads by previous generations. While we need the initial subscription response, we don't actually use ReadStep updates, which end up wasting interconnect capacity at coordinator nodes. Unsubscribe from these updates in datashard and timecast as soon as possible, which is a backwards compatible change. Later we could switch to a more efficient API.

Fixes #13548.

@snaury snaury self-assigned this Jan 20, 2025
Copy link

github-actions bot commented Jan 20, 2025

2025-01-20 13:24:11 UTC Pre-commit check linux-x86_64-relwithdebinfo for dde198c has started.
2025-01-20 13:24:26 UTC Artifacts will be uploaded here
2025-01-20 13:27:34 UTC ya make is running...
🟡 2025-01-20 14:17:22 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
18802 17460 0 6 1213 123

2025-01-20 14:18:57 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-01-20 14:30:52 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
230 (only retried tests) 108 0 0 0 122

🟢 2025-01-20 14:30:58 UTC Build successful.
🟢 2025-01-20 14:31:19 UTC ydbd size 2.1 GiB changed* by +7.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 04f4156 merge: dde198c diff diff %
ydbd size 2 216 627 368 Bytes 2 216 634 496 Bytes +7.0 KiB +0.000%
ydbd stripped size 468 629 104 Bytes 468 630 192 Bytes +1.1 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Jan 20, 2025

2025-01-20 13:24:14 UTC Pre-commit check linux-x86_64-release-asan for dde198c has started.
2025-01-20 13:24:26 UTC Artifacts will be uploaded here
2025-01-20 13:27:26 UTC ya make is running...
🟡 2025-01-20 14:31:14 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11379 11325 0 15 9 30

2025-01-20 14:32:14 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-01-20 14:44:13 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
82 (only retried tests) 51 0 1 4 26

2025-01-20 14:44:23 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-01-20 14:56:07 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
53 (only retried tests) 28 0 0 0 25

🟢 2025-01-20 14:56:14 UTC Build successful.
🟢 2025-01-20 14:56:41 UTC ydbd size 3.6 GiB changed* by +18.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 04f4156 merge: dde198c diff diff %
ydbd size 3 852 980 728 Bytes 3 852 999 136 Bytes +18.0 KiB +0.000%
ydbd stripped size 1 346 195 888 Bytes 1 346 204 592 Bytes +8.5 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@snaury snaury marked this pull request as ready for review January 20, 2025 15:46
@snaury snaury requested a review from azevaykin January 20, 2025 15:46
@snaury snaury merged commit dbe9d9b into ydb-platform:main Jan 20, 2025
13 checks passed
@snaury snaury deleted the fix-13548-unnecessary-readsteps branch January 20, 2025 16:35
the-ancient-1 pushed a commit to the-ancient-1/ydb that referenced this pull request Jan 21, 2025
snaury added a commit to snaury/ydb that referenced this pull request Jan 21, 2025
snaury added a commit to snaury/ydb that referenced this pull request Jan 21, 2025
azevaykin pushed a commit to azevaykin/ydb that referenced this pull request Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DataShard: avoid unnecessary ReadStep subscriptions
2 participants