Skip to content

YQ-39-47 Add EgressDataRate sensor to pq write actor #12846

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jan 14, 2025
Merged

YQ-39-47 Add EgressDataRate sensor to pq write actor #12846

merged 3 commits into from
Jan 14, 2025

Conversation

kardymonds
Copy link
Collaborator

Changelog entry

...

Changelog category

  • Not for changelog (changelog entry is not required)

Additional information

...

@kardymonds kardymonds requested a review from a team as a code owner December 20, 2024 15:07
@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 20, 2024 •
edited
Loading

⚪ 2024-12-20 15:11:40 UTC Pre-commit check linux-x86_64-relwithdebinfo for adf9b33 has started.
⚪ 2024-12-20 15:11:51 UTC Artifacts will be uploaded here
⚪ 2024-12-20 15:14:49 UTC ya make is running...
🟡 2024-12-20 16:11:08 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
28625 25680 0 1 2833 111

⚪ 2024-12-20 16:13:42 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-12-20 16:31:54 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
157 (only retried tests) 48 0 0 0 109

🟢 2024-12-20 16:32:01 UTC Build successful.
🟢 2024-12-20 16:32:19 UTC ydbd size 2.1 GiB changed* by +2.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 33d586d merge: adf9b33 diff diff %
ydbd size 2 235 568 712 Bytes 2 235 570 712 Bytes +2.0 KiB +0.000%
ydbd stripped size 477 930 768 Bytes 477 931 152 Bytes +384 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 20, 2024 •
edited
Loading

⚪ 2024-12-20 15:12:09 UTC Pre-commit check linux-x86_64-release-asan for adf9b33 has started.
⚪ 2024-12-20 15:12:20 UTC Artifacts will be uploaded here
⚪ 2024-12-20 15:15:12 UTC ya make is running...
🟡 2024-12-20 16:15:35 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11940 11872 0 13 10 45

🟢 2024-12-20 16:16:34 UTC Build successful.
🟢 2024-12-20 16:16:56 UTC ydbd size 3.6 GiB changed* by +4.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 33d586d merge: adf9b33 diff diff %
ydbd size 3 866 289 120 Bytes 3 866 293 296 Bytes +4.1 KiB +0.000%
ydbd stripped size 1 350 523 536 Bytes 1 350 524 432 Bytes +896 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

dorooleg
dorooleg reviewed Dec 20, 2024
View reviewed changes
ydb/library/yql/providers/pq/async_io/dq_pq_write_actor.cpp Outdated
@@ -103,6 +103,7 @@ class TDqPqWriteActor : public NActors::TActor<TDqPqWriteActor>, public IDqCompu
InFlyData = task->GetCounter("InFlyData");
AlreadyWritten = task->GetCounter("AlreadyWritten");
FirstContinuationTokenMs = task->GetCounter("FirstContinuationTokenMs");
EgressBytes = task->GetCounter("EgressBytes");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deriv = true?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

сделал также (не deriv) как и EgressBytes в subsystem/task_controller.
По названию это не производная.
Можно и deriv = true и переименовать в EgressDataRate.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А фиг знает почему там не deriv сделали, возможно стоит поправит. EgressDataRate мне тоже нравится

Copy link
Collaborator

@dorooleg dorooleg Dec 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://a.yandex-team.ru/arcadia/contrib/ydb/library/yql/providers/dq/actors/task_controller_impl.h?rev=r15598787#L289-291
А для публичных метрик уже по нормальному сделали на основе subsystem/task_controller/EgressBytes

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Переделал на EgressDataRate

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 3, 2025 •
edited
Loading

⚪ 2025-01-03 09:48:26 UTC Pre-commit check linux-x86_64-release-asan for 6c60833 has started.
⚪ 2025-01-03 09:48:37 UTC Artifacts will be uploaded here
⚪ 2025-01-03 09:51:28 UTC ya make is running...
🟡 2025-01-03 10:44:50 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11347 11286 0 24 9 28

⚪ 2025-01-03 10:45:49 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-01-03 10:59:02 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
101 (only retried tests) 61 0 9 7 24

⚪ 2025-01-03 10:59:10 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-01-03 11:10:29 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
69 (only retried tests) 36 0 3 5 25

🟢 2025-01-03 11:10:36 UTC Build successful.
🟢 2025-01-03 11:11:07 UTC ydbd size 3.6 GiB changed* by +4.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 77b7c35 merge: 6c60833 diff diff %
ydbd size 3 872 511 544 Bytes 3 872 515 768 Bytes +4.1 KiB +0.000%
ydbd stripped size 1 350 376 080 Bytes 1 350 377 040 Bytes +960 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 3, 2025 •
edited
Loading

⚪ 2025-01-03 09:48:37 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6c60833 has started.
⚪ 2025-01-03 09:48:49 UTC Artifacts will be uploaded here
⚪ 2025-01-03 09:51:45 UTC ya make is running...
🟡 2025-01-03 10:38:50 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
21482 20003 0 3 1353 123

⚪ 2025-01-03 10:41:01 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-01-03 10:52:30 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
184 (only retried tests) 64 0 0 0 120

🟢 2025-01-03 10:52:37 UTC Build successful.
🟢 2025-01-03 10:52:56 UTC ydbd size 2.1 GiB changed* by +1.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 77b7c35 merge: 6c60833 diff diff %
ydbd size 2 243 476 656 Bytes 2 243 478 592 Bytes +1.9 KiB +0.000%
ydbd stripped size 472 241 552 Bytes 472 241 872 Bytes +320 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@kardymonds kardymonds changed the title YQ-39-47 Add EgressBytes sensor to pq write actor YQ-39-47 Add EgressDataRate sensor to pq write actor Jan 10, 2025
@kardymonds kardymonds merged commit ecc265c into ydb-platform:main Jan 14, 2025
13 checks passed
kardymonds added a commit to kardymonds/ydb that referenced this pull request Jan 14, 2025
@kardymonds
YQ-39-47 Add EgressDataRate sensor to pq write actor (ydb-platform#12846
9969cb5 
)
azevaykin pushed a commit to azevaykin/ydb that referenced this pull request Feb 3, 2025
@kardymonds @azevaykin
YQ-39-47 Add EgressDataRate sensor to pq write actor (ydb-platform#12846
73ee9cf 
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yumkam yumkam yumkam approved these changes

@dorooleg dorooleg dorooleg approved these changes

Assignees
No one assigned
Labels
not-for-changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kardymonds @yumkam @dorooleg