Skip to content

healthcheck config #14860

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 7, 2025
Merged

Conversation

StekPerepolnen
Copy link
Collaborator

@StekPerepolnen StekPerepolnen commented Feb 20, 2025

Changelog entry

added healthcheck config

Changelog category

  • Improvement

Copy link

github-actions bot commented Feb 20, 2025

🟢 2025-02-20 17:04:56 UTC The validation of the Pull Request description is successful.

Copy link

github-actions bot commented Feb 20, 2025

2025-02-20 17:05:08 UTC Pre-commit check linux-x86_64-release-asan for 93a0f61 has started.
2025-02-20 17:05:24 UTC Artifacts will be uploaded here
2025-02-20 17:08:44 UTC ya make is running...
🟡 2025-02-20 18:53:11 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13800 13720 0 32 11 37

2025-02-20 18:55:34 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-02-20 19:08:43 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
126 (only retried tests) 86 0 1 3 36

2025-02-20 19:08:52 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-02-20 19:21:04 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
71 (only retried tests) 36 0 0 2 33

🟢 2025-02-20 19:21:11 UTC Build successful.
🟢 2025-02-20 19:21:42 UTC ydbd size 3.6 GiB changed* by +98.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 91bc1bd merge: 93a0f61 diff diff %
ydbd size 3 895 668 296 Bytes 3 895 768 792 Bytes +98.1 KiB +0.003%
ydbd stripped size 1 364 068 448 Bytes 1 364 109 760 Bytes +40.3 KiB +0.003%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Feb 20, 2025

2025-02-20 17:12:30 UTC Pre-commit check linux-x86_64-relwithdebinfo for 93a0f61 has started.
2025-02-20 17:13:17 UTC Artifacts will be uploaded here
2025-02-20 17:16:34 UTC ya make is running...
🟡 2025-02-20 18:52:43 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
28038 25434 0 10 2460 134

2025-02-20 18:56:00 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-02-20 19:09:28 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
213 (only retried tests) 83 0 0 2 128

🟢 2025-02-20 19:09:38 UTC Build successful.
🟢 2025-02-20 19:10:01 UTC ydbd size 2.1 GiB changed* by +43.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 91bc1bd merge: 93a0f61 diff diff %
ydbd size 2 239 508 360 Bytes 2 239 553 112 Bytes +43.7 KiB +0.002%
ydbd stripped size 474 551 128 Bytes 474 559 448 Bytes +8.1 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@vporyadke vporyadke linked an issue Feb 25, 2025 that may be closed by this pull request
Copy link

github-actions bot commented Feb 27, 2025

2025-02-27 15:20:23 UTC Pre-commit check linux-x86_64-release-asan for 87abbd2 has started.
2025-02-27 15:20:39 UTC Artifacts will be uploaded here
2025-02-27 15:23:55 UTC ya make is running...
🟡 2025-02-27 17:07:39 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13717 13517 0 131 38 31

2025-02-27 17:09:07 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-02-27 17:24:48 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
268 (only retried tests) 233 0 6 1 28

2025-02-27 17:25:02 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-02-27 17:30:08 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
66 (only retried tests) 39 0 0 0 27

🟢 2025-02-27 17:30:15 UTC Build successful.
🟢 2025-02-27 17:30:42 UTC ydbd size 3.7 GiB changed* by +99.6 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 14389e2 merge: 87abbd2 diff diff %
ydbd size 3 985 411 184 Bytes 3 985 513 192 Bytes +99.6 KiB +0.003%
ydbd stripped size 1 387 765 576 Bytes 1 387 807 720 Bytes +41.2 KiB +0.003%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Feb 27, 2025

2025-02-27 15:20:56 UTC Pre-commit check linux-x86_64-relwithdebinfo for 87abbd2 has started.
2025-02-27 15:21:11 UTC Artifacts will be uploaded here
2025-02-27 15:24:19 UTC ya make is running...
🟡 2025-02-27 16:47:18 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
28122 25525 0 1 2465 131

2025-02-27 16:50:30 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-02-27 17:01:53 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
198 (only retried tests) 74 0 0 0 124

🟢 2025-02-27 17:02:00 UTC Build successful.
🟢 2025-02-27 17:02:17 UTC ydbd size 2.1 GiB changed* by +47.8 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 14389e2 merge: 87abbd2 diff diff %
ydbd size 2 286 685 096 Bytes 2 286 734 032 Bytes +47.8 KiB +0.002%
ydbd stripped size 479 417 824 Bytes 479 430 304 Bytes +12.2 KiB +0.003%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link
Member

@CyberROFL CyberROFL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also need to add a subscription to the config. Or what's the plan here? @vporyadke FYI

@adameat adameat removed their request for review March 3, 2025 03:02
Copy link

github-actions bot commented Mar 6, 2025

2025-03-06 13:16:28 UTC Pre-commit check linux-x86_64-release-asan for dfa9014 has started.
2025-03-06 13:17:01 UTC Artifacts will be uploaded here
2025-03-06 13:20:53 UTC ya make is running...
2025-03-06 13:24:43 UTC Check cancelled

Copy link

github-actions bot commented Mar 6, 2025

2025-03-06 13:18:16 UTC Pre-commit check linux-x86_64-relwithdebinfo for dfa9014 has started.
2025-03-06 13:18:31 UTC Artifacts will be uploaded here
2025-03-06 13:21:42 UTC ya make is running...
2025-03-06 13:24:41 UTC Check cancelled

Copy link

github-actions bot commented Mar 6, 2025

2025-03-06 13:28:22 UTC Pre-commit check linux-x86_64-release-asan for 366510b has started.
2025-03-06 13:28:37 UTC Artifacts will be uploaded here
2025-03-06 13:32:16 UTC ya make is running...
🟡 2025-03-06 15:11:33 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13762 13633 0 83 13 33

2025-03-06 15:13:06 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-06 15:26:29 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
181 (only retried tests) 144 0 3 2 32

2025-03-06 15:26:41 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-06 15:38:23 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
66 (only retried tests) 33 0 0 1 32

🟢 2025-03-06 15:38:32 UTC Build successful.
🟢 2025-03-06 15:39:02 UTC ydbd size 3.7 GiB changed* by +46.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 9187047 merge: 366510b diff diff %
ydbd size 3 995 105 248 Bytes 3 995 152 760 Bytes +46.4 KiB +0.001%
ydbd stripped size 1 389 063 240 Bytes 1 389 081 672 Bytes +18.0 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 6, 2025

2025-03-06 13:28:25 UTC Pre-commit check linux-x86_64-relwithdebinfo for 366510b has started.
2025-03-06 13:28:39 UTC Artifacts will be uploaded here
2025-03-06 13:31:56 UTC ya make is running...
🟢 2025-03-06 15:01:44 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
28167 25574 0 0 2471 122

🟢 2025-03-06 15:04:13 UTC Build successful.
🟢 2025-03-06 15:04:31 UTC ydbd size 2.1 GiB changed* by +37.6 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 9187047 merge: 366510b diff diff %
ydbd size 2 293 953 776 Bytes 2 293 992 248 Bytes +37.6 KiB +0.002%
ydbd stripped size 480 565 568 Bytes 480 574 400 Bytes +8.6 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@@ -1768,6 +1768,14 @@ message THiveConfig {
optional uint64 NodeRestartsForPenalty = 85 [default = 3];
}

message THealthCheckConfig {
optional uint32 NodeRestartsPerPeriodYellowThreshold = 1 [default = 10];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do I set the duration of the "period"?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that part is configured through Hive config

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very unexpected. Do you have an improvement plan?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 05:42:20 UTC Pre-commit check linux-x86_64-release-asan for 98257fc has started.
2025-03-07 05:42:35 UTC Artifacts will be uploaded here
2025-03-07 05:44:57 UTC Check cancelled

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 05:42:21 UTC Pre-commit check linux-x86_64-relwithdebinfo for 98257fc has started.
2025-03-07 05:42:36 UTC Artifacts will be uploaded here
2025-03-07 05:44:57 UTC Check cancelled

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 05:46:19 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6b6b50f has started.
2025-03-07 05:46:33 UTC Artifacts will be uploaded here
2025-03-07 05:48:01 UTC Check cancelled

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 05:46:19 UTC Pre-commit check linux-x86_64-release-asan for 6b6b50f has started.
2025-03-07 05:46:35 UTC Artifacts will be uploaded here
2025-03-07 05:48:01 UTC Check cancelled

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 05:52:22 UTC Pre-commit check linux-x86_64-relwithdebinfo for 407ac86 has started.
2025-03-07 05:52:37 UTC Artifacts will be uploaded here
2025-03-07 06:12:17 UTC ya make is running...
🟡 2025-03-07 07:41:27 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
28172 25571 0 2 2477 122

2025-03-07 07:43:51 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-07 07:55:06 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
187 (only retried tests) 80 0 0 0 107

🟢 2025-03-07 07:55:13 UTC Build successful.
🟢 2025-03-07 07:55:32 UTC ydbd size 2.1 GiB changed* by +47.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 653965a merge: 407ac86 diff diff %
ydbd size 2 293 420 832 Bytes 2 293 469 696 Bytes +47.7 KiB +0.002%
ydbd stripped size 480 451 296 Bytes 480 460 896 Bytes +9.4 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 05:52:22 UTC Pre-commit check linux-x86_64-release-asan for 407ac86 has started.
2025-03-07 05:52:36 UTC Artifacts will be uploaded here
2025-03-07 06:12:26 UTC ya make is running...
🟡 2025-03-07 07:58:27 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13767 13558 0 147 25 37

2025-03-07 07:59:45 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-07 08:14:18 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
270 (only retried tests) 221 0 10 7 32

2025-03-07 08:14:28 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-07 08:26:10 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
85 (only retried tests) 49 0 0 7 29

🟢 2025-03-07 08:26:20 UTC Build successful.
🟢 2025-03-07 08:26:48 UTC ydbd size 3.7 GiB changed* by +77.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 653965a merge: 407ac86 diff diff %
ydbd size 3 994 259 064 Bytes 3 994 337 928 Bytes +77.0 KiB +0.002%
ydbd stripped size 1 388 747 112 Bytes 1 388 771 368 Bytes +23.7 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

vporyadke
vporyadke previously approved these changes Mar 7, 2025
@@ -143,6 +143,7 @@ message TConfigItem {
GroupedMemoryLimiterConfig = 82;
ReplicationConfigItem = 83;
CompPrioritiesConfig = 85;
HealthCheckConfigItem = 89;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
HealthCheckConfigItem = 89;
HealthCheckConfigItem = 86;

?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the item number should match the configuration number in AppConfig so that the console knows which subscribers to send updates to. If the number is changed, the test fails.

Co-authored-by: Ilnaz Nizametdinov <i.nizametdinov@gmail.com>
Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 10:52:02 UTC Pre-commit check linux-x86_64-release-asan for 3a97be8 has started.
2025-03-07 10:52:36 UTC Artifacts will be uploaded here
2025-03-07 10:56:22 UTC ya make is running...
🟡 2025-03-07 12:41:58 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13769 13567 0 135 30 37

2025-03-07 12:43:15 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-07 12:56:17 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
260 (only retried tests) 219 0 5 3 33

2025-03-07 12:56:32 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-07 13:16:45 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
72 (only retried tests) 40 0 0 0 32

🟢 2025-03-07 13:16:55 UTC Build successful.
🟢 2025-03-07 13:17:28 UTC ydbd size 3.7 GiB changed* by +73.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 301cd78 merge: 3a97be8 diff diff %
ydbd size 3 994 599 656 Bytes 3 994 674 440 Bytes +73.0 KiB +0.002%
ydbd stripped size 1 388 829 736 Bytes 1 388 849 896 Bytes +19.7 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 10:53:17 UTC Pre-commit check linux-x86_64-relwithdebinfo for 3a97be8 has started.
2025-03-07 10:53:28 UTC Artifacts will be uploaded here
2025-03-07 10:56:46 UTC ya make is running...
🟡 2025-03-07 12:37:10 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
28174 25585 0 4 2471 114

2025-03-07 12:39:32 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-07 12:52:01 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
169 (only retried tests) 60 0 0 0 109

🟢 2025-03-07 12:52:09 UTC Build successful.
🟢 2025-03-07 12:54:42 UTC ydbd size 2.1 GiB changed* by +44.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 301cd78 merge: 3a97be8 diff diff %
ydbd size 2 293 646 360 Bytes 2 293 691 496 Bytes +44.1 KiB +0.002%
ydbd stripped size 480 484 640 Bytes 480 490 336 Bytes +5.6 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@StekPerepolnen StekPerepolnen requested a review from CyberROFL March 7, 2025 11:22
@StekPerepolnen StekPerepolnen merged commit c7ef100 into ydb-platform:main Mar 7, 2025
12 checks passed
@StekPerepolnen StekPerepolnen deleted the healthcheck-config branch March 7, 2025 16:37
StekPerepolnen added a commit to StekPerepolnen/ydb that referenced this pull request Mar 19, 2025
Co-authored-by: Ilnaz Nizametdinov <i.nizametdinov@gmail.com>
adameat pushed a commit to adameat/ydb that referenced this pull request Mar 20, 2025
Co-authored-by: Ilnaz Nizametdinov <i.nizametdinov@gmail.com>
blinkov pushed a commit that referenced this pull request Mar 21, 2025
Co-authored-by: Ilnaz Nizametdinov <i.nizametdinov@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Config for Healtcheck
3 participants