Skip to content

Throttle compute-intensive tasks #4891

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Aug 2, 2024
Merged

Throttle compute-intensive tasks #4891

merged 9 commits into from
Aug 2, 2024

Conversation

@ssmike
Copy link
Contributor

@ssmike ssmike commented May 27, 2024

Changelog entry

Implement cpu limiter in node service

Changelog category

  • New feature

@github-actions
Copy link

github-actions bot commented May 27, 2024 •
edited
Loading

⚪ 2024-05-27 14:00:01 UTC Pre-commit check for 18eeef7 has started.
⚪ 2024-05-27 14:02:31 UTC Build linux-x86_64-release-asan is running...
🔴 2024-05-27 14:09:21 UTC Build failed. see the build logs.
🔴 2024-05-27 14:09:23 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented May 27, 2024 •
edited
Loading

⚪ 2024-05-27 14:02:28 UTC Pre-commit check for 18eeef7 has started.
⚪ 2024-05-27 14:05:01 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-05-27 14:12:12 UTC Build failed. see the build logs.

@github-actions
Copy link

github-actions bot commented May 27, 2024 •
edited
Loading

⚪ 2024-05-27 14:02:30 UTC Pre-commit check for 18eeef7 has started.
⚪ 2024-05-27 14:04:59 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-05-27 14:11:31 UTC Build failed. see the build logs.
🔴 2024-05-27 14:11:32 UTC Tests run skipped.

@ssmike
Copy link
Contributor Author

ssmike commented May 27, 2024

#4889

@github-actions
Copy link

github-actions bot commented May 28, 2024 •
edited
Loading

⚪ 2024-05-28 04:15:43 UTC Pre-commit check for f77fad4 has started.
⚪ 2024-05-28 04:18:36 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-05-28 04:55:18 UTC Build failed. see the build logs.

@github-actions
Copy link

github-actions bot commented May 28, 2024 •
edited
Loading

⚪ 2024-05-28 04:15:52 UTC Pre-commit check for f77fad4 has started.
⚪ 2024-05-28 04:18:44 UTC Build linux-x86_64-release-asan is running...
🔴 2024-05-28 04:58:43 UTC Build failed. see the build logs.
🔴 2024-05-28 04:58:44 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented May 28, 2024 •
edited
Loading

⚪ 2024-05-28 04:15:54 UTC Pre-commit check for f77fad4 has started.
⚪ 2024-05-28 04:18:41 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-05-28 04:58:26 UTC Build failed. see the build logs.
🔴 2024-05-28 04:58:27 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented May 30, 2024 •
edited
Loading

⚪ 2024-05-30 13:20:03 UTC Pre-commit check for d5373e6 has started.
⚪ 2024-05-30 13:22:42 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-05-30 13:59:43 UTC Build failed. see the build logs.

@github-actions
Copy link

github-actions bot commented May 30, 2024 •
edited
Loading

⚪ 2024-05-30 13:22:16 UTC Pre-commit check for d5373e6 has started.
⚪ 2024-05-30 13:25:04 UTC Build linux-x86_64-release-asan is running...
🔴 2024-05-30 14:05:06 UTC Build failed. see the build logs.
🔴 2024-05-30 14:05:08 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented May 30, 2024 •
edited
Loading

⚪ 2024-05-30 13:50:00 UTC Pre-commit check for d5373e6 has started.
⚪ 2024-05-30 13:52:44 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-05-30 14:30:25 UTC Build failed. see the build logs.
🔴 2024-05-30 14:30:27 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented Jun 4, 2024 •
edited
Loading

⚪ 2024-06-04 13:35:03 UTC Pre-commit check for 298454e has started.
⚪ 2024-06-04 13:37:56 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-06-04 14:14:11 UTC Build failed. see the build logs.

@github-actions
Copy link

github-actions bot commented Jun 4, 2024 •
edited
Loading

⚪ 2024-06-04 13:36:08 UTC Pre-commit check for 298454e has started.
⚪ 2024-06-04 13:38:55 UTC Build linux-x86_64-release-asan is running...
🔴 2024-06-04 14:15:51 UTC Build failed. see the build logs.
🔴 2024-06-04 14:15:52 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented Jun 4, 2024 •
edited
Loading

⚪ 2024-06-04 13:47:36 UTC Pre-commit check for 298454e has started.
⚪ 2024-06-04 13:50:21 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-06-04 14:29:16 UTC Build failed. see the build logs.
🔴 2024-06-04 14:29:18 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented Jun 4, 2024 •
edited
Loading

⚪ 2024-06-04 19:57:27 UTC Pre-commit check for e9116af has started.
⚪ 2024-06-04 20:00:15 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-06-04 20:39:26 UTC Build failed. see the build logs.

@github-actions
Copy link

github-actions bot commented Jun 4, 2024 •
edited
Loading

⚪ 2024-06-04 20:05:41 UTC Pre-commit check for e9116af has started.
⚪ 2024-06-04 20:08:36 UTC Build linux-x86_64-release-asan is running...
🔴 2024-06-04 20:50:54 UTC Build failed. see the build logs.
🔴 2024-06-04 20:50:55 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented Jun 4, 2024 •
edited
Loading

⚪ 2024-06-04 20:07:43 UTC Pre-commit check for e9116af has started.
⚪ 2024-06-04 20:10:33 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-06-04 20:50:46 UTC Build failed. see the build logs.
🔴 2024-06-04 20:50:47 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented Jun 5, 2024 •
edited
Loading

⚪ 2024-06-05 01:50:44 UTC Pre-commit check for e327f9d has started.
⚪ 2024-06-05 01:53:33 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-06-05 01:56:02 UTC Build failed. see the build logs.

@github-actions
Copy link

github-actions bot commented Jun 5, 2024 •
edited
Loading

⚪ 2024-06-05 01:50:51 UTC Pre-commit check for e327f9d has started.
⚪ 2024-06-05 01:53:43 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-06-05 01:56:28 UTC Build failed. see the build logs.
🔴 2024-06-05 01:56:30 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented Jun 5, 2024 •
edited
Loading

⚪ 2024-06-05 01:51:06 UTC Pre-commit check for e327f9d has started.
⚪ 2024-06-05 01:54:00 UTC Build linux-x86_64-release-asan is running...
🔴 2024-06-05 01:56:51 UTC Build failed. see the build logs.
🔴 2024-06-05 01:56:52 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented Jun 5, 2024 •
edited
Loading

⚪ 2024-06-05 19:08:02 UTC Pre-commit check for 2658cee has started.
⚪ 2024-06-05 19:10:50 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-06-05 19:46:24 UTC Build failed. see the build logs.

@github-actions
Copy link

github-actions bot commented Jun 5, 2024 •
edited
Loading

⚪ 2024-06-05 19:08:45 UTC Pre-commit check for 2658cee has started.
⚪ 2024-06-05 19:11:38 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-06-05 19:49:18 UTC Build failed. see the build logs.
🔴 2024-06-05 19:49:19 UTC Tests run skipped.

@github-actions
Copy link

github-actions bot commented Jun 5, 2024 •
edited
Loading

⚪ 2024-06-05 19:08:48 UTC Pre-commit check for 2658cee has started.
⚪ 2024-06-05 19:11:35 UTC Build linux-x86_64-release-asan is running...
🔴 2024-06-05 19:48:56 UTC Build failed. see the build logs.
🔴 2024-06-05 19:48:57 UTC Tests run skipped.

@ssmike ssmike force-pushed the cpu-quotas branch 2 times, most recently from 0642a8e to ba6c45f Compare June 10, 2024 20:50
@ssmike ssmike force-pushed the cpu-quotas branch 3 times, most recently from ea1d8cd to d118c98 Compare June 26, 2024 19:57
@github-actions
Copy link

github-actions bot commented Jun 26, 2024 •
edited
Loading

⚪ 2024-06-26 20:01:06 UTC Pre-commit check for b32acb4 has started.
⚪ 2024-06-26 20:03:54 UTC Build linux-x86_64-release-asan is running...
⚫ 2024-06-26 20:28:35 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Jul 31, 2024 •
edited
Loading

⚪ 2024-07-31 11:47:17 UTC Pre-commit check for cc9c01c has started.
⚪ 2024-07-31 11:50:28 UTC Check linux-x86_64-relwithdebinfo is running...
⚫ 2024-07-31 12:50:36 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Jul 31, 2024 •
edited
Loading

⚪ 2024-07-31 12:55:48 UTC Pre-commit check for 52480d3 has started.
⚪ 2024-07-31 12:58:54 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-07-31 13:40:32 UTC Build successful.

@github-actions
Copy link

github-actions bot commented Jul 31, 2024 •
edited
Loading

⚪ 2024-07-31 12:57:05 UTC Pre-commit check for 52480d3 has started.
⚪ 2024-07-31 13:00:47 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-07-31 14:52:59 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39768 34462 0 2 5286 18

⚪ 2024-07-31 14:57:15 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-07-31 15:04:15 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15 (only retried tests) 5 0 1 0 9

⚪ 2024-07-31 15:07:52 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-07-31 15:14:53 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11 (only retried tests) 2 0 1 0 8

🟢 2024-07-31 15:18:37 UTC Build successful.
🟡 2024-07-31 15:19:14 UTC ydbd size 8.1 GiB changed* by +1.2 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: e464b73 merge: 52480d3 diff diff %
ydbd size 8 678 737 472 Bytes 8 680 044 016 Bytes +1.2 MiB +0.015%
ydbd stripped size 471 916 032 Bytes 471 976 384 Bytes +58.9 KiB +0.013%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Jul 31, 2024 •
edited
Loading

⚪ 2024-07-31 12:58:05 UTC Pre-commit check for 52480d3 has started.
⚪ 2024-07-31 13:01:14 UTC Check linux-x86_64-release-asan is running...
🔴 2024-07-31 15:18:08 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11033 10988 0 11 22 12

🟢 2024-07-31 15:19:24 UTC Build successful.
🔴 2024-07-31 15:19:55 UTC ydbd size 5.5 GiB changed* by +276.4 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: e464b73 merge: 52480d3 diff diff %
ydbd size 5 571 429 760 Bytes 5 861 238 520 Bytes +276.4 MiB +5.202%
ydbd stripped size 1 199 304 144 Bytes 1 467 759 472 Bytes +256.0 MiB +22.384%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Jul 31, 2024 •
edited
Loading

⚪ 2024-07-31 19:08:23 UTC Pre-commit check for 51ddbec has started.
⚪ 2024-07-31 19:11:40 UTC Check linux-x86_64-release-asan is running...
🔴 2024-07-31 21:34:09 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11034 10990 0 9 22 13

🟢 2024-07-31 21:35:26 UTC Build successful.
🟡 2024-07-31 21:35:55 UTC ydbd size 5.4 GiB changed* by +728.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6316570 merge: 51ddbec diff diff %
ydbd size 5 841 313 560 Bytes 5 842 059 904 Bytes +728.9 KiB +0.013%
ydbd stripped size 1 463 911 024 Bytes 1 464 079 856 Bytes +164.9 KiB +0.012%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Jul 31, 2024 •
edited
Loading

⚪ 2024-07-31 19:08:27 UTC Pre-commit check for 51ddbec has started.
⚪ 2024-07-31 19:11:49 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-07-31 19:52:31 UTC Build successful.

@github-actions
Copy link

github-actions bot commented Jul 31, 2024 •
edited
Loading

⚪ 2024-07-31 19:08:42 UTC Pre-commit check for 51ddbec has started.
⚪ 2024-07-31 19:11:55 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-07-31 21:31:34 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39793 34465 0 5 5309 14

⚪ 2024-07-31 21:36:19 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-07-31 21:47:27 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
25 (only retried tests) 10 0 2 5 8

⚪ 2024-07-31 21:50:47 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🟢 2024-07-31 21:57:19 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14 (only retried tests) 6 0 0 0 8

🟢 2024-07-31 22:00:43 UTC Build successful.
🟡 2024-07-31 22:01:20 UTC ydbd size 8.1 GiB changed* by +1.2 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6316570 merge: 51ddbec diff diff %
ydbd size 8 672 038 392 Bytes 8 673 340 864 Bytes +1.2 MiB +0.015%
ydbd stripped size 472 048 680 Bytes 472 108 968 Bytes +58.9 KiB +0.013%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 1, 2024 •
edited
Loading

⚪ 2024-08-01 12:59:13 UTC Pre-commit check for 6ef277a has started.
⚪ 2024-08-01 13:02:32 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-01 15:32:24 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39795 34465 0 9 5308 13

⚪ 2024-08-01 15:36:56 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-01 15:48:00 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
27 (only retried tests) 9 0 6 4 8

⚪ 2024-08-01 15:51:34 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-01 15:59:39 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
18 (only retried tests) 6 0 4 0 8

🟢 2024-08-01 16:03:22 UTC Build successful.
🟡 2024-08-01 16:03:57 UTC ydbd size 8.1 GiB changed* by +1.2 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: b17ef08 merge: 6ef277a diff diff %
ydbd size 8 673 312 640 Bytes 8 674 614 624 Bytes +1.2 MiB +0.015%
ydbd stripped size 472 099 752 Bytes 472 152 584 Bytes +51.6 KiB +0.011%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 1, 2024 •
edited
Loading

⚪ 2024-08-01 13:00:28 UTC Pre-commit check for 6ef277a has started.
⚪ 2024-08-01 13:05:19 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-01 15:21:12 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11043 10991 0 2 36 14

🟢 2024-08-01 15:22:28 UTC Build successful.
🟡 2024-08-01 15:22:57 UTC ydbd size 5.4 GiB changed* by +738.5 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6b1225c merge: 6ef277a diff diff %
ydbd size 5 842 235 624 Bytes 5 842 991 808 Bytes +738.5 KiB +0.013%
ydbd stripped size 1 464 081 552 Bytes 1 464 251 888 Bytes +166.3 KiB +0.012%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Aug 1, 2024 •
edited
Loading

⚪ 2024-08-01 12:59:31 UTC Pre-commit check for 6ef277a has started.
⚪ 2024-08-01 13:04:08 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-01 13:44:52 UTC Build successful.

@gridnevvvit gridnevvvit self-requested a review August 1, 2024 17:44
@ssmike ssmike merged commit e277963 into ydb-platform:main Aug 2, 2024
@ssmike ssmike deleted the cpu-quotas branch August 2, 2024 06:15
@StekPerepolnen StekPerepolnen mentioned this pull request Aug 8, 2024
Merged
This was referenced Aug 8, 2024
Merged
Merged
ssmike added a commit to ssmike/ydb that referenced this pull request Sep 9, 2024
@ssmike
Throttle compute-intensive tasks (ydb-platform#4891)
0390b8f
lll-phill-lll pushed a commit to lll-phill-lll/ydb that referenced this pull request Sep 9, 2024
@ssmike @lll-phill-lll
Throttle compute-intensive tasks (ydb-platform#4891)
72c7ab5
ssmike added a commit to ssmike/ydb that referenced this pull request Sep 10, 2024
@ssmike
Throttle compute-intensive tasks (ydb-platform#4891)
adc972c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gridnevvvit gridnevvvit gridnevvvit approved these changes

Assignees

No one assigned

Labels

new-feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ssmike @gridnevvvit