Autoscaling control plane #4486

niksaveliev · 2024-05-14T07:51:33Z

Changelog entry

Changelog category

Not for changelog (changelog entry is not required)

Additional information

...

github-actions · 2024-05-14T07:54:48Z

⚪ 2024-05-14 07:54:48 UTC Pre-commit check for 1bfbba5 has started.
⚪ 2024-05-14 07:54:50 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-14 08:32:20 UTC Build successful.

github-actions · 2024-05-14T07:54:59Z

⚪ 2024-05-14 07:54:59 UTC Pre-commit check for 1bfbba5 has started.
⚪ 2024-05-14 07:55:01 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-14 08:35:04 UTC Build successful.
⚪ 2024-05-14 08:36:58 UTC Tests are running...
🔴 2024-05-14 10:33:33 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
73502	60298	0	8	13188	8

github-actions · 2024-05-14T07:55:10Z

⚪ 2024-05-14 07:55:10 UTC Pre-commit check for 1bfbba5 has started.
⚪ 2024-05-14 07:55:12 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-14 08:36:49 UTC Build successful.
⚪ 2024-05-14 08:38:36 UTC Tests are running...
🔴 2024-05-14 10:28:56 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14652	14539	0	48	57	8

nshestakov · 2024-05-14T08:40:41Z

ydb/public/lib/ydb_cli/commands/ydb_service_topic.cpp

+        auto thresholdTime = config.Opts->AddLongOption("autoscale-threshold-time", "Duration in seconds of high or low load before automatically scale the number of partitions")
+            .Optional()
+            .StoreResult(&ScaleThresholdTime_);
+        auto upThresholdPercent = config.Opts->AddLongOption("scale-up-threshold-percent", "The load percentage at which the number of partitions will increase.")


а не надо ли и здесь иметь префикс autoscale ?

не знаю) Но имена параметров все-равно еще будем отдельно обсуждать

ydb/public/lib/ydb_cli/commands/ydb_service_topic.cpp

github-actions · 2024-05-14T12:59:29Z

⚪ 2024-05-14 12:59:29 UTC Pre-commit check for 805e3ad has started.
⚪ 2024-05-14 12:59:31 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-14 13:40:35 UTC Build successful.
⚪ 2024-05-14 13:42:18 UTC Tests are running...
🔴 2024-05-14 15:32:47 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14660	14485	0	84	78	13

github-actions · 2024-05-14T13:01:13Z

⚪ 2024-05-14 13:01:13 UTC Pre-commit check for 805e3ad has started.
⚪ 2024-05-14 13:01:15 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-14 13:35:51 UTC Build successful.

github-actions · 2024-05-14T13:01:32Z

⚪ 2024-05-14 13:01:32 UTC Pre-commit check for 805e3ad has started.
⚪ 2024-05-14 13:01:34 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-14 13:38:46 UTC Build successful.
⚪ 2024-05-14 13:44:29 UTC Tests are running...
🔴 2024-05-14 15:35:36 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
73510	60241	0	48	13212	9

github-actions · 2024-05-16T22:12:21Z

⚪ 2024-05-16 22:12:21 UTC Pre-commit check for cab911a has started.
⚪ 2024-05-16 22:12:24 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-16 22:49:20 UTC Build successful.

github-actions · 2024-05-16T22:12:32Z

⚪ 2024-05-16 22:12:32 UTC Pre-commit check for cab911a has started.
⚪ 2024-05-16 22:12:35 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-16 22:51:21 UTC Build successful.
⚪ 2024-05-16 22:53:22 UTC Tests are running...
🔴 2024-05-17 00:46:02 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
73861	61103	0	54	12692	12

github-actions · 2024-05-16T22:12:37Z

⚪ 2024-05-16 22:12:36 UTC Pre-commit check for cab911a has started.
⚪ 2024-05-16 22:12:39 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-16 22:54:10 UTC Build successful.
⚪ 2024-05-16 22:56:07 UTC Tests are running...
🔴 2024-05-17 00:55:26 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15014	14696	0	103	200	15

github-actions · 2024-05-22T13:49:20Z

⚪ 2024-05-22 13:49:20 UTC Pre-commit check for e85ed6b has started.
⚪ 2024-05-22 13:49:22 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-22 14:33:07 UTC Build successful.
⚪ 2024-05-22 14:34:53 UTC Tests are running...
🔴 2024-05-22 16:38:30 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15053	14919	0	62	55	17

github-actions · 2024-05-22T13:51:53Z

⚪ 2024-05-22 13:51:50 UTC Pre-commit check for e85ed6b has started.
⚪ 2024-05-22 13:51:56 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-22 14:32:08 UTC Build successful.

github-actions · 2024-05-22T13:52:05Z

⚪ 2024-05-22 13:52:04 UTC Pre-commit check for e85ed6b has started.
⚪ 2024-05-22 13:52:07 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-22 14:30:41 UTC Build successful.
⚪ 2024-05-22 14:32:34 UTC Tests are running...
🔴 2024-05-22 16:29:07 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
74050	61327	0	20	12696	7

github-actions · 2024-05-23T13:02:37Z

⚪ 2024-05-23 13:02:36 UTC Pre-commit check for 9b4e9da has started.
⚪ 2024-05-23 13:05:30 UTC Build linux-x86_64-release-clang14 is running...
⚫ 2024-05-23 13:07:34 UTC Check cancelled

github-actions · 2024-05-23T13:04:37Z

⚪ 2024-05-23 13:04:37 UTC Pre-commit check for 9b4e9da has started.
⚫ 2024-05-23 13:07:21 UTC Check cancelled

github-actions · 2024-05-23T13:06:20Z

⚪ 2024-05-23 13:06:19 UTC Pre-commit check for 9b4e9da has started.
⚫ 2024-05-23 13:07:21 UTC Check cancelled

github-actions · 2024-05-23T13:08:46Z

⚪ 2024-05-23 13:08:46 UTC Pre-commit check for d79fd16 has started.
⚫ 2024-05-23 13:09:10 UTC Check cancelled

github-actions · 2024-05-23T13:08:52Z

⚪ 2024-05-23 13:08:52 UTC Pre-commit check for d79fd16 has started.
⚫ 2024-05-23 13:09:10 UTC Check cancelled

github-actions · 2024-05-23T13:10:20Z

⚪ 2024-05-23 13:10:19 UTC Pre-commit check for 22f51fb has started.
⚪ 2024-05-23 13:13:09 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-23 13:51:40 UTC Build successful.

github-actions · 2024-05-23T13:12:41Z

⚪ 2024-05-23 13:12:41 UTC Pre-commit check for 22f51fb has started.
⚪ 2024-05-23 13:15:35 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-23 13:54:41 UTC Build successful.
⚪ 2024-05-23 13:54:56 UTC Tests are running...
🔴 2024-05-23 15:04:21 UTC Test run completed, no test results found for commit 774abac. Please check build logs.
⚫ 2024-05-23 15:04:38 UTC Check cancelled

github-actions · 2024-05-23T13:12:42Z

⚪ 2024-05-23 13:12:42 UTC Pre-commit check for 22f51fb has started.
⚪ 2024-05-23 13:15:35 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-23 13:55:13 UTC Build successful.
⚪ 2024-05-23 13:55:25 UTC Tests are running...
🔴 2024-05-23 15:04:24 UTC Test run completed, no test results found for commit 774abac. Please check build logs.
⚫ 2024-05-23 15:04:44 UTC Check cancelled

github-actions · 2024-05-23T15:06:38Z

⚪ 2024-05-23 15:06:38 UTC Pre-commit check for 73832fd has started.
⚪ 2024-05-23 15:09:37 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-23 15:38:08 UTC Build successful.
⚪ 2024-05-23 15:38:22 UTC Tests are running...
🔴 2024-05-23 17:42:18 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15195	15070	0	56	58	11

github-actions · 2024-05-23T15:06:39Z

⚪ 2024-05-23 15:06:38 UTC Pre-commit check for 73832fd has started.
⚪ 2024-05-23 15:09:22 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-23 15:35:07 UTC Build successful.

github-actions · 2024-05-23T15:08:18Z

⚪ 2024-05-23 15:08:17 UTC Pre-commit check for 73832fd has started.
⚪ 2024-05-23 15:11:08 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-23 15:37:48 UTC Build successful.
⚪ 2024-05-23 15:37:57 UTC Tests are running...
🔴 2024-05-23 17:35:20 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
74235	61521	0	5	12700	9

github-actions · 2024-05-23T22:59:23Z

⚪ 2024-05-23 22:59:23 UTC Pre-commit check for added5e has started.
⚪ 2024-05-23 23:02:21 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-23 23:40:34 UTC Build successful.

github-actions · 2024-05-26T21:39:05Z

⚪ 2024-05-26 21:39:04 UTC Pre-commit check for e32024a has started.
⚪ 2024-05-26 21:42:07 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-26 21:44:39 UTC Build successful.
⚪ 2024-05-26 21:44:51 UTC Tests are running...
🔴 2024-05-26 21:50:57 UTC Test run completed, no test results found for commit b55e463. Please check build logs.
⚫ 2024-05-26 21:51:48 UTC Check cancelled

github-actions · 2024-05-26T21:56:42Z

⚪ 2024-05-26 21:56:42 UTC Pre-commit check for 38cb630 has started.
⚪ 2024-05-26 21:59:40 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-26 22:01:44 UTC Build successful.

github-actions · 2024-05-26T21:56:50Z

⚪ 2024-05-26 21:56:50 UTC Pre-commit check for 38cb630 has started.
⚪ 2024-05-26 21:59:50 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-26 22:02:02 UTC Build successful.
⚪ 2024-05-26 22:02:16 UTC Tests are running...
🔴 2024-05-27 00:11:51 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15204	15078	0	54	61	11

github-actions · 2024-05-26T21:56:51Z

⚪ 2024-05-26 21:56:50 UTC Pre-commit check for 38cb630 has started.
⚪ 2024-05-26 21:59:53 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-26 22:02:07 UTC Build successful.
⚪ 2024-05-26 22:02:20 UTC Tests are running...
🔴 2024-05-27 00:01:49 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
74310	61596	0	4	12703	7

github-actions bot added the new-feature label May 14, 2024

nshestakov reviewed May 14, 2024

View reviewed changes

ydb/public/lib/ydb_cli/commands/ydb_service_topic.cpp Outdated Show resolved Hide resolved

niksaveliev added not-for-changelog and removed new-feature labels May 16, 2024

niksaveliev force-pushed the autoscaling-control-plane branch from 2e16b14 to 751ac1b Compare May 23, 2024 13:06

revert CLI changes

ca3174e

nshestakov approved these changes May 27, 2024

View reviewed changes

niksaveliev merged commit 0a916d0 into ydb-platform:main May 27, 2024

This was referenced May 27, 2024

Cli changes for autoscaling #4871

Closed

Fix getting mid range #4885

Merged

GrigoriyPA mentioned this pull request May 28, 2024

KqpRun fixed config settings passing, added monitoring #4921

Merged

niksaveliev mentioned this pull request May 29, 2024

Autoscaling logs #4976

Merged

This was referenced May 29, 2024

YQ-3292 fixed kqprun ResourceBrokerConfig passing #4994

Merged

Added ExecutionId for script execution logs #5014

Merged

niksaveliev mentioned this pull request May 30, 2024

Forbid autoscaling with size retention #5054

Merged

StekPerepolnen mentioned this pull request May 31, 2024

vdisk evict #5093

Merged

This was referenced Jun 6, 2024

Mute split by load ut #5264

Closed

Mute split by load ut #5267

Closed

Mute split by load ut #5272

Merged

Fix build #5310

Merged

serbel324 mentioned this pull request Jun 7, 2024

Don't mark slow disks if there is only one #5340

Merged

niksaveliev mentioned this pull request Jun 10, 2024

Fix split by load ut #5383

Merged

This was referenced Jun 11, 2024

metadata cache keeps merged issues #5461

Merged

json_hc metadata cache #5465

Merged

json tenantinfo status metadata cache #5543

Merged

json query transaction mode #5574

Merged

niksaveliev mentioned this pull request Jun 17, 2024

Fix win build #5622

Closed

This was referenced Jun 19, 2024

hc api doc #5703

Merged

move time difference issue under database level #5859

Merged

metadata requests flags fix #5860

Merged

StekPerepolnen mentioned this pull request Jun 28, 2024

oidc-proxy new iam 2 #6059

Merged

Autoscaling control plane #4486

Autoscaling control plane #4486

Uh oh!

Conversation

niksaveliev commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog entry

Changelog category

Additional information

Uh oh!

github-actions bot commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nshestakov May 14, 2024

Choose a reason for hiding this comment

Uh oh!

niksaveliev May 14, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

niksaveliev commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 22, 2024 •

edited

Loading

github-actions bot commented May 22, 2024 •

edited

Loading

github-actions bot commented May 22, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 26, 2024 •

edited

Loading

github-actions bot commented May 26, 2024 •

edited

Loading

github-actions bot commented May 26, 2024 •

edited

Loading

github-actions bot commented May 26, 2024 •

edited

Loading