Skip to content

Add column shard Controller parameters to AppData #8000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Aug 22, 2024

Conversation

swalrus1
Copy link
Collaborator

@swalrus1 swalrus1 commented Aug 19, 2024

  • add default values used by Controller in columnshard to AppData to allow configuring them on a real cluster
  • refactor CSController

Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 10:31:07 UTC Pre-commit check for 2c2cbd2 has started.
2024-08-19 10:35:21 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-19 10:57:24 UTC Build failed. see the logs.

YDB_ACCESSOR(std::optional<TDuration>, OptimizerFreshnessCheckDuration, TDuration::Zero());
YDB_ACCESSOR_DEF(std::optional<TDuration>, CompactionActualizationLag);
YDB_ACCESSOR_DEF(std::optional<TDuration>, TasksActualizationLag);
YDB_ACCESSOR_DEF(std::optional<TDuration>, OverrideRequestsTracePingCheckPeriod);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Переименовал поля в Override*, чтобы не было коллизии между Get* и виртуальными методами контроллера

ivanmorozov333
ivanmorozov333 previously approved these changes Aug 19, 2024
Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 10:37:34 UTC Pre-commit check for 2c2cbd2 has started.
2024-08-19 10:41:08 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-19 11:06:01 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 10:43:41 UTC Pre-commit check for 2c2cbd2 has started.
2024-08-19 10:47:50 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-19 11:12:37 UTC Build failed. see the logs.

@swalrus1 swalrus1 marked this pull request as ready for review August 19, 2024 10:45
Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 14:05:18 UTC Pre-commit check for 234ec92 has started.
2024-08-19 14:08:14 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-19 14:29:41 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 14:05:20 UTC Pre-commit check for 234ec92 has started.
2024-08-19 14:08:16 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-19 14:27:07 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 14:05:29 UTC Pre-commit check for 234ec92 has started.
2024-08-19 14:08:30 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-19 14:31:16 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 15:19:36 UTC Pre-commit check for a7c80a9 has started.
2024-08-19 15:22:34 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-19 17:16:12 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40435 35006 0 71 5313 45

2024-08-19 17:19:57 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-19 17:31:05 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
129 (only retried tests) 38 0 62 0 29

2024-08-19 17:31:14 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-19 17:42:05 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
111 (only retried tests) 21 0 61 0 29

🟢 2024-08-19 17:42:12 UTC Build successful.
🟡 2024-08-19 17:42:49 UTC ydbd size 8.1 GiB changed* by +429.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 9b567af merge: a7c80a9 diff diff %
ydbd size 8 743 150 728 Bytes 8 743 590 696 Bytes +429.7 KiB +0.005%
ydbd stripped size 475 702 312 Bytes 475 721 928 Bytes +19.2 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 15:19:39 UTC Pre-commit check for a7c80a9 has started.
2024-08-19 15:22:50 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-19 18:04:59 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11538 11401 0 78 23 36

🟢 2024-08-19 18:06:00 UTC Build successful.
🟡 2024-08-19 18:06:30 UTC ydbd size 5.5 GiB changed* by +233.2 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 9b567af merge: a7c80a9 diff diff %
ydbd size 5 883 563 800 Bytes 5 883 802 568 Bytes +233.2 KiB +0.004%
ydbd stripped size 1 477 973 904 Bytes 1 478 026 480 Bytes +51.3 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 19, 2024

2024-08-19 15:20:03 UTC Pre-commit check for a7c80a9 has started.
2024-08-19 15:23:11 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-19 16:01:03 UTC Build successful.

Copy link

github-actions bot commented Aug 20, 2024

2024-08-20 08:03:36 UTC Pre-commit check for 4f508af has started.
2024-08-20 08:06:37 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-20 08:37:44 UTC Build successful.

Copy link

github-actions bot commented Aug 20, 2024

2024-08-20 08:03:59 UTC Pre-commit check for 4f508af has started.
2024-08-20 08:07:14 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-20 10:00:38 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40420 35060 0 12 5315 33

2024-08-20 10:04:05 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-20 10:14:14 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
61 (only retried tests) 26 0 7 1 27

2024-08-20 10:14:23 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-20 10:22:06 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
49 (only retried tests) 16 0 6 0 27

🟢 2024-08-20 10:22:14 UTC Build successful.
🟡 2024-08-20 10:22:53 UTC ydbd size 8.1 GiB changed* by +421.5 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: b0abac7 merge: 4f508af diff diff %
ydbd size 8 747 173 072 Bytes 8 747 604 736 Bytes +421.5 KiB +0.005%
ydbd stripped size 475 745 960 Bytes 475 757 256 Bytes +11.0 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 20, 2024

2024-08-20 08:04:32 UTC Pre-commit check for 4f508af has started.
2024-08-20 08:07:38 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-20 10:04:36 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11334 11248 0 26 23 37

🟢 2024-08-20 10:05:49 UTC Build successful.
🟡 2024-08-20 10:06:21 UTC ydbd size 5.5 GiB changed* by +233.2 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: b0abac7 merge: 4f508af diff diff %
ydbd size 5 886 490 344 Bytes 5 886 729 112 Bytes +233.2 KiB +0.004%
ydbd stripped size 1 478 181 008 Bytes 1 478 233 584 Bytes +51.3 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 20, 2024

2024-08-20 14:23:21 UTC Pre-commit check for a3fa29d has started.
2024-08-20 14:28:48 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-20 17:09:48 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40423 35083 0 10 5314 16

2024-08-20 17:13:21 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟡 2024-08-20 17:20:18 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
30 (only retried tests) 16 0 6 0 8

2024-08-20 17:20:25 UTC Failed tests rerun (try 3) linux-x86_64-relwithdebinfo is running...
🔴 2024-08-20 17:28:17 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19 (only retried tests) 5 0 6 0 8

🟢 2024-08-20 17:28:24 UTC Build successful.
🟡 2024-08-20 17:29:00 UTC ydbd size 8.1 GiB changed* by +417.6 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 767ff64 merge: a3fa29d diff diff %
ydbd size 8 748 356 632 Bytes 8 748 784 248 Bytes +417.6 KiB +0.005%
ydbd stripped size 475 797 864 Bytes 475 805 128 Bytes +7.1 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 20, 2024

2024-08-20 14:23:23 UTC Pre-commit check for a3fa29d has started.
2024-08-20 14:29:03 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-20 16:28:54 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11353 11256 0 43 31 23

🟢 2024-08-20 16:29:53 UTC Build successful.
🟡 2024-08-20 16:30:29 UTC ydbd size 5.5 GiB changed* by +237.2 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 767ff64 merge: a3fa29d diff diff %
ydbd size 5 887 494 088 Bytes 5 887 736 960 Bytes +237.2 KiB +0.004%
ydbd stripped size 1 478 366 640 Bytes 1 478 423 312 Bytes +55.3 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 20, 2024

2024-08-20 14:29:36 UTC Pre-commit check for a3fa29d has started.
2024-08-20 14:32:33 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-20 14:51:30 UTC Build successful.

Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 07:42:49 UTC Pre-commit check for 79953d0 has started.
2024-08-21 07:45:50 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-21 08:23:42 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 07:43:28 UTC Pre-commit check for 79953d0 has started.
2024-08-21 07:47:20 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-21 08:25:11 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 07:44:45 UTC Pre-commit check for 79953d0 has started.
2024-08-21 07:47:45 UTC Check linux-x86_64-release-clang14 is running...
🔴 2024-08-21 08:28:01 UTC Build failed. see the logs.

Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 10:26:36 UTC Pre-commit check for c3417a5 has started.
2024-08-21 10:30:22 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-21 12:57:48 UTC Too many tests failed, NOT going to retry
🔴 2024-08-21 12:58:16 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11649 10750 0 584 269 46

🟢 2024-08-21 12:59:22 UTC Build successful.
🟡 2024-08-21 12:59:55 UTC ydbd size 5.5 GiB changed* by +194.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: ed392b6 merge: c3417a5 diff diff %
ydbd size 5 887 868 816 Bytes 5 888 068 296 Bytes +194.8 KiB +0.003%
ydbd stripped size 1 478 478 576 Bytes 1 478 509 072 Bytes +29.8 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 10:27:05 UTC Pre-commit check for c3417a5 has started.
2024-08-21 10:30:04 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-21 10:36:26 UTC Build successful.

Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 10:27:10 UTC Pre-commit check for c3417a5 has started.
2024-08-21 10:30:14 UTC Check linux-x86_64-relwithdebinfo is running...
🔴 2024-08-21 12:44:38 UTC Too many tests failed, NOT going to retry
🔴 2024-08-21 12:44:56 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40745 34553 0 571 5572 49

🟢 2024-08-21 12:48:32 UTC Build successful.
🟡 2024-08-21 12:49:07 UTC ydbd size 8.1 GiB changed* by +411.6 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: ed392b6 merge: c3417a5 diff diff %
ydbd size 8 748 583 272 Bytes 8 749 004 720 Bytes +411.6 KiB +0.005%
ydbd stripped size 475 832 936 Bytes 475 839 240 Bytes +6.2 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@swalrus1 swalrus1 force-pushed the add-control-vars-to-app-data branch from 2010107 to 34e2589 Compare August 21, 2024 14:56
Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 15:25:37 UTC Pre-commit check for e6cc6f2 has started.
2024-08-21 15:29:58 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-21 17:16:56 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11360 11283 0 30 31 16

🟢 2024-08-21 17:18:00 UTC Build successful.
🟡 2024-08-21 17:18:30 UTC ydbd size 5.5 GiB changed* by +231.4 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 8e44a97 merge: e6cc6f2 diff diff %
ydbd size 5 888 643 904 Bytes 5 888 880 848 Bytes +231.4 KiB +0.004%
ydbd stripped size 1 478 666 608 Bytes 1 478 719 376 Bytes +51.5 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 15:27:06 UTC Pre-commit check for e6cc6f2 has started.
2024-08-21 15:30:06 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-21 17:20:39 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40451 35108 0 10 5319 14

2024-08-21 17:24:12 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-21 17:32:40 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
25 (only retried tests) 15 0 0 1 9

🟢 2024-08-21 17:32:47 UTC Build successful.
🟡 2024-08-21 17:33:22 UTC ydbd size 8.1 GiB changed* by +426.1 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 8e44a97 merge: e6cc6f2 diff diff %
ydbd size 8 749 629 152 Bytes 8 750 065 440 Bytes +426.1 KiB +0.005%
ydbd stripped size 475 887 432 Bytes 475 897 576 Bytes +9.9 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 21, 2024

2024-08-21 15:30:05 UTC Pre-commit check for e6cc6f2 has started.
2024-08-21 15:37:36 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-21 16:13:55 UTC Build successful.

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 07:49:24 UTC Pre-commit check for 8a1c198 has started.
2024-08-22 07:49:48 UTC Check cancelled

@swalrus1 swalrus1 force-pushed the add-control-vars-to-app-data branch from 1bcb98b to 1bc67f8 Compare August 22, 2024 07:49
Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 07:51:19 UTC Pre-commit check for f3b3c05 has started.
2024-08-22 07:54:21 UTC Check linux-x86_64-release-clang14 is running...
🟢 2024-08-22 08:28:55 UTC Build successful.

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 07:51:31 UTC Pre-commit check for f3b3c05 has started.
2024-08-22 07:54:32 UTC Check linux-x86_64-release-asan is running...
🔴 2024-08-22 09:55:52 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11365 11280 0 27 33 25

🟢 2024-08-22 09:56:50 UTC Build successful.
🟡 2024-08-22 09:57:22 UTC ydbd size 5.5 GiB changed* by +449.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 892d691 merge: f3b3c05 diff diff %
ydbd size 5 888 764 656 Bytes 5 889 225 216 Bytes +449.8 KiB +0.008%
ydbd stripped size 1 478 697 328 Bytes 1 478 806 224 Bytes +106.3 KiB +0.007%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 22, 2024

2024-08-22 07:51:40 UTC Pre-commit check for f3b3c05 has started.
2024-08-22 07:54:39 UTC Check linux-x86_64-relwithdebinfo is running...
🟡 2024-08-22 09:45:23 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40459 35106 0 13 5320 20

2024-08-22 09:49:06 UTC Failed tests rerun (try 2) linux-x86_64-relwithdebinfo is running...
🟢 2024-08-22 09:57:44 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
33 (only retried tests) 24 0 0 1 8

🟢 2024-08-22 09:57:51 UTC Build successful.
🟡 2024-08-22 09:58:28 UTC ydbd size 8.1 GiB changed* by +452.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: f2788b5 merge: f3b3c05 diff diff %
ydbd size 8 750 016 472 Bytes 8 750 480 056 Bytes +452.7 KiB +0.005%
ydbd stripped size 475 913 224 Bytes 475 923 240 Bytes +9.8 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@swalrus1 swalrus1 merged commit 3e3ffd1 into ydb-platform:main Aug 22, 2024
10 of 12 checks passed
@swalrus1 swalrus1 deleted the add-control-vars-to-app-data branch August 22, 2024 10:58
rekby pushed a commit to rekby-forks/ydb that referenced this pull request Aug 28, 2024
stanislav-shchetinin pushed a commit to stanislav-shchetinin/ydb that referenced this pull request Aug 30, 2024
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 10, 2024
Conflicts:
	ydb/core/tx/columnshard/columnshard.cpp
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 11, 2024
Conflicts:
	ydb/core/tx/columnshard/columnshard.cpp
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 13, 2024
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 14, 2024
zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Sep 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants