YQ-4084 Add available external data sources #15192

kardymonds · 2025-02-28T11:52:40Z

Changelog entry

...

Changelog category

Not for changelog (changelog entry is not required)

Description for reviewers

добавляется возможность включать внешние источники по отдельности (ObjectStorage / Clickhouse ...)
планируется на всех dedicated db включить только ObjectStorage
...

github-actions · 2025-02-28T11:53:50Z

🟢 2025-03-06 11:39:47 UTC The validation of the Pull Request description is successful.

github-actions · 2025-02-28T11:53:54Z

⚪ 2025-02-28 11:53:53 UTC Pre-commit check linux-x86_64-release-asan for 5ede95e has started.
⚪ 2025-02-28 11:54:41 UTC Artifacts will be uploaded here
⚪ 2025-02-28 11:57:56 UTC ya make is running...
🟡 2025-02-28 13:42:52 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13739	13561	0	109	32	37

⚪ 2025-02-28 13:44:14 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-02-28 13:59:55 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
244 (only retried tests)	208	0	4	0	32

⚪ 2025-02-28 14:00:08 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-02-28 14:11:41 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
66 (only retried tests)	36	0	0	0	30

🟢 2025-02-28 14:11:48 UTC Build successful.
🟢 2025-02-28 14:12:20 UTC ydbd size 3.7 GiB changed* by +38.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `6964fca`	merge: `5ede95e`	diff	diff %
ydbd size	3 986 751 664 Bytes	3 986 790 936 Bytes	+38.4 KiB	+0.001%
ydbd stripped size	1 388 130 952 Bytes	1 388 137 896 Bytes	+6.8 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-02-28T11:54:33Z

⚪ 2025-02-28 11:54:33 UTC Pre-commit check linux-x86_64-relwithdebinfo for 5ede95e has started.
⚪ 2025-02-28 11:54:47 UTC Artifacts will be uploaded here
⚪ 2025-02-28 11:58:07 UTC ya make is running...
🟡 2025-02-28 13:39:32 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
28153	25543	0	7	2466	137

⚪ 2025-02-28 13:42:12 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-02-28 13:58:16 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
211 (only retried tests)	86	0	1	0	124

⚪ 2025-02-28 13:58:27 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-02-28 14:09:13 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
186 (only retried tests)	61	0	1	0	124

🟢 2025-02-28 14:09:26 UTC Build successful.
🟢 2025-02-28 14:09:50 UTC ydbd size 2.1 GiB changed* by +20.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `6964fca`	merge: `5ede95e`	diff	diff %
ydbd size	2 287 439 912 Bytes	2 287 460 728 Bytes	+20.3 KiB	+0.001%
ydbd stripped size	479 541 344 Bytes	479 542 464 Bytes	+1.1 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

ydb/core/driver_lib/run/run.cpp

github-actions · 2025-03-03T08:57:49Z

⚪ 2025-03-03 08:57:48 UTC Pre-commit check linux-x86_64-release-asan for 6d550e2 has started.
⚪ 2025-03-03 08:58:04 UTC Artifacts will be uploaded here
⚪ 2025-03-03 09:01:10 UTC ya make is running...
🟡 2025-03-03 10:35:25 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13700	13527	0	119	19	35

⚪ 2025-03-03 10:36:37 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-03 10:49:25 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
216 (only retried tests)	182	0	3	1	30

⚪ 2025-03-03 10:49:36 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-03 11:00:47 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
63 (only retried tests)	36	0	0	1	26

🟢 2025-03-03 11:00:57 UTC Build successful.
🟢 2025-03-03 11:01:24 UTC ydbd size 3.7 GiB changed* by +53.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `b0bd8f9`	merge: `6d550e2`	diff	diff %
ydbd size	3 988 298 944 Bytes	3 988 353 584 Bytes	+53.4 KiB	+0.001%
ydbd stripped size	1 388 482 184 Bytes	1 388 496 680 Bytes	+14.2 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-03-03T08:57:59Z

⚪ 2025-03-03 08:57:58 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6d550e2 has started.
⚪ 2025-03-03 08:58:35 UTC Artifacts will be uploaded here
⚪ 2025-03-03 09:02:03 UTC ya make is running...
🟡 2025-03-03 10:30:00 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
28114	25509	0	1	2469	135

⚪ 2025-03-03 10:32:21 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-03 10:49:31 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
206 (only retried tests)	80	0	0	0	126

🟢 2025-03-03 10:49:40 UTC Build successful.
🟢 2025-03-03 10:49:58 UTC ydbd size 2.1 GiB changed* by +27.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `b0bd8f9`	merge: `6d550e2`	diff	diff %
ydbd size	2 288 336 672 Bytes	2 288 364 736 Bytes	+27.4 KiB	+0.001%
ydbd stripped size	479 642 688 Bytes	479 645 280 Bytes	+2.5 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

ydb/core/kqp/host/kqp_host.cpp

ydb/core/kqp/ut/scheme/kqp_scheme_ut.cpp

github-actions · 2025-03-04T06:27:24Z

⚪ 2025-03-04 06:27:23 UTC Pre-commit check linux-x86_64-relwithdebinfo for 7665081 has started.
⚪ 2025-03-04 06:27:37 UTC Artifacts will be uploaded here
⚪ 2025-03-04 06:30:45 UTC ya make is running...
🟡 2025-03-04 07:38:35 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
28124	25516	0	4	2464	140

⚪ 2025-03-04 07:41:33 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-04 07:52:29 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
215 (only retried tests)	90	0	0	0	125

🟢 2025-03-04 07:52:39 UTC Build successful.
🟢 2025-03-04 07:53:00 UTC ydbd size 2.1 GiB changed* by +29.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `3a5cfed`	merge: `7665081`	diff	diff %
ydbd size	2 289 031 112 Bytes	2 289 060 808 Bytes	+29.0 KiB	+0.001%
ydbd stripped size	479 730 560 Bytes	479 733 344 Bytes	+2.7 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-03-04T06:29:39Z

⚪ 2025-03-04 06:29:38 UTC Pre-commit check linux-x86_64-release-asan for 7665081 has started.
⚪ 2025-03-04 06:29:53 UTC Artifacts will be uploaded here
⚪ 2025-03-04 06:33:08 UTC ya make is running...
🟡 2025-03-04 08:05:31 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13705	13438	0	159	74	34

⚪ 2025-03-04 08:07:12 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-04 08:25:07 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
355 (only retried tests)	316	0	9	0	30

⚪ 2025-03-04 08:25:18 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-04 08:37:32 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
71 (only retried tests)	42	0	1	0	28

🟢 2025-03-04 08:37:42 UTC Build successful.
🟢 2025-03-04 08:38:17 UTC ydbd size 3.7 GiB changed* by +52.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `3a5cfed`	merge: `7665081`	diff	diff %
ydbd size	3 989 522 472 Bytes	3 989 576 088 Bytes	+52.4 KiB	+0.001%
ydbd stripped size	1 388 750 408 Bytes	1 388 761 896 Bytes	+11.2 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

gridnevvvit · 2025-03-04T10:36:17Z

ydb/core/kqp/ut/scheme/kqp_scheme_ut.cpp

+            );)";
+        result = session.ExecuteSchemeQuery(query2).GetValueSync();
+        UNIT_ASSERT_VALUES_EQUAL_C(result.GetStatus(), EStatus::SUCCESS, result.GetIssues().ToString());
+    }


please add unit test with query service.

Добавил в этот же тест

gridnevvvit · 2025-03-04T10:39:53Z

ydb/core/base/appdata_fwd.h

@@ -287,6 +287,8 @@ struct TAppData {
    // Tracing configurator (look for tracing config in ydb/core/jaeger_tracing/actors_tracing_control)
    TIntrusivePtr<NKikimr::NJaegerTracing::TSamplingThrottlingConfigurator> TracingConfigurator;

+    TSet<TString> AvailableExternalDataSources;


what is the specific reason to put this data into the appdata? maybe it's better to transfer it as argument of the actor?

github-actions · 2025-03-04T17:10:39Z

⚪ 2025-03-04 17:10:38 UTC Pre-commit check linux-x86_64-release-asan for 11d57cc has started.
⚪ 2025-03-04 17:10:54 UTC Artifacts will be uploaded here
⚪ 2025-03-04 17:14:07 UTC ya make is running...
🟡 2025-03-04 18:57:12 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13691	13536	0	91	24	40

⚪ 2025-03-04 18:58:31 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-04 19:12:41 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
206 (only retried tests)	164	0	6	2	34

⚪ 2025-03-04 19:12:54 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-04 19:27:09 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
70 (only retried tests)	36	0	0	1	33

🟢 2025-03-04 19:27:22 UTC Build successful.
🟢 2025-03-04 19:27:52 UTC ydbd size 3.7 GiB changed* by +60.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `b21a377`	merge: `11d57cc`	diff	diff %
ydbd size	3 990 845 880 Bytes	3 990 907 368 Bytes	+60.0 KiB	+0.002%
ydbd stripped size	1 388 991 368 Bytes	1 389 003 944 Bytes	+12.3 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-03-04T17:13:00Z

⚪ 2025-03-04 17:13:00 UTC Pre-commit check linux-x86_64-relwithdebinfo for 11d57cc has started.
⚪ 2025-03-04 17:13:32 UTC Artifacts will be uploaded here
⚪ 2025-03-04 17:16:39 UTC ya make is running...
🟡 2025-03-04 18:46:00 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
28096	25505	0	4	2469	118

⚪ 2025-03-04 18:48:29 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-04 19:01:58 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
178 (only retried tests)	70	0	0	0	108

🟢 2025-03-04 19:02:05 UTC Build successful.
🟢 2025-03-04 19:02:23 UTC ydbd size 2.1 GiB changed* by +30.8 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `b21a377`	merge: `11d57cc`	diff	diff %
ydbd size	2 290 069 208 Bytes	2 290 100 768 Bytes	+30.8 KiB	+0.001%
ydbd stripped size	479 945 248 Bytes	479 952 576 Bytes	+7.2 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

ydb/core/external_sources/external_source_factory.cpp

github-actions · 2025-03-05T05:19:04Z

⚪ 2025-03-05 05:19:04 UTC Pre-commit check linux-x86_64-relwithdebinfo for a6a65da has started.
⚪ 2025-03-05 05:19:18 UTC Artifacts will be uploaded here
⚪ 2025-03-05 05:22:25 UTC ya make is running...
🟡 2025-03-05 06:49:25 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
28096	25508	0	1	2465	122

⚪ 2025-03-05 06:51:44 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-05 07:02:56 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
173 (only retried tests)	67	0	0	0	106

🟢 2025-03-05 07:03:06 UTC Build successful.
🟢 2025-03-05 07:03:32 UTC ydbd size 2.1 GiB changed* by +49.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `fb4d6bb`	merge: `a6a65da`	diff	diff %
ydbd size	2 290 063 784 Bytes	2 290 114 840 Bytes	+49.9 KiB	+0.002%
ydbd stripped size	479 946 656 Bytes	479 955 552 Bytes	+8.7 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-03-05T05:19:13Z

⚪ 2025-03-05 05:19:12 UTC Pre-commit check linux-x86_64-release-asan for a6a65da has started.
⚪ 2025-03-05 05:19:28 UTC Artifacts will be uploaded here
⚪ 2025-03-05 05:22:41 UTC ya make is running...
🟡 2025-03-05 07:06:06 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13688	13548	0	89	15	36

⚪ 2025-03-05 07:07:16 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-05 07:21:32 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
198 (only retried tests)	154	0	7	3	34

⚪ 2025-03-05 07:21:44 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-05 07:33:48 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
74 (only retried tests)	39	0	0	2	33

🟢 2025-03-05 07:33:58 UTC Build successful.
🟢 2025-03-05 07:34:30 UTC ydbd size 3.7 GiB changed* by +60.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `b0f9318`	merge: `a6a65da`	diff	diff %
ydbd size	3 990 870 208 Bytes	3 990 931 752 Bytes	+60.1 KiB	+0.002%
ydbd stripped size	1 388 995 016 Bytes	1 389 007 592 Bytes	+12.3 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-03-10T07:51:09Z

⚪ 2025-03-10 07:51:08 UTC Pre-commit check linux-x86_64-release-asan for 301d28b has started.
⚪ 2025-03-10 07:51:23 UTC Artifacts will be uploaded here
⚪ 2025-03-10 07:54:38 UTC ya make is running...
🟡 2025-03-10 09:36:21 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13804	13577	0	176	16	35

⚪ 2025-03-10 09:37:32 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-10 09:51:44 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
328 (only retried tests)	221	0	74	3	30

⚪ 2025-03-10 09:51:56 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-10 10:05:26 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
184 (only retried tests)	99	0	54	1	30

🟢 2025-03-10 10:05:35 UTC Build successful.
🟢 2025-03-10 10:06:10 UTC ydbd size 3.7 GiB changed* by +76.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `59ab890`	merge: `301d28b`	diff	diff %
ydbd size	3 996 382 224 Bytes	3 996 460 488 Bytes	+76.4 KiB	+0.002%
ydbd stripped size	1 389 073 864 Bytes	1 389 087 976 Bytes	+13.8 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-03-10T13:41:13Z

⚪ 2025-03-10 13:41:12 UTC Pre-commit check linux-x86_64-release-asan for 7038884 has started.
⚪ 2025-03-10 13:41:27 UTC Artifacts will be uploaded here
⚪ 2025-03-10 13:44:31 UTC ya make is running...
🟡 2025-03-10 15:18:36 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13804	13639	0	121	10	34

⚪ 2025-03-10 15:20:10 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-10 15:34:41 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
258 (only retried tests)	163	0	54	8	33

⚪ 2025-03-10 15:34:53 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-10 15:48:30 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
175 (only retried tests)	107	0	28	7	33

🟢 2025-03-10 15:48:42 UTC Build successful.
🟢 2025-03-10 15:49:17 UTC ydbd size 3.7 GiB changed* by +82.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `c149b45`	merge: `7038884`	diff	diff %
ydbd size	3 994 884 584 Bytes	3 994 968 688 Bytes	+82.1 KiB	+0.002%
ydbd stripped size	1 388 591 688 Bytes	1 388 610 280 Bytes	+18.2 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-03-10T13:41:33Z

⚪ 2025-03-10 13:41:32 UTC Pre-commit check linux-x86_64-relwithdebinfo for 7038884 has started.
⚪ 2025-03-10 13:41:48 UTC Artifacts will be uploaded here
⚪ 2025-03-10 13:45:05 UTC ya make is running...
🟡 2025-03-10 15:21:11 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
28218	25624	0	8	2468	118

⚪ 2025-03-10 15:24:17 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-10 15:40:55 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
181 (only retried tests)	71	0	1	0	109

⚪ 2025-03-10 15:41:06 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-03-10 15:53:54 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
154 (only retried tests)	47	0	1	0	106

🟢 2025-03-10 15:54:03 UTC Build successful.
🟢 2025-03-10 15:54:26 UTC ydbd size 2.1 GiB changed* by +44.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `c149b45`	merge: `7038884`	diff	diff %
ydbd size	2 293 856 560 Bytes	2 293 902 376 Bytes	+44.7 KiB	+0.002%
ydbd stripped size	480 432 384 Bytes	480 444 896 Bytes	+12.2 KiB	+0.003%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

…nal-data-source

github-actions · 2025-03-11T06:43:25Z

⚪ 2025-03-11 06:43:24 UTC Pre-commit check linux-x86_64-release-asan for 70dd147 has started.
⚪ 2025-03-11 06:43:39 UTC Artifacts will be uploaded here
⚪ 2025-03-11 06:46:49 UTC ya make is running...
🟡 2025-03-11 08:25:14 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13805	13624	0	113	30	38

⚪ 2025-03-11 08:26:25 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-11 08:40:30 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
283 (only retried tests)	195	0	53	1	34

⚪ 2025-03-11 08:40:42 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-11 08:52:44 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
171 (only retried tests)	109	0	30	0	32

🟢 2025-03-11 08:52:53 UTC Build successful.
🟡 2025-03-11 08:53:22 UTC ydbd size 3.7 GiB changed* by +102.3 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `c805655`	merge: `70dd147`	diff	diff %
ydbd size	3 995 244 368 Bytes	3 995 349 096 Bytes	+102.3 KiB	+0.003%
ydbd stripped size	1 388 725 448 Bytes	1 388 748 360 Bytes	+22.4 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-03-11T06:43:25Z

⚪ 2025-03-11 06:43:25 UTC Pre-commit check linux-x86_64-relwithdebinfo for 70dd147 has started.
⚪ 2025-03-11 06:43:39 UTC Artifacts will be uploaded here
⚪ 2025-03-11 06:46:42 UTC ya make is running...
🟡 2025-03-11 08:12:15 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
28219	25633	0	1	2470	115

⚪ 2025-03-11 08:14:31 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-11 08:25:26 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
164 (only retried tests)	59	0	1	0	104

⚪ 2025-03-11 08:25:37 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-03-11 08:35:41 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
147 (only retried tests)	42	0	1	0	104

🟢 2025-03-11 08:35:51 UTC Build successful.
🟢 2025-03-11 08:36:12 UTC ydbd size 2.1 GiB changed* by +50.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `c805655`	merge: `70dd147`	diff	diff %
ydbd size	2 294 046 184 Bytes	2 294 097 392 Bytes	+50.0 KiB	+0.002%
ydbd stripped size	480 442 432 Bytes	480 454 656 Bytes	+11.9 KiB	+0.003%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-03-11T10:01:14Z

⚪ 2025-03-11 10:01:14 UTC Pre-commit check linux-x86_64-release-asan for f7ec41a has started.
⚪ 2025-03-11 10:01:29 UTC Artifacts will be uploaded here
⚪ 2025-03-11 10:04:40 UTC ya make is running...
⚫ 2025-03-11 11:22:51 UTC Check cancelled

github-actions · 2025-03-11T10:02:05Z

⚪ 2025-03-11 10:02:04 UTC Pre-commit check linux-x86_64-relwithdebinfo for f7ec41a has started.
⚪ 2025-03-11 10:02:19 UTC Artifacts will be uploaded here
⚪ 2025-03-11 10:05:32 UTC ya make is running...
⚫ 2025-03-11 11:22:49 UTC Check cancelled

GrigoriyPA · 2025-03-11T11:15:21Z

ydb/core/protos/config.proto

@@ -1077,6 +1077,7 @@ message TQueryServiceConfig {
    optional uint64 ProgressStatsPeriodMs = 14 [default = 0]; // 0 = disabled
    optional uint32 QueryTimeoutDefaultSeconds = 19 [default = 1800];
    optional bool EnableMatchRecognize = 20 [default = false];
+    repeated string AvailableExternalDataSources = 22; // List of external data source types; enabled all if empty


Надо исправить коммент, теперь disabled if empty

github-actions · 2025-03-11T11:25:09Z

⚪ 2025-03-11 11:25:09 UTC Pre-commit check linux-x86_64-relwithdebinfo for 19e624b has started.
⚪ 2025-03-11 11:25:25 UTC Artifacts will be uploaded here
⚪ 2025-03-11 11:28:42 UTC ya make is running...
🟢 2025-03-11 13:02:04 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
28251	25667	0	0	2468	116

🟢 2025-03-11 13:05:10 UTC Build successful.
🟢 2025-03-11 13:05:28 UTC ydbd size 2.1 GiB changed* by +46.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `7959b23`	merge: `19e624b`	diff	diff %
ydbd size	2 296 127 960 Bytes	2 296 175 192 Bytes	+46.1 KiB	+0.002%
ydbd stripped size	480 730 336 Bytes	480 738 592 Bytes	+8.1 KiB	+0.002%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-03-11T11:25:38Z

⚪ 2025-03-11 11:25:38 UTC Pre-commit check linux-x86_64-release-asan for 19e624b has started.
⚪ 2025-03-11 11:25:42 UTC Artifacts will be uploaded here
⚪ 2025-03-11 11:28:51 UTC ya make is running...
🟡 2025-03-11 13:08:35 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13835	13659	0	124	17	35

⚪ 2025-03-11 13:09:53 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-11 13:31:42 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
274 (only retried tests)	185	0	54	1	34

⚪ 2025-03-11 13:31:57 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-11 13:45:18 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
166 (only retried tests)	89	0	44	0	33

🟢 2025-03-11 13:45:28 UTC Build successful.
🟢 2025-03-11 13:45:59 UTC ydbd size 3.7 GiB changed* by +94.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `7959b23`	merge: `19e624b`	diff	diff %
ydbd size	3 998 881 888 Bytes	3 998 978 328 Bytes	+94.2 KiB	+0.002%
ydbd stripped size	1 389 594 216 Bytes	1 389 608 872 Bytes	+14.3 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

kardymonds · 2025-03-11T18:09:01Z

По опыту делать пустой список по-умолчанию как "всё разрешено" очень больно стрельнёт в будущем, когда окажется, что хотят выключить какой-то один, а для этого нужно перечислить все кроме этого одного, и всё это плохо мержится в конфигах. В grpc в итоге пришли к двум спискам: enable (чтобы включать то что выключено по-умолчанию) и disable (чтобы явно выключать).

Убрал условие что пустой список обозначает что все разрешено. Теперь нужно явно указывать все разрешенные.

check external data sources

c47ec48

kardymonds requested a review from a team as a code owner February 28, 2025 11:52

github-actions bot added the not-for-changelog label Feb 28, 2025

vitalyisaev2 requested changes Feb 28, 2025

View reviewed changes

ydb/core/driver_lib/run/run.cpp Outdated Show resolved Hide resolved

check sources type

46a4685

GrigoriyPA reviewed Mar 3, 2025

View reviewed changes

ydb/core/kqp/host/kqp_host.cpp Outdated Show resolved Hide resolved

ydb/core/kqp/ut/scheme/kqp_scheme_ut.cpp Show resolved Hide resolved

move checks to fqsetup

ef18d5b

vitalyisaev2 previously approved these changes Mar 4, 2025

View reviewed changes

GrigoriyPA previously approved these changes Mar 4, 2025

View reviewed changes

gridnevvvit reviewed Mar 4, 2025

View reviewed changes

move checks to factory

f185660

kardymonds dismissed stale reviews from GrigoriyPA and vitalyisaev2 via f185660 March 4, 2025 17:04

kardymonds requested a review from a team as a code owner March 4, 2025 17:04

uzhastik reviewed Mar 4, 2025

View reviewed changes

ydb/core/external_sources/external_source_factory.cpp Outdated Show resolved Hide resolved

ydb/core/external_sources/external_source_factory.cpp Outdated Show resolved Hide resolved

add const

0e28479

GrigoriyPA previously approved these changes Mar 5, 2025

View reviewed changes

gridnevvvit previously approved these changes Mar 5, 2025

View reviewed changes

try to fix tests

80da0da

kardymonds added 3 commits March 11, 2025 06:32

mv database type to header

da3516c

remove appconfig

7cbae36

Merge remote-tracking branch 'upstream/main' into YQ-4084-check-exter…

c66906f

…nal-data-source

rm space

576c45b

GrigoriyPA reviewed Mar 11, 2025

View reviewed changes

remove comment / fix kqprun config

8eeb376

GrigoriyPA approved these changes Mar 11, 2025

View reviewed changes

gridnevvvit approved these changes Mar 11, 2025

View reviewed changes

ijon approved these changes Mar 11, 2025

View reviewed changes

kardymonds merged commit 3de4a6c into ydb-platform:main Mar 11, 2025
12 checks passed

kardymonds added a commit to kardymonds/ydb that referenced this pull request Mar 11, 2025

YQ-4084 Add available external data sources (ydb-platform#15192)

4f5d350

kardymonds added a commit to kardymonds/ydb that referenced this pull request Mar 11, 2025

YQ-4084 Add available external data sources (ydb-platform#15192)

7a422f5

serbel324 mentioned this pull request Mar 13, 2025

Add compaction selected strategy counters #15689

Merged

alexnick88 mentioned this pull request Mar 19, 2025

provide database in schemecache request #15938

Closed

adameat pushed a commit to adameat/ydb that referenced this pull request Mar 20, 2025

YQ-4084 Add available external data sources (ydb-platform#15192)

e147d5d

blinkov pushed a commit that referenced this pull request Mar 21, 2025

YQ-4084 Add available external data sources (#15192)

abd7c1a

lberserq pushed a commit to lberserq/ydb that referenced this pull request Mar 28, 2025

YQ-4084 Add available external data sources (ydb-platform#15192)

e3b7ffe

kardymonds added a commit to kardymonds/ydb that referenced this pull request Apr 21, 2025

YQ-4084 Add available external data sources (ydb-platform#15192)

c569d7b

kardymonds added a commit to kardymonds/ydb that referenced this pull request May 5, 2025

YQ-4084 Add available external data sources (ydb-platform#15192)

1d8e8c4

YQ-4084 Add available external data sources #15192

YQ-4084 Add available external data sources #15192

Uh oh!

Conversation

kardymonds commented Feb 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog entry

Changelog category

Description for reviewers

Uh oh!

github-actions bot commented Feb 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gridnevvvit Mar 4, 2025

Choose a reason for hiding this comment

Uh oh!

kardymonds Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

gridnevvvit Mar 4, 2025

Choose a reason for hiding this comment

Uh oh!

kardymonds Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

kardymonds commented Feb 28, 2025 •

edited

Loading

github-actions bot commented Feb 28, 2025 •

edited

Loading

github-actions bot commented Feb 28, 2025 •

edited

Loading

github-actions bot commented Feb 28, 2025 •

edited

Loading

github-actions bot commented Mar 3, 2025 •

edited

Loading

github-actions bot commented Mar 3, 2025 •

edited

Loading

github-actions bot commented Mar 4, 2025 •

edited

Loading

github-actions bot commented Mar 4, 2025 •

edited

Loading

github-actions bot commented Mar 4, 2025 •

edited

Loading

github-actions bot commented Mar 4, 2025 •

edited

Loading

github-actions bot commented Mar 5, 2025 •

edited

Loading

github-actions bot commented Mar 5, 2025 •

edited

Loading

github-actions bot commented Mar 10, 2025 •

edited

Loading

github-actions bot commented Mar 10, 2025 •

edited

Loading

github-actions bot commented Mar 10, 2025 •

edited

Loading

github-actions bot commented Mar 11, 2025 •

edited

Loading

github-actions bot commented Mar 11, 2025 •

edited

Loading

github-actions bot commented Mar 11, 2025 •

edited

Loading

github-actions bot commented Mar 11, 2025 •

edited

Loading

github-actions bot commented Mar 11, 2025 •

edited

Loading

github-actions bot commented Mar 11, 2025 •

edited

Loading