[YQ-1997] Support for ReplaceIfExists flag in SchemeShard for External Data Source and External Table #1431

Alnen · 2024-01-30T15:54:04Z

Changelog entry

...

Changelog category

New feature

Additional information

...

github-actions · 2024-01-30T16:33:55Z

⚪ 2024-01-30 16:33:54 UTC Pre-commit check for 0459184 has started.
⚪ 2024-01-30 16:33:56 UTC Build linux-x86_64-release-asan is running...
🟢 2024-01-30 17:11:36 UTC Build successful.
⚪ 2024-01-30 17:11:45 UTC Tests are running...
🔴 2024-01-30 18:49:28 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
16033	15847	0	49	105	32

github-actions · 2024-01-30T16:34:10Z

⚪ 2024-01-30 16:34:09 UTC Pre-commit check for 0459184 has started.
⚪ 2024-01-30 16:34:11 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-01-30 17:09:48 UTC Build successful.
⚪ 2024-01-30 17:10:02 UTC Tests are running...
🔴 2024-01-30 18:40:46 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
60298	50929	0	30	9311	28

github-actions · 2024-01-31T09:14:48Z

⚪ 2024-01-31 09:14:48 UTC Pre-commit check for 16db29a has started.
⚪ 2024-01-31 09:14:49 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-01-31 09:54:05 UTC Build successful.
⚪ 2024-01-31 09:54:16 UTC Tests are running...
🔴 2024-01-31 11:08:27 UTC Test run completed, no test results found for commit 73b8d3c. Please check build logs.
⚫ 2024-01-31 11:08:31 UTC Check cancelled

github-actions · 2024-01-31T09:18:08Z

⚪ 2024-01-31 09:18:08 UTC Pre-commit check for 16db29a has started.
⚪ 2024-01-31 09:18:10 UTC Build linux-x86_64-release-asan is running...
🟢 2024-01-31 09:54:17 UTC Build successful.
⚪ 2024-01-31 09:54:29 UTC Tests are running...
🔴 2024-01-31 11:08:23 UTC Test run completed, no test results found for commit 73b8d3c. Please check build logs.
⚫ 2024-01-31 11:08:26 UTC Check cancelled

github-actions · 2024-01-31T11:09:09Z

⚪ 2024-01-31 11:09:08 UTC Pre-commit check for 6f62003 has started.
⚪ 2024-01-31 11:09:10 UTC Build linux-x86_64-release-asan is running...
⚫ 2024-01-31 11:12:28 UTC Check cancelled

github-actions · 2024-01-31T11:09:45Z

⚪ 2024-01-31 11:09:45 UTC Pre-commit check for 6f62003 has started.
⚪ 2024-01-31 11:09:47 UTC Build linux-x86_64-relwithdebinfo is running...
⚫ 2024-01-31 11:12:29 UTC Check cancelled

github-actions · 2024-01-31T11:12:57Z

⚪ 2024-01-31 11:12:57 UTC Pre-commit check for 30e65ce has started.
⚪ 2024-01-31 11:12:58 UTC Build linux-x86_64-release-asan is running...
⚫ 2024-01-31 11:44:43 UTC Check cancelled

github-actions · 2024-01-31T11:13:40Z

⚪ 2024-01-31 11:13:40 UTC Pre-commit check for 30e65ce has started.
⚪ 2024-01-31 11:13:41 UTC Build linux-x86_64-relwithdebinfo is running...
⚫ 2024-01-31 11:44:44 UTC Check cancelled

ydb/core/tx/schemeshard/schemeshard_path.cpp

ydb/core/tx/schemeshard/schemeshard__operation_create_external_table.cpp

ydb/core/tx/schemeshard/ut_external_data_source/ut_external_data_source.cpp

UgnineSirdis · 2024-01-31T10:22:44Z

ydb/core/tx/schemeshard/ut_external_data_source/ut_external_data_source.cpp

+            UNIT_ASSERT_VALUES_EQUAL(externalDataSourceDescription.GetVersion(), 2);
+            UNIT_ASSERT_VALUES_EQUAL(externalDataSourceDescription.GetLocation(), "https://s3.cloud.net/my_new_bucket");
+            UNIT_ASSERT_EQUAL(externalDataSourceDescription.GetAuth().identity_case(), NKikimrSchemeOp::TAuth::kNone);
+        }


We also need test that checks failure when object with the same name but another type already exists.

And also test that checks failure when we have external data source, but ReplaceIfExists flag is not set in command

UgnineSirdis · 2024-01-31T10:23:21Z

ydb/core/tx/schemeshard/ut_external_table/ut_external_table.cpp

+            UNIT_ASSERT_VALUES_EQUAL(columns.Get(1).GetName(), "value");
+            UNIT_ASSERT_VALUES_EQUAL(columns.Get(1).GetType(), "Uint64");
+            UNIT_ASSERT_VALUES_EQUAL(columns.Get(1).GetNotNull(), false);
+        }


We also need test that checks failure when object with the same name but another type already exists.

And also test that checks failure when we have external table, but ReplaceIfExists flag is not set in command

ydb/core/tx/schemeshard/ut_external_table/ut_external_table.cpp

ydb/core/tx/schemeshard/schemeshard__operation_replace_external_data_source.cpp

ydb/core/tx/schemeshard/schemeshard__operation_replace_external_table.cpp

Co-authored-by: Vasily Gerasimov <UgnineSirdis@ydb.tech>

github-actions · 2024-01-31T11:51:56Z

⚪ 2024-01-31 11:51:55 UTC Pre-commit check for f768c52 has started.
⚪ 2024-01-31 11:51:58 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-01-31 12:00:28 UTC Build failed. see the build logs.
⚪ 2024-01-31 12:01:23 UTC Tests are running...
🔴 2024-01-31 12:22:55 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
51462	41945	0	266	9239	12

github-actions · 2024-01-31T11:55:08Z

⚪ 2024-01-31 11:55:08 UTC Pre-commit check for f768c52 has started.
⚪ 2024-01-31 11:55:10 UTC Build linux-x86_64-release-asan is running...
🔴 2024-01-31 12:05:27 UTC Build failed. see the build logs.
⚪ 2024-01-31 12:05:37 UTC Tests are running...
🔴 2024-01-31 12:21:13 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
7507	7232	0	261	3	11

github-actions · 2024-01-31T12:48:57Z

⚪ 2024-01-31 12:48:57 UTC Pre-commit check for 9ae4db2 has started.
⚪ 2024-01-31 12:48:59 UTC Build linux-x86_64-relwithdebinfo is running...
⚫ 2024-01-31 12:53:55 UTC Check cancelled

github-actions · 2024-01-31T12:48:59Z

⚪ 2024-01-31 12:48:58 UTC Pre-commit check for 9ae4db2 has started.
⚪ 2024-01-31 12:49:01 UTC Build linux-x86_64-release-asan is running...
⚫ 2024-01-31 12:53:55 UTC Check cancelled

github-actions · 2024-01-31T13:16:26Z

⚪ 2024-01-31 13:16:26 UTC Pre-commit check for a708e5f has started.
⚪ 2024-01-31 13:16:29 UTC Build linux-x86_64-relwithdebinfo is running...
⚫ 2024-01-31 13:23:34 UTC Check cancelled

github-actions · 2024-01-31T13:18:56Z

⚪ 2024-01-31 13:18:55 UTC Pre-commit check for a708e5f has started.
⚪ 2024-01-31 13:18:57 UTC Build linux-x86_64-release-asan is running...
⚫ 2024-01-31 13:23:34 UTC Check cancelled

ydb/core/tx/schemeshard/schemeshard_impl.cpp

Initial implementation

3e756cc

github-actions bot added the new-feature label Jan 30, 2024

Cleaned code ydb-platform#1

84f575d

Tests ydb-platform#1

73b8d3c

Fixed PR comments ydb-platform#1

b2e29bf

Fixed PR comments ydb-platform#2

eff009d

UgnineSirdis requested changes Jan 31, 2024

View reviewed changes

ydb/core/tx/schemeshard/schemeshard__operation_replace_external_data_source.cpp Outdated Show resolved Hide resolved

ydb/core/tx/schemeshard/schemeshard__operation_replace_external_data_source.cpp Outdated Show resolved Hide resolved

UgnineSirdis requested changes Jan 31, 2024

View reviewed changes

Alnen and others added 3 commits January 31, 2024 14:44

Update ydb/core/tx/schemeshard/ut_external_table/ut_external_table.cpp

0d715a8

Co-authored-by: Vasily Gerasimov <UgnineSirdis@ydb.tech>

Update ydb/core/tx/schemeshard/ut_external_table/ut_external_table.cpp

37e2203

Co-authored-by: Vasily Gerasimov <UgnineSirdis@ydb.tech>

Fixed PR comments ydb-platform#3

249a4cc

Fixed PR comments ydb-platform#4

fc108f7

Alnen added 2 commits January 31, 2024 15:53

Fixed PR comments ydb-platform#5

02ad968

Merge branch 'main' into YQ-1997-ss-replace-support

fb9d578

ijon reviewed Jan 31, 2024

View reviewed changes

ydb/core/tx/schemeshard/schemeshard_impl.cpp Show resolved Hide resolved

vitstn mentioned this pull request Feb 7, 2024

YQL-17725 drop b and t suffixes #1660

Merged

GrigoriyPA mentioned this pull request Feb 7, 2024

KqpRun add results into clear execution #1661

Merged

This was referenced Feb 7, 2024

YQL-17725 tune doc #1667

Merged

Switch pg_parser BC tests on again #1709

Merged

unmute test #1711

Merged

ildar-khisambeev mentioned this pull request Feb 8, 2024

LOGBROKER-8783 bugfix #1713

Merged

vitstn mentioned this pull request Feb 8, 2024

Mark mutable_id for YT writes #1744

Merged

niksaveliev mentioned this pull request Feb 9, 2024

Fix pq writer and few renames #1757

Merged

GrigoriyPA mentioned this pull request Feb 9, 2024

Moved optimizers from yql/providers/dq to yql/dq #1758

Merged

pavelvelikhov mentioned this pull request Feb 9, 2024

Cannonized two plans #1773

Merged

niksaveliev mentioned this pull request Feb 12, 2024

Workload read without consumer and metrics fixes #1792

Merged

starlinskiy mentioned this pull request Feb 12, 2024

runtime dispatching #1800

Closed

vitstn mentioned this pull request Feb 12, 2024

quick fix for column name of select version() #1801

Merged

GrigoriyPA mentioned this pull request Feb 12, 2024

YQ 2786 fix test fq restarts #1810

Merged

vitstn mentioned this pull request Feb 12, 2024

Support of database sys cache #1817

Merged

starlinskiy mentioned this pull request Feb 12, 2024

runtime dispatching (fixed) #1847

Merged

pavelvelikhov mentioned this pull request Feb 13, 2024

Fixed a problem with simplified plan JSONs #1878

Merged

This was referenced Feb 13, 2024

Case insensivity for pg_catalog tables/cluster, columns. Support of pg_class:relam, and some columns in pg_database & pg_namespace #1893

Merged

fixed pg_get_userbyid function #1936

Merged

support of AnyNonArray type, fixes for array_agg #1958

Merged

pavelvelikhov mentioned this pull request Feb 15, 2024

Fixed a problem with plans in PG ut #1974

Merged

niksaveliev mentioned this pull request Feb 16, 2024

Fix kafka with enabled proxy #2017

Merged

vitstn mentioned this pull request Feb 16, 2024

mock of savepoints #2026

Merged

aakulaga-ydb mentioned this pull request Feb 24, 2024

Fix TzTimestamp wrong encoding (YQL-17920) #2222

Merged

GrigoriyPA mentioned this pull request Feb 26, 2024

YQ-2898 fix query failed with unknown uncommitted upload issue #2240

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[YQ-1997] Support for ReplaceIfExists flag in SchemeShard for External Data Source and External Table #1431

[YQ-1997] Support for ReplaceIfExists flag in SchemeShard for External Data Source and External Table #1431

Alnen commented Jan 30, 2024

github-actions bot commented Jan 30, 2024 •

edited

Loading

github-actions bot commented Jan 30, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

UgnineSirdis Jan 31, 2024

UgnineSirdis Jan 31, 2024

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

[YQ-1997] Support for ReplaceIfExists flag in SchemeShard for External Data Source and External Table #1431

[YQ-1997] Support for ReplaceIfExists flag in SchemeShard for External Data Source and External Table #1431

Conversation

Alnen commented Jan 30, 2024

Changelog entry

Changelog category

Additional information

github-actions bot commented Jan 30, 2024 • edited Loading

github-actions bot commented Jan 30, 2024 • edited Loading

github-actions bot commented Jan 31, 2024 • edited Loading

github-actions bot commented Jan 31, 2024 • edited Loading

github-actions bot commented Jan 31, 2024 • edited Loading

github-actions bot commented Jan 31, 2024 • edited Loading

github-actions bot commented Jan 31, 2024 • edited Loading

github-actions bot commented Jan 31, 2024 • edited Loading

UgnineSirdis Jan 31, 2024

Choose a reason for hiding this comment

UgnineSirdis Jan 31, 2024

Choose a reason for hiding this comment

github-actions bot commented Jan 31, 2024 • edited Loading

github-actions bot commented Jan 31, 2024 • edited Loading

github-actions bot commented Jan 31, 2024 • edited Loading

github-actions bot commented Jan 31, 2024 • edited Loading

github-actions bot commented Jan 31, 2024 • edited Loading

github-actions bot commented Jan 31, 2024 • edited Loading

github-actions bot commented Jan 30, 2024 •

edited

Loading

github-actions bot commented Jan 30, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading

github-actions bot commented Jan 31, 2024 •

edited

Loading