Skip to content

Conversation

@jepett0
Copy link
Contributor

@jepett0 jepett0 commented Sep 19, 2024

We want YDB CLI to return view query text on ydb scheme describe view_to_describe command.

There is no generic GRPC service that can return SchemeShard's private protobuf info to YDB CLI. We have to choose from the following options:

  1. Create a dedicated GRPC service for views and add only the DescribeView call there. This approach seems to be the currently adopted one. See the recently added replication service and object storage service which both have a single RPC in them.
  2. Add the DescribeView call to the TableService. I'm not sure if adding the DescribeView call here is a good option since this service is concerned with tables only.
  3. Use the legacy GRPC service (a.k.a. Message Bus) to issue a generic SchemeDescribe request that returns a private SchemeShard's protobuf. It would differ significantly from what YDB CLI is currently doing to describe a scheme object, but it would provide a good solution for a generic describe scheme object problem. Whenever we decide that we are ready to remove the legacy TGrpcClient we would have to rewrite this code too. However, I don't think it would take too much time to rewrite it.

The current PR implements the third approach.

@github-actions
Copy link

github-actions bot commented Sep 19, 2024

2024-09-19 09:00:58 UTC Pre-commit check linux-x86_64-release-clang14 for d92e886 has started.
2024-09-19 09:01:24 UTC Artifacts will be uploaded here
2024-09-19 09:03:53 UTC ya make is running...
🟢 2024-09-19 09:07:30 UTC Build successful.

@github-actions
Copy link

github-actions bot commented Sep 19, 2024

2024-09-19 09:01:15 UTC Pre-commit check linux-x86_64-release-asan for d92e886 has started.
2024-09-19 09:01:25 UTC Artifacts will be uploaded here
2024-09-19 09:03:54 UTC ya make is running...
🟢 2024-09-19 09:13:46 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
636 633 0 0 2 1

🟢 2024-09-19 09:13:55 UTC Build successful.
🔴 2024-09-19 09:14:13 UTC ydbd size 5.6 GiB changed* by +3.6 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 5321b21 merge: d92e886 diff diff %
ydbd size 6 054 163 304 Bytes 6 057 897 376 Bytes +3.6 MiB +0.062%
ydbd stripped size 1 514 626 000 Bytes 1 515 324 272 Bytes +681.9 KiB +0.046%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Sep 19, 2024

2024-09-19 09:01:43 UTC Pre-commit check linux-x86_64-relwithdebinfo for d92e886 has started.
2024-09-19 09:01:53 UTC Artifacts will be uploaded here
2024-09-19 09:04:19 UTC ya make is running...
🟢 2024-09-19 09:12:08 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
651 649 0 0 2 0

🟢 2024-09-19 09:12:18 UTC Build successful.
🔴 2024-09-19 09:12:36 UTC ydbd size 8.4 GiB changed* by +2.1 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 5321b21 merge: d92e886 diff diff %
ydbd size 9 037 917 320 Bytes 9 040 157 432 Bytes +2.1 MiB +0.025%
ydbd stripped size 489 046 440 Bytes 489 048 776 Bytes +2.3 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

if (response.status() != NKikimr::NMsgBusProxy::MSTATUS_OK) {
throw yexception() << "scheme describe error status: " << response.status()
<< ", issues: " << NYql::IssuesFromMessageAsString(response.issues());
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't written unit tests yet, because I'm not sure this approach would be the accepted one, but if you are interested in the output of the ydb scheme describe command, here is how it looks like:

...$ ydb scheme describe v
<view> v

Query text: select * from t

...$ ydb scheme describe vvv
Status: SCHEME_ERROR
Issues: 
<main>: Error: Path not found

I'm not sure where the error message is generated. You can notice that its format is different from what I have written for the response.status() != MSTATUS_OK case.

However, error handling is covered 🤷

auto clientConfig = NYdbGrpc::TGRpcClientConfig(config.Address);
if (config.EnableSsl) {
clientConfig.EnableSsl = config.EnableSsl;
clientConfig.SslCredentials.pem_root_certs = config.CaCerts;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested:

  • no authentication
  • YDB token authentication

I imagine I would need to test all of the other methods to be sure that the GRPC client is configured correctly 😢

@jepett0 jepett0 requested a review from CyberROFL September 19, 2024 09:24
@jepett0 jepett0 marked this pull request as ready for review September 19, 2024 09:24
@jepett0 jepett0 requested a review from ijon September 19, 2024 09:24
@@ -1,5 +1,10 @@
#include "ydb_service_scheme.h"

#include <ydb/core/protos/flat_scheme_op.pb.h>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No way.

@blinkov
Copy link
Member

blinkov commented Sep 19, 2024

@jepett0, the legacy MessageBus API is in the middle of being removed from the codebase; you need to go with other options.

@jepett0
Copy link
Contributor Author

jepett0 commented Sep 19, 2024

Will implement using the approach 1. See PR

@jepett0 jepett0 closed this Sep 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants