Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve snapshots and add some tests #5144

Merged
merged 5 commits into from
Dec 29, 2022

Conversation

pengweisong
Copy link
Contributor

@pengweisong pengweisong commented Dec 28, 2022

What type of PR is this?

  • bug
  • feature
  • enhancement

What problem(s) does this PR solve?

Issue(s) number:

Description:

Fix Bug:

  1. check running job may mutate data before create snapshot.
  2. when create snapshot failed in some storage host, unlock the writing before return.

Add tests for Create/Drop/ListSnapshots

How do you solve it?

Special notes for your reviewer, ex. impact of this fix, design document, etc:

Checklist:

Tests:

  • Unit test(positive and negative cases)
  • Function test
  • Performance test
  • N/A

Affects:

  • Documentation affected (Please add the label if documentation needs to be modified.)
  • Incompatibility (If it breaks the compatibility, please describe it and add the label.)
  • If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).)
  • Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to be reflected in release notes and how to describe:

ex. Fixed the bug .....

Copy link
Contributor

@critical27 critical27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job.

BTW, I recall one detail: you'd better check the snapshot related timeout in AdminClient, there is an issue that

  • create snapshot with a big dataset, AdminClient will report timeout,
  • and AdminClient has retry default with 3 times, which will create 3 snapshot in storage
  • and user will get a failed (I don't remember precisely), but storage will create 3 snapshot

You could mock it in storage by adding a sleep

Copy link
Contributor

@critical27 critical27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job

@Sophie-Xie Sophie-Xie merged commit ffaa254 into vesoft-inc:master Dec 29, 2022
@@ -84,7 +84,7 @@ std::shared_ptr<ClientType> ThriftClientManager<ClientType>::client(const HostAd
}
});
auto clientChannel = apache::thrift::RocketClientChannel::newChannel(std::move(socket));
if (timeout > 0) {
if (timeout >= 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timeout is a uint.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is supposed to, I'll remove the condition then.

@czpmango
Copy link
Contributor

Please check this warning:

[  0%] Building CXX object src/clients/meta/CMakeFiles/meta_client_obj.dir/MetaClient.cpp.o
In file included from /home/kyle.cao/workspace/fork/nebula/src/common/thrift/ThriftClientManager.h:46,
                 from /home/kyle.cao/workspace/fork/nebula/src/clients/meta/MetaClient.h:27,
                 from /home/kyle.cao/workspace/fork/nebula/src/clients/meta/MetaClient.cpp:6:
/home/kyle.cao/workspace/fork/nebula/src/common/thrift/ThriftClientManager-inl.h: In instantiation of ‘std::shared_ptr<_Tp> nebula::thrift::ThriftClientManager<ClientType>::client(const nebula::HostAddr&, folly::EventBase*, bool, uint32_t) [with ClientType = nebula::meta::cpp2::MetaServiceAsyncClient; uint32_t = unsigned int]’:
/home/kyle.cao/workspace/fork/nebula/src/clients/meta/MetaClient.cpp:715:49:   required from ‘void nebula::meta::MetaClient::getResponse(Request, RemoteFunc, RespGenerator, folly::Promise<nebula::StatusOr<Response> >, bool, int32_t, int32_t) [with Request = nebula::meta::cpp2::AdminJobReq; RemoteFunc = nebula::meta::MetaClient::submitJob(nebula::GraphSpaceID, nebula::meta::cpp2::JobOp, nebula::meta::cpp2::JobType, std::vector<std::__cxx11::basic_string<char> >)::<lambda(auto:157, auto:158)>; RespGenerator = nebula::meta::MetaClient::submitJob(nebula::GraphSpaceID, nebula::meta::cpp2::JobOp, nebula::meta::cpp2::JobType, std::vector<std::__cxx11::basic_string<char> >)::<lambda(nebula::meta::cpp2::AdminJobResp&&)>; RpcResponse = nebula::meta::cpp2::AdminJobResp; Response = nebula::meta::cpp2::AdminJobResult; int32_t = int]’
/home/kyle.cao/workspace/fork/nebula/src/clients/meta/MetaClient.cpp:1216:14:   required from here
/home/kyle.cao/workspace/fork/nebula/src/common/thrift/ThriftClientManager-inl.h:87:15: error: comparison of unsigned expression in ‘>= 0’ is always true [-Werror=type-limits]
   87 |   if (timeout >= 0) {
      |       ~~~~~~~~^~~~
cc1plus: all warnings being treated as errors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants