Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance StoragePerf #721

Merged
merged 7 commits into from
Aug 26, 2019
Merged

Enhance StoragePerf #721

merged 7 commits into from
Aug 26, 2019

Conversation

darionyaphet
Copy link
Contributor

@darionyaphet darionyaphet commented Aug 2, 2019

In this PR, support config data's size per message and multi message per request.

@dangleptr
Copy link
Contributor

dangleptr commented Aug 2, 2019

Not only value size. Currently, one thread could sent at most 1000 requests. We should break the limit. One task could sent N requests each time.

critical27
critical27 previously approved these changes Aug 6, 2019
@dangleptr dangleptr mentioned this pull request Aug 7, 2019
@sherman-the-tank
Copy link
Member

Can someone explain what's the purpose of this PR?

@darionyaphet darionyaphet changed the title Confinable properties's data size in StoragePerf Enhance StoragePerf Aug 11, 2019
@darionyaphet
Copy link
Contributor Author

In this PR I have support two features:

  • Support confinable properties's data size when using storage_perf to do stress test.
  • Support send multi message per request.

Maybe I should update it's title. @sherman-the-tank

CHECK_GT(interval, 0) << "qpsPerThread should not large than 1000, interval " << interval;
int32_t radix = qpsPerThread / 1000;
int32_t slotSize = sizeof(slots_) / sizeof(int32_t);
if (qpsPerThread >= 1000) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we don't need this if statement?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch & fixed :)

@@ -51,6 +82,7 @@ class Perf {
CHECK(mClient_->waitForMetadReady());
client_ = std::make_unique<StorageClient>(threadPool_, mClient_.get());
time::Duration duration;
static uint32_t interval = 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you mean 100ms here? According to my understanding, you split 1 second to 10 * 100ms, and set corresponding qps of each slot.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I got it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10 ms is a cycle. slots_ is the sending plan for a cycle. So it should sleep 1 ms.

@darionyaphet darionyaphet added the ready-for-testing PR: ready for the CI test label Aug 21, 2019
@nebula-community-bot
Copy link
Member

Unit testing passed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

1 similar comment
@nebula-community-bot
Copy link
Member

Unit testing passed.

Copy link
Contributor

@dangleptr dangleptr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent!

@@ -0,0 +1,28 @@
### Storage Perf Tool

`_build/storage_perf` is the performance tool to test the storage service.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done

@nebula-community-bot
Copy link
Member

Unit testing passed.

1 similar comment
@nebula-community-bot
Copy link
Member

Unit testing passed.

@dangleptr dangleptr merged commit 33534aa into vesoft-inc:master Aug 26, 2019
tong-hao pushed a commit to tong-hao/nebula that referenced this pull request Jun 1, 2021
* data's size per request and multi message per request

* append random message

* fix dangleptr's comment
liwenhui-soul pushed a commit to liwenhui-soul/nebula that referenced this pull request May 10, 2022
#### What type of PR is this?
- [ ] bug
- [x] feature
- [ ] enhancement

#### What does this PR do?


#### Which issue(s)/PR(s) this PR relates to?

close vesoft-inc#3448
  
#### Special notes for your reviewer, ex. impact of this fix, etc:

#### Additional context:


#### Checklist:
- [x] Documentation affected (Please add the label if documentation needs to be modified.)
- [ ] Incompatible (If it is incompatible, please describe it and add corresponding label.)
- [ ] Need to cherry-pick (If need to cherry-pick to some branches, please label the destination version(s).)
- [x] Performance impacted: Consumes more CPU/Memory


#### Release notes:
Please confirm whether to reflect in release notes and how to describe:
> Support segment auto-increasing id                                                     `


Migrated from vesoft-inc#3550

Co-authored-by: jakevin <30525741+jackwener@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants