-
Notifications
You must be signed in to change notification settings - Fork 5.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
external: set the period of checking merge iter hotspot according to the key size #48063
Conversation
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #48063 +/- ##
================================================
+ Coverage 71.6145% 72.7621% +1.1475%
================================================
Files 1401 1424 +23
Lines 405923 413847 +7924
================================================
+ Hits 290700 301124 +10424
+ Misses 95448 93774 -1674
+ Partials 19775 18949 -826
Flags with carried forward coverage won't be shown. Click here to find out more.
|
} | ||
// We check the hotspot when the elements size is almost the same as the concurrent reader buffer size. | ||
// So that we don't drop too many bytes if the hotspot shifts to other files. | ||
if sampleKeySize == 0 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's a bit safer to check sampleKeySize/sampleKeyCnt
is zero
/retest |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: lance6716, tangenta The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
In response to a cherrypick label: new pull request created to branch |
What problem does this PR solve?
Issue Number: close #47923
Problem Summary:
If a hotspot is detected, the reader would change to concurrent mode and read 32M data by default.
If the hotspot shifts to other files, the previous reader will drop the unread byte to release memory. If dropping too many bytes, the whole performance would be very bad.
What is changed and how it works?
Set the period of checking merge iter hotspot according to the key size.
For example, if the average key size is 0.1K (by sampling), then we can check the hotspot every
(32M / 0.1K) = 320000 KVs.
Check List
Tests
before:
Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.