-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[improve] [broker] improve the restart broker can load bundle as one of the best brokers when selectBroker in LeastResourceUsageWithWeight #16766
Merged
gaozhangmin
merged 1 commit into
apache:master
from
Nicklee007:improve-historyusage-clean-for-restart-broker
Dec 9, 2022
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Nicklee007
force-pushed
the
improve-historyusage-clean-for-restart-broker
branch
from
August 2, 2022 03:12
e386176
to
8cf80d6
Compare
Nicklee007
force-pushed
the
improve-historyusage-clean-for-restart-broker
branch
4 times, most recently
from
August 26, 2022 08:32
bd24ac3
to
3a4ae32
Compare
Nicklee007
force-pushed
the
improve-historyusage-clean-for-restart-broker
branch
4 times, most recently
from
September 5, 2022 08:24
b5f8001
to
cb48cba
Compare
Nicklee007
force-pushed
the
improve-historyusage-clean-for-restart-broker
branch
from
September 21, 2022 09:16
cb48cba
to
c83b387
Compare
The pr had no activity for 30 days, mark with Stale label. |
Nicklee007
force-pushed
the
improve-historyusage-clean-for-restart-broker
branch
from
December 7, 2022 12:59
c83b387
to
3c0c603
Compare
/pulsarbot run-failure-checks |
HQebupt
approved these changes
Dec 8, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
gaozhangmin
approved these changes
Dec 8, 2022
Codecov Report
@@ Coverage Diff @@
## master #16766 +/- ##
============================================
+ Coverage 44.79% 45.20% +0.41%
- Complexity 10026 10817 +791
============================================
Files 703 763 +60
Lines 68808 73575 +4767
Branches 7375 7912 +537
============================================
+ Hits 30822 33260 +2438
- Misses 34378 36592 +2214
- Partials 3608 3723 +115
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Demogorgon314
pushed a commit
to Demogorgon314/pulsar
that referenced
this pull request
Dec 26, 2022
…of the best brokers when selectBroker in LeastResourceUsageWithWeight (apache#16766) Co-authored-by: nicklixinyang <nicklixinyang@didiglobal.com>
Demogorgon314
pushed a commit
to Demogorgon314/pulsar
that referenced
this pull request
Dec 29, 2022
…of the best brokers when selectBroker in LeastResourceUsageWithWeight (apache#16766) Co-authored-by: nicklixinyang <nicklixinyang@didiglobal.com>
lifepuzzlefun
pushed a commit
to lifepuzzlefun/pulsar
that referenced
this pull request
Jan 10, 2023
…of the best brokers when selectBroker in LeastResourceUsageWithWeight (apache#16766) Co-authored-by: nicklixinyang <nicklixinyang@didiglobal.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
As the
brokerAvgResourceUsageWithWeight
stored the broker's history resource usage, if a broker is high load before restart, after the broker restart always show a high load history resource usage, andloadBalancerHistoryResourcePercentage = 0.9
, which will cause the first few times the broker can not be select as best broker to load bundles. Only whenselectBrokerForAssignment
invoke few time's , thebrokerAvgResourceUsageWithWeight
could be neutralized by the current resourceUsage.But bundles unload is not a frequently operator if not disable
doLoadShedding
,it will wait a long time; So if a broker restarted we can clean the brokers history resource usage to guaranty the no bundles or no msgRate broker can be select as the one of best brokers.pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LeastResourceUsageWithWeight.java
Lines 99 to 100 in bc94643
Modifications
Documentation
doc-not-needed
(Please explain why)
Matching PR in forked repository
PR in forked repository:Nicklee007#3