Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: ScoreBasedBalancer may balance more segments to new online querynode #37652

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

weiliu1031
Copy link
Contributor

issue: #37651
this PR introduce two constriant to avoid balance too aggresive on new online querynode:

  1. sort the collections with total row count, balance the largest collection first
  2. add a data percent limit on the balance plan to single querynode, in case of move too much data to new node in one round.

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: weiliu1031
To complete the pull request process, please assign czs007 after the PR has been reviewed.
You can assign the PR to them by writing /assign @czs007 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot added the size/L Denotes a PR that changes 100-499 lines. label Nov 13, 2024
@mergify mergify bot added dco-passed DCO check passed. kind/bug Issues or changes related a bug labels Nov 13, 2024
Copy link

codecov bot commented Nov 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.07%. Comparing base (1304b40) to head (c95aaee).
Report is 231 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (1304b40) and HEAD (c95aaee). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (1304b40) HEAD (c95aaee)
2 1
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           master   #37652       +/-   ##
===========================================
- Coverage   80.57%   68.07%   -12.50%     
===========================================
  Files        1357      290     -1067     
  Lines      190448    25455   -164993     
===========================================
- Hits       153449    17328   -136121     
+ Misses      31588     8127    -23461     
+ Partials     5411        0     -5411     
Components Coverage Δ
Client ∅ <ø> (∅)
Core 68.07% <ø> (ø)
Go ∅ <ø> (∅)

see 1067 files with indirect coverage changes

Copy link
Contributor

mergify bot commented Nov 13, 2024

@weiliu1031 E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

…node

this PR introduce two constriant to avoid balance too aggresive on new
online querynode:

1. sort the collections with total row count, balance the largest
   collection first
2. add a data percent limit on the balance plan to single querynode,
   in case of move too much data to new node in one round.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
@mergify mergify bot added the ci-passed label Nov 14, 2024
@mergify mergify bot removed the ci-passed label Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dco-passed DCO check passed. kind/bug Issues or changes related a bug size/L Denotes a PR that changes 100-499 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants