scheduler: balance-hot-region-scheduler supports TiFlash hot-write #3900

HunDunDM · 2021-07-22T06:25:31Z

Signed-off-by: HunDunDM hundundm@gmail.com

related #3671

What problem does this PR solve?

What is changed and how it works?

scheduler: balance-hot-region-scheduler supports TiFlash hot-write

Check List

Tests

Unit test

Code changes

Has configuration change

Side effects

Related changes

PR to update pingcap/docs/pingcap/docs-cn: docs-cn#6835

Release note

Support hot-region balance scheduling for TiFlash

ti-chi-bot · 2021-07-22T06:25:32Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

Yisaer
lhy1024

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

codecov · 2021-07-27T15:04:29Z

Codecov Report

Merging #3900 (e314dac) into master (21fef03) will increase coverage by 0.03%.
The diff coverage is 93.65%.

@@            Coverage Diff             @@
##           master    #3900      +/-   ##
==========================================
+ Coverage   74.55%   74.58%   +0.03%     
==========================================
  Files         247      247              
  Lines       25276    25372      +96     
==========================================
+ Hits        18844    18924      +80     
- Misses       4755     4768      +13     
- Partials     1677     1680       +3

Flag	Coverage Δ
unittests	`74.58% <93.65%> (+0.03%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
server/config/config.go	`74.84% <ø> (ø)`
server/core/store.go	`80.00% <ø> (ø)`
server/schedulers/hot_region.go	`83.16% <84.88%> (-2.38%)`	⬇️
pkg/mock/mockcluster/mockcluster.go	`95.85% <95.23%> (+0.13%)`	⬆️
server/schedulers/utils.go	`94.96% <98.40%> (+2.07%)`	⬆️
server/config/persist_options.go	`91.53% <100.00%> (+0.03%)`	⬆️
server/schedule/filter/filters.go	`90.69% <100.00%> (ø)`
server/schedule/region_scatterer.go	`85.08% <100.00%> (ø)`
server/schedulers/hot_region_config.go	`86.71% <100.00%> (+0.76%)`	⬆️
server/schedulers/shuffle_hot_region.go	`65.65% <100.00%> (+0.35%)`	⬆️
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 21fef03...e314dac. Read the comment docs.

Signed-off-by: HunDunDM <hundundm@gmail.com>

pkg/mock/mockcluster/mockcluster.go

…hot-write-sche # Conflicts: # server/core/store.go

Signed-off-by: HunDunDM <hundundm@gmail.com>

server/schedulers/hot_region_config.go

server/schedulers/hot_region.go

Signed-off-by: HunDunDM <hundundm@gmail.com>

nolouch

lgtm

nolouch · 2021-08-07T07:39:54Z

/merge

ti-chi-bot · 2021-08-07T07:39:55Z

@nolouch: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2021-08-07T07:39:56Z

This pull request has been accepted and is ready to merge.

Commit hash: d69523c

Signed-off-by: HunDunDM <hundundm@gmail.com>

ti-chi-bot requested review from disksing and Yisaer July 22, 2021 06:25

HunDunDM force-pushed the tiflash-hot-write-sche branch from 2bc3c1a to 0f8a97e Compare July 26, 2021 11:37

ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 26, 2021

HunDunDM force-pushed the tiflash-hot-write-sche branch from c82c23f to 6a7e679 Compare July 27, 2021 13:47

HunDunDM added a commit to HunDunDM/pd that referenced this pull request Jul 28, 2021

scheduler: balance-hot-region-scheduler supports TiFlash (tikv#3900)

888581c

HunDunDM added a commit to HunDunDM/pd that referenced this pull request Jul 28, 2021

scheduler: balance-hot-region-scheduler supports TiFlash (tikv#3900)

27f1f98

Signed-off-by: HunDunDM <hundundm@gmail.com>

HunDunDM force-pushed the tiflash-hot-write-sche branch from 71cc0cc to 1a4fd74 Compare July 28, 2021 12:20

ti-chi-bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jul 30, 2021

HunDunDM force-pushed the tiflash-hot-write-sche branch from d226c5a to ef48dc9 Compare August 3, 2021 10:21

HunDunDM marked this pull request as ready for review August 3, 2021 10:25

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 3, 2021

HunDunDM added component/scheduler Scheduler logic. priority/P1 The issue has P1 priority. labels Aug 3, 2021

HunDunDM marked this pull request as draft August 3, 2021 13:12

ti-chi-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Aug 3, 2021

HunDunDM force-pushed the tiflash-hot-write-sche branch from ef48dc9 to 97b4ee1 Compare August 3, 2021 14:40

ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 3, 2021

HunDunDM force-pushed the tiflash-hot-write-sche branch 2 times, most recently from 9c6df82 to 105cef3 Compare August 4, 2021 10:18

HunDunDM marked this pull request as ready for review August 4, 2021 10:23

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 4, 2021

HunDunDM force-pushed the tiflash-hot-write-sche branch from 105cef3 to 9c9191a Compare August 4, 2021 12:59

HunDunDM added 4 commits August 6, 2021 16:41

address comment

91a0b5e

Signed-off-by: HunDunDM <hundundm@gmail.com>

fix typo

51edb56

Signed-off-by: HunDunDM <hundundm@gmail.com>

mockcluster supports tiflash learner

d1b47a1

Signed-off-by: HunDunDM <hundundm@gmail.com>

add unit test

f945cc2

Signed-off-by: HunDunDM <hundundm@gmail.com>

HunDunDM force-pushed the tiflash-hot-write-sche branch from 89a9d1d to f945cc2 Compare August 6, 2021 13:16

ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 6, 2021

lhy1024 reviewed Aug 6, 2021

View reviewed changes

pkg/mock/mockcluster/mockcluster.go Outdated Show resolved Hide resolved

pkg/mock/mockcluster/mockcluster.go Outdated Show resolved Hide resolved

HunDunDM added 2 commits August 6, 2021 21:29

Merge commit '21fef0309c2383e66b67976d4e00a0447693b37e' into tiflash-…

f8b4e73

…hot-write-sche # Conflicts: # server/core/store.go

fix typo

8a00df6

Signed-off-by: HunDunDM <hundundm@gmail.com>

ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 6, 2021

refine test

6d5ff6f

Signed-off-by: HunDunDM <hundundm@gmail.com>

lhy1024 approved these changes Aug 6, 2021

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 6, 2021

nolouch reviewed Aug 6, 2021

View reviewed changes

server/schedulers/hot_region_config.go Show resolved Hide resolved

server/schedulers/hot_region.go Show resolved Hide resolved

HunDunDM added 3 commits August 7, 2021 02:40

add comment

6668317

Signed-off-by: HunDunDM <hundundm@gmail.com>

add comment

f1c0e7f

Signed-off-by: HunDunDM <hundundm@gmail.com>

adjust config

d69523c

Signed-off-by: HunDunDM <hundundm@gmail.com>

nolouch approved these changes Aug 7, 2021

View reviewed changes

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 7, 2021

Merge branch 'master' into tiflash-hot-write-sche

e314dac

ti-chi-bot merged commit 8481fab into tikv:master Aug 7, 2021

HunDunDM added a commit to HunDunDM/pd that referenced this pull request Aug 10, 2021

scheduler: balance-hot-region-scheduler supports TiFlash (tikv#3900)

4c54226

Signed-off-by: HunDunDM <hundundm@gmail.com>

HunDunDM deleted the tiflash-hot-write-sche branch August 10, 2021 09:59

HunDunDM mentioned this pull request Aug 10, 2021

pd-control: add new config of balance-hot-region-scheduler pingcap/docs-cn#6835

Merged

12 tasks

HunDunDM changed the title ~~scheduler: balance-hot-region-scheduler supports TiFlash~~ scheduler: balance-hot-region-scheduler supports TiFlash hot-write Sep 8, 2021

JaySon-Huang mentioned this pull request Sep 7, 2023

PD can't redistribute the hot write regions among TiFlash nodes #3261

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

scheduler: balance-hot-region-scheduler supports TiFlash hot-write #3900

scheduler: balance-hot-region-scheduler supports TiFlash hot-write #3900

HunDunDM commented Jul 22, 2021 •

edited

Loading

ti-chi-bot commented Jul 22, 2021 •

edited

Loading

codecov bot commented Jul 27, 2021 •

edited

Loading

nolouch left a comment

nolouch commented Aug 7, 2021

ti-chi-bot commented Aug 7, 2021

ti-chi-bot commented Aug 7, 2021

scheduler: balance-hot-region-scheduler supports TiFlash hot-write #3900

scheduler: balance-hot-region-scheduler supports TiFlash hot-write #3900

Conversation

HunDunDM commented Jul 22, 2021 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Jul 22, 2021 • edited Loading

codecov bot commented Jul 27, 2021 • edited Loading

Codecov Report

nolouch left a comment

Choose a reason for hiding this comment

nolouch commented Aug 7, 2021

ti-chi-bot commented Aug 7, 2021

ti-chi-bot commented Aug 7, 2021

HunDunDM commented Jul 22, 2021 •

edited

Loading

ti-chi-bot commented Jul 22, 2021 •

edited

Loading

codecov bot commented Jul 27, 2021 •

edited

Loading