core: optimize the processing speed of RegionHeartbeat #3739

HunDunDM · 2021-06-04T01:43:05Z

What problem does this PR solve?

The current heartbeat processing speed can be increased.
close Avoid update region tree when region key range is not updated #3635

What is changed and how it works?

Move hotStat out of the read lock
Move storage.DeleteRegion out of the write lock
Simple processing logic is used for RegionHeartbeat that only has statistics updated.

Check List

Tests

Unit test

Release note

None

Signed-off-by: HunDunDM <hundundm@gmail.com>

…ted statistics Signed-off-by: HunDunDM <hundundm@gmail.com>

Signed-off-by: HunDunDM <hundundm@gmail.com>

…lock Signed-off-by: HunDunDM <hundundm@gmail.com>

Signed-off-by: HunDunDM <hundundm@gmail.com>

codecov · 2021-06-04T02:09:24Z

Codecov Report

Merging #3739 (ab69442) into master (ab061b5) will decrease coverage by 0.10%.
The diff coverage is 89.90%.

@@            Coverage Diff             @@
##           master    #3739      +/-   ##
==========================================
- Coverage   75.15%   75.05%   -0.11%     
==========================================
  Files         244      244              
  Lines       24478    24504      +26     
==========================================
- Hits        18397    18392       -5     
- Misses       4468     4490      +22     
- Partials     1613     1622       +9

Flag	Coverage Δ
unittests	`75.05% <89.90%> (-0.11%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
server/cluster/cluster.go	`82.84% <57.69%> (+0.08%)`	⬆️
server/core/basic_cluster.go	`90.72% <100.00%> (+0.12%)`	⬆️
server/core/region.go	`90.41% <100.00%> (+0.46%)`	⬆️
server/core/region_tree.go	`94.61% <100.00%> (+0.04%)`	⬆️
server/schedule/range_cluster.go	`87.03% <100.00%> (ø)`
pkg/tempurl/tempurl.go	`45.00% <0.00%> (-15.00%)`	⬇️
server/region_syncer/server.go	`83.33% <0.00%> (-6.07%)`	⬇️
server/tso/global_allocator.go	`70.55% <0.00%> (-5.00%)`	⬇️
pkg/etcdutil/etcdutil.go	`84.70% <0.00%> (-3.53%)`	⬇️
server/schedulers/random_merge.go	`60.00% <0.00%> (-3.34%)`	⬇️
... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ab061b5...ab69442. Read the comment docs.

Signed-off-by: HunDunDM <hundundm@gmail.com>

nolouch

rest LGTM

server/core/region.go

Signed-off-by: HunDunDM <hundundm@gmail.com>

nolouch

Lgtm

nolouch · 2021-06-05T12:59:24Z

PTAL @rleungx @lhy1024

ti-chi-bot · 2021-06-05T13:58:33Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

lhy1024
nolouch

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

nolouch · 2021-06-05T14:03:24Z

/merge

ti-chi-bot · 2021-06-05T14:03:25Z

@nolouch: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2021-06-05T14:03:27Z

This pull request has been accepted and is ready to merge.

Commit hash: ab69442

HunDunDM · 2021-06-05T14:28:52Z

/run-integration-common-test

* Unify SetRegion and AddRegion Signed-off-by: HunDunDM <hundundm@gmail.com> * core: optimize the processing speed of RegionHeartbeat with only updated statistics Signed-off-by: HunDunDM <hundundm@gmail.com> * rename Signed-off-by: HunDunDM <hundundm@gmail.com> * refine PreCheckPutRegion Signed-off-by: HunDunDM <hundundm@gmail.com> * Reduce the code during the holding period of the read lock and write lock Signed-off-by: HunDunDM <hundundm@gmail.com> * Move PreCheckPutRegion out of the read lock Signed-off-by: HunDunDM <hundundm@gmail.com> * remove logic about traceRegionFlow Signed-off-by: HunDunDM <hundundm@gmail.com> * merge SetRegion and addRegion Signed-off-by: HunDunDM <hundundm@gmail.com>

HunDunDM added 5 commits June 4, 2021 08:07

Unify SetRegion and AddRegion

ef5acd4

Signed-off-by: HunDunDM <hundundm@gmail.com>

core: optimize the processing speed of RegionHeartbeat with only upda…

906f977

…ted statistics Signed-off-by: HunDunDM <hundundm@gmail.com>

rename

fe44e42

Signed-off-by: HunDunDM <hundundm@gmail.com>

refine PreCheckPutRegion

c047086

Signed-off-by: HunDunDM <hundundm@gmail.com>

Reduce the code during the holding period of the read lock and write …

5486b2a

…lock Signed-off-by: HunDunDM <hundundm@gmail.com>

ti-chi-bot added the release-note-none Denotes a PR that doesn't merit a release note. label Jun 4, 2021

ti-chi-bot requested review from rleungx and Yisaer June 4, 2021 01:43

Move PreCheckPutRegion out of the read lock

f511a92

Signed-off-by: HunDunDM <hundundm@gmail.com>

remove logic about traceRegionFlow

e21145f

Signed-off-by: HunDunDM <hundundm@gmail.com>

nolouch added the priority/P0 The issue has P0 priority. label Jun 4, 2021

nolouch reviewed Jun 4, 2021

View reviewed changes

server/core/region.go Outdated Show resolved Hide resolved

server/core/region.go Show resolved Hide resolved

HunDunDM added 2 commits June 4, 2021 23:40

merge SetRegion and addRegion

72677bd

Signed-off-by: HunDunDM <hundundm@gmail.com>

Merge branch 'master' into region-hb-stat

ab69442

nolouch approved these changes Jun 4, 2021

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 4, 2021

lhy1024 approved these changes Jun 5, 2021

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 5, 2021

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 5, 2021

ti-chi-bot merged commit b5909a7 into tikv:master Jun 5, 2021

HunDunDM deleted the region-hb-stat branch June 5, 2021 14:51

ti-chi-bot added do-not-merge/invalid-commit-message and removed do-not-merge/invalid-commit-message labels Nov 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

core: optimize the processing speed of RegionHeartbeat #3739

core: optimize the processing speed of RegionHeartbeat #3739

HunDunDM commented Jun 4, 2021 •

edited

Loading

codecov bot commented Jun 4, 2021 •

edited

Loading

nolouch left a comment

nolouch left a comment

nolouch commented Jun 5, 2021

ti-chi-bot commented Jun 5, 2021

nolouch commented Jun 5, 2021

ti-chi-bot commented Jun 5, 2021

ti-chi-bot commented Jun 5, 2021

HunDunDM commented Jun 5, 2021

core: optimize the processing speed of RegionHeartbeat #3739

core: optimize the processing speed of RegionHeartbeat #3739

Conversation

HunDunDM commented Jun 4, 2021 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

codecov bot commented Jun 4, 2021 • edited Loading

Codecov Report

nolouch left a comment

Choose a reason for hiding this comment

nolouch left a comment

Choose a reason for hiding this comment

nolouch commented Jun 5, 2021

ti-chi-bot commented Jun 5, 2021

nolouch commented Jun 5, 2021

ti-chi-bot commented Jun 5, 2021

ti-chi-bot commented Jun 5, 2021

HunDunDM commented Jun 5, 2021

HunDunDM commented Jun 4, 2021 •

edited

Loading

codecov bot commented Jun 4, 2021 •

edited

Loading