Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

copr: support load_based_replica_read_threshold #40742

Merged
merged 4 commits into from
Mar 3, 2023
Merged

copr: support load_based_replica_read_threshold #40742

merged 4 commits into from
Mar 3, 2023

Conversation

sticnarf
Copy link
Contributor

@sticnarf sticnarf commented Jan 20, 2023 •
edited
Loading

Signed-off-by: Yilin Chen sticnarf@gmail.com

What problem does this PR solve?

Issue Number: close #41664

Refers to the TiKV RFC tikv/rfcs#105 and support setting threshold for coprocessor requests.

What is changed and how it works?

Add a session and global sysvar tidb_load_based_replica_read_threshold.

If the estimated waiting duration at TiKV exceeds the threshold, replica read is automatically enabled.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jan 20, 2023 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • cfzjywxk
  • you06

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added do-not-merge/invalid-title do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 20, 2023
@sticnarf sticnarf changed the title Add load_based_replica_read_threshold sysvar copr: support load_based_replica_read_threshold Jan 20, 2023
@ti-chi-bot ti-chi-bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed do-not-merge/invalid-title labels Jan 20, 2023
@sticnarf sticnarf force-pushed the load-based-replica-read branch from b7e1b37 to 941d230 Compare February 1, 2023 06:05
@ti-chi-bot ti-chi-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Feb 1, 2023
@sticnarf sticnarf force-pushed the load-based-replica-read branch 2 times, most recently from 62e68b7 to 478fe8c Compare February 8, 2023 09:11
@sticnarf sticnarf mentioned this pull request Feb 8, 2023
Merged
@sticnarf sticnarf force-pushed the load-based-replica-read branch from 478fe8c to 46a8797 Compare February 14, 2023 05:55
@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 21, 2023
@sticnarf sticnarf force-pushed the load-based-replica-read branch from 46a8797 to c8c3fcc Compare February 22, 2023 07:50
@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 22, 2023
@sticnarf sticnarf force-pushed the load-based-replica-read branch from c8c3fcc to 0521651 Compare February 22, 2023 08:18
@sticnarf sticnarf marked this pull request as ready for review February 22, 2023 08:20
@sticnarf sticnarf requested a review from a team as a code owner February 22, 2023 08:20
@ti-chi-bot ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 22, 2023
@pingcap pingcap deleted a comment from ti-chi-bot Feb 22, 2023
@pingcap pingcap deleted a comment from ti-chi-bot Feb 22, 2023
@sticnarf sticnarf requested review from you06 and cfzjywxk February 22, 2023 09:01
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Feb 22, 2023
you06
you06 reviewed Feb 23, 2023
View reviewed changes
store/copr/coprocessor.go Outdated
@@ -1137,6 +1137,7 @@ func (worker *copIteratorWorker) handleTaskOnce(bo *Backoffer, task *copTask, ch
TaskId: worker.req.TaskID,
RequestSource: task.requestSource.GetRequestSource(),
ResourceGroupName: worker.req.ResourceGroupName,
BusyThresholdMs: uint32(worker.req.StoreBusyThreshold.Milliseconds()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm worried about how to deal with the batched tasks, if we send them to the replica of the main task, there will be many region miss errors...

@sticnarf sticnarf force-pushed the load-based-replica-read branch from 0521651 to a40f2b3 Compare February 24, 2023 03:58
@sticnarf
Add load_based_replica_read_threshold sysvar
2569a1c 
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
@sticnarf
Disable load-based replica read when store batched tasks exist
fee5ef4 
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
@sticnarf sticnarf force-pushed the load-based-replica-read branch from a40f2b3 to fee5ef4 Compare February 24, 2023 08:55
@sticnarf
Copy link
Contributor Author

/test all

@ti-chi-bot
Copy link
Member

@sticnarf: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test all

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@you06
Copy link
Contributor

you06 commented Feb 24, 2023

/ok-to-test

@ti-chi-bot ti-chi-bot added the ok-to-test Indicates a PR is ready to be tested. label Feb 24, 2023
@you06
Copy link
Contributor

you06 commented Feb 24, 2023

/test all

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 1, 2023
@you06
Copy link
Contributor

you06 commented Mar 1, 2023

/test unit-test

@you06
Copy link
Contributor

you06 commented Mar 2, 2023

@cfzjywxk I think we can merge this PR.

@cfzjywxk
Copy link
Contributor

cfzjywxk commented Mar 2, 2023

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 8b7c3d1

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 2, 2023
@ti-chi-bot ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Mar 3, 2023
@sticnarf
Copy link
Contributor Author

sticnarf commented Mar 3, 2023

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 00bbf43

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 3, 2023
@hawkingrei
Copy link
Member

/retest

@you06
Copy link
Contributor

you06 commented Mar 3, 2023

/build

@you06
Copy link
Contributor

you06 commented Mar 3, 2023

/retest

@ti-chi-bot ti-chi-bot merged commit ac6a9eb into pingcap:master Mar 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@you06 you06 you06 approved these changes

@cfzjywxk cfzjywxk cfzjywxk approved these changes

Assignees
No one assigned
Labels
ok-to-test Indicates a PR is ready to be tested. release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support load-based replica read
5 participants
@sticnarf @ti-chi-bot @you06 @cfzjywxk @hawkingrei