Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

statistics: skip reading mysql.stats_histograms if cached stats is up-to-date #24175

Merged
merged 3 commits into from
Apr 22, 2021
Merged

statistics: skip reading mysql.stats_histograms if cached stats is up-to-date #24175

merged 3 commits into from
Apr 22, 2021

Conversation

eurekaka
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #24096

Problem Summary:

There is no write in the cluster, while mysql.stats_histograms is repeatedly read.

What is changed and how it works?

What's Changed:

Skip reading mysql.stats_histograms if the cached stats is up-to-date.

Related changes

N/A

Check List

Tests

  • Unit test

Side effects

N/A

Release note

  • Skip reading mysql.stats_histograms if cached stats is up-to-date

@eurekaka eurekaka added type/enhancement The issue or PR belongs to an enhancement. sig/planner SIG: Planner component/statistics labels Apr 20, 2021
@eurekaka eurekaka requested a review from a team as a code owner April 20, 2021 09:36
@eurekaka eurekaka requested review from hanfei1991 and removed request for a team April 20, 2021 09:36
@ti-chi-bot ti-chi-bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Apr 20, 2021
@eurekaka eurekaka requested review from time-and-fate, winoros and crazycs520 and removed request for hanfei1991 April 20, 2021 09:36
@eurekaka eurekaka mentioned this pull request Apr 20, 2021
Closed
@zhouqiang-cl
Copy link
Contributor

/run-all-tests

winoros
winoros approved these changes Apr 20, 2021
View reviewed changes
Copy link
Member

@winoros winoros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change LGTM, reading meta in 3 leases should not cost too much

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Apr 20, 2021
@crazycs520
Copy link
Contributor

/lgtm

@ti-chi-bot
Copy link
Member

@crazycs520: /lgtm is only allowed for the reviewers in list.

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@time-and-fate
Copy link
Member

The test (*testStatsSuite).TestVersion failed.

Background:

tidb/statistics/handle/handle.go

Lines 837 to 839 in 59b99ce

// If we didn't find a Column or Index in tableInfo, we won't load the histogram for it.
// But don't worry, next lease the ddl will be updated, and we will load a same table for two times to
// avoid error.

When loading stats, we only load columns/indexes which exist in the infoschema.

The test case is basically asserting that when infoschema in a tidb get updated (added a column in the test case) but stats in the tikv doesn't change (which means no version of the stats is changed), (*Handle).Update(infoschema.InfoSchema) should update the stats cache according to the changes of infoschema.

Before this PR, this is achieved by this redundant loading of stats that already has version up to date. But now it's removed in this PR.

I think changes in this PR are reasonable, but we still need to decide what's the correct behavior when scenarios similar to the test case happen.

@dianqihanwangzi
Copy link
Contributor

/run-all-tests

@ti-chi-bot ti-chi-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Apr 21, 2021
@eurekaka
Copy link
Contributor Author

/run-all-tests

time-and-fate
time-and-fate approved these changes Apr 21, 2021
View reviewed changes
Copy link
Member

@time-and-fate time-and-fate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • time-and-fate
  • winoros

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 21, 2021
@time-and-fate
Copy link
Member

/run-integration-common-test

@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 21, 2021
@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 22, 2021
@eurekaka
Copy link
Contributor Author

/run-unit-test

1 similar comment
@eurekaka
Copy link
Contributor Author

/run-unit-test

@ti-chi-bot ti-chi-bot merged commit 734084f into pingcap:master Apr 22, 2021
@eurekaka eurekaka deleted the skip_update branch April 22, 2021 09:41
@ti-srebot ti-srebot mentioned this pull request Apr 27, 2021
Merged
@ti-srebot
Copy link
Contributor

cherry pick to release-5.0 in PR #24317

@eurekaka
Copy link
Contributor Author

/run-cherry-picker

@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 failed

@ti-srebot ti-srebot mentioned this pull request Apr 28, 2021
Merged
@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 in PR #24352

eurekaka added a commit to eurekaka/tidb that referenced this pull request Apr 28, 2021
@eurekaka
statistics: skip reading mysql.stats_histograms if cached stats is up…
fc6b9f3 
…-to-date (pingcap#24175)
eurekaka added a commit to ti-srebot/tidb that referenced this pull request Apr 28, 2021
@eurekaka
cherry pick pingcap#24175 to release-4.0
432bc38 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
ti-chi-bot pushed a commit that referenced this pull request May 10, 2021
@ti-srebot
statistics: skip reading mysql.stats_histograms if cached stats is up…
06ec53c 
…-to-date (#24175) (#24352)
eurekaka added a commit to ti-srebot/tidb that referenced this pull request May 28, 2021
@eurekaka
cherry pick pingcap#24175 to release-5.0
9ca407e 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
ti-chi-bot pushed a commit that referenced this pull request May 28, 2021
@ti-srebot
statistics: skip reading mysql.stats_histograms if cached stats is up…
9d6ed28 
…-to-date (#24175) (#24317)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@winoros winoros winoros approved these changes

@time-and-fate time-and-fate time-and-fate approved these changes

@crazycs520 crazycs520 Awaiting requested review from crazycs520

Assignees
No one assigned
Labels
component/statistics needs-cherry-pick-release-5.0 severity/critical sig/planner SIG: Planner size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

statistic load useless stats_histograms endlessly
8 participants
@eurekaka @zhouqiang-cl @crazycs520 @ti-chi-bot @time-and-fate @dianqihanwangzi @ti-srebot @winoros