statistics: add metrics for unneeded analyze table #54822

hawkingrei · 2024-07-23T04:22:58Z

What problem does this PR solve?

Issue Number: close #54823

Problem Summary:

What changed and how does it work?

The primary changes in pkg/statistics/handle/cache/statscache.go file.
the rest are code migrations to avoid cyclic imports.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)

I test in the 1m tables cluster. we can see this metrics in the grafana.

No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

codecov · 2024-07-23T04:57:18Z

Codecov Report

Attention: Patch coverage is 90.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 56.9952%. Comparing base (f774ef6) to head (2a88ef3).
Report is 38 commits behind head on master.

Additional details and impacted files

@@                Coverage Diff                @@
##             master     #54822         +/-   ##
=================================================
- Coverage   74.7207%   56.9952%   -17.7256%     
=================================================
  Files          1557       1684        +127     
  Lines        363500     631235     +267735     
=================================================
+ Hits         271610     359774      +88164     
- Misses        72240     247422     +175182     
- Partials      19650      24039       +4389

Flag	Coverage Δ
integration	`37.6885% <85.0000%> (?)`
unit	`72.1299% <90.0000%> (-1.5151%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`53.3749% <ø> (-1.8246%)`	⬇️
parser	`∅ <ø> (∅)`
br	`51.9165% <ø> (+4.2288%)`	⬆️

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

hawkingrei · 2024-07-23T06:46:10Z

/retest

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

hawkingrei · 2024-07-24T03:53:50Z

/retest

hawkingrei · 2024-07-24T05:42:37Z

/retest

Rustin170506

Do we need to update the grafana json file?

pkg/statistics/table.go

Co-authored-by: 二手掉包工程师 <rustin.liu@gmail.com>

hawkingrei · 2024-07-25T07:35:43Z

Do we need to update the grafana json file?

No, because this type is generated dynamically by the grafana.

ti-chi-bot · 2024-07-25T07:53:21Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hi-rustin, winoros

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [hi-rustin,winoros]
~~pkg/statistics/OWNERS~~ [hi-rustin,winoros]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2024-07-25T07:53:25Z

[LGTM Timeline notifier]

Timeline:

2024-07-24 12:41:39.274428107 +0000 UTC m=+1048921.265369577: ☑️ agreed by winoros.
2024-07-25 07:53:23.497835995 +0000 UTC m=+1118025.488777466: ☑️ agreed by hi-rustin.

hawkingrei · 2024-07-25T08:21:27Z

/retest

hawkingrei · 2024-07-26T01:05:51Z

/retest

* upstream/master: (93 commits) disjoinset: add generic impl (pingcap#54917) planner: derive index filters for mv index paths (pingcap#54877) br: cli refactor backup error handling logic (pingcap#54697) expression: fix infinity loop in `timestampadd` (pingcap#54916) planner: import more expand test. (pingcap#54962) planner: use code-gen to generate CloneForPlanCache method for some operators (pingcap#54957) test: fix flaky test TestFailSchemaSyncer (pingcap#54958) planner: move logical show into logicalop pkg. (pingcap#54928) privilege: Remove TestAbnormalMySQLTable (pingcap#54925) resource_control: support unlimited keyword when setting the resource group (pingcap#54704) ddl: fix a data race on localRowCntListener Written() (pingcap#54484) lightning: fix SET SESSION on connection pool (pingcap#54927) planner: move logical mem-table to logicalop pkg. (pingcap#54903) table: Add `CachedTableSupport` and `TemporaryTableSupport` for `MutateContext` (pingcap#54900) executor: fix index_hash_join hang when context canceled (pingcap#54855) statistics: add metrics for unneeded analyze table (pingcap#54822) *: refine pipelined dml benchmarks (pingcap#54844) ddl: assign table IDs for jobs submitted to queue (pingcap#54880) *: avoid using Tables field of model.DBInfo, use API instead (pingcap#52302) planner: remove useless check (pingcap#54907) ...

hawkingrei · 2024-08-22T03:32:08Z

/cherrypick release-8.1
/cherrypick release-7.5

ti-chi-bot · 2024-08-22T03:32:50Z

@hawkingrei: new pull request created to branch release-7.5: #55577.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2024-08-22T03:33:31Z

@hawkingrei: new pull request created to branch release-8.1: #55578.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

close #54823

ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 23, 2024

ti-chi-bot bot removed do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked labels Jul 23, 2024

hawkingrei added 2 commits July 23, 2024 13:40

statistics: add metrics for unneeded analyze table

fb56168

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

statistics: add metrics for unneeded analyze table

aa6287a

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

hawkingrei force-pushed the add_health_for_uneligibleAnalyze branch from 69d932a to aa6287a Compare July 23, 2024 05:41

hawkingrei requested a review from winoros July 23, 2024 09:55

update

0907f8a

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

winoros approved these changes Jul 24, 2024

View reviewed changes

ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 24, 2024

hawkingrei requested a review from Rustin170506 July 25, 2024 03:55

Rustin170506 reviewed Jul 25, 2024

View reviewed changes

pkg/statistics/table.go Outdated Show resolved Hide resolved

Update pkg/statistics/table.go

2a88ef3

Co-authored-by: 二手掉包工程师 <rustin.liu@gmail.com>

Rustin170506 approved these changes Jul 25, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 25, 2024

ti-chi-bot bot merged commit 7e73ddc into pingcap:master Jul 26, 2024
23 checks passed

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Aug 22, 2024

This is an automated cherry-pick of pingcap#54822

2848d33

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Aug 22, 2024

statistics: add metrics for unneeded analyze table (#54822) #55577

Merged

13 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Aug 22, 2024

This is an automated cherry-pick of pingcap#54822

2f8e42c

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Aug 22, 2024

statistics: add metrics for unneeded analyze table (#54822) #55578

Open

13 tasks

hawkingrei deleted the add_health_for_uneligibleAnalyze branch August 22, 2024 03:36

ti-chi-bot bot pushed a commit that referenced this pull request Aug 25, 2024

statistics: add metrics for unneeded analyze table (#54822) (#55577)

47279f5

close #54823

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

statistics: add metrics for unneeded analyze table #54822

statistics: add metrics for unneeded analyze table #54822

hawkingrei commented Jul 23, 2024 •

edited

Loading

codecov bot commented Jul 23, 2024 •

edited

Loading

hawkingrei commented Jul 23, 2024

hawkingrei commented Jul 24, 2024

hawkingrei commented Jul 24, 2024

Rustin170506 left a comment •

edited

Loading

hawkingrei commented Jul 25, 2024 •

edited

Loading

ti-chi-bot bot commented Jul 25, 2024

ti-chi-bot bot commented Jul 25, 2024

hawkingrei commented Jul 25, 2024

hawkingrei commented Jul 26, 2024

hawkingrei commented Aug 22, 2024

ti-chi-bot commented Aug 22, 2024

ti-chi-bot commented Aug 22, 2024

statistics: add metrics for unneeded analyze table #54822

statistics: add metrics for unneeded analyze table #54822

Conversation

hawkingrei commented Jul 23, 2024 • edited Loading

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

codecov bot commented Jul 23, 2024 • edited Loading

Codecov Report

hawkingrei commented Jul 23, 2024

hawkingrei commented Jul 24, 2024

hawkingrei commented Jul 24, 2024

Rustin170506 left a comment • edited Loading

Choose a reason for hiding this comment

hawkingrei commented Jul 25, 2024 • edited Loading

ti-chi-bot bot commented Jul 25, 2024

ti-chi-bot bot commented Jul 25, 2024

[LGTM Timeline notifier]

hawkingrei commented Jul 25, 2024

hawkingrei commented Jul 26, 2024

hawkingrei commented Aug 22, 2024

ti-chi-bot commented Aug 22, 2024

ti-chi-bot commented Aug 22, 2024

hawkingrei commented Jul 23, 2024 •

edited

Loading

codecov bot commented Jul 23, 2024 •

edited

Loading

Rustin170506 left a comment •

edited

Loading

hawkingrei commented Jul 25, 2024 •

edited

Loading