addindex: add import speed metric #60904

CbcWestwolf · 2025-04-28T12:06:46Z

What problem does this PR solve?

Issue Number: close #60925

Problem Summary:

See the description in the issue.

What changed and how does it work?

We can utilize the common metric in pkg/lightning to get the total writing size

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

tiup playground --db.binpath=./bin/tidb-server --db 2 --kv 4
prepare data: sysbench oltp_read_write prepare --mysql-host=127.0.0.1 --mysql-port=4000 --mysql-user=root --mysql-db=test --table-size=100000000
alter table sbtest1 add index i(k, c, pad) using different adding index ways and write speed

admin show ddl jobs where job_type = 'add index';
+--------+---------+------------+-----------+--------------+-----------+----------+-----------+----------------------------+----------------------------+----------------------------+--------+-----------------------------------------------------------------+
| JOB_ID | DB_NAME | TABLE_NAME | JOB_TYPE  | SCHEMA_STATE | SCHEMA_ID | TABLE_ID | ROW_COUNT | CREATE_TIME                | START_TIME                 | END_TIME                   | STATE  | COMMENTS                                                        |
+--------+---------+------------+-----------+--------------+-----------+----------+-----------+----------------------------+----------------------------+----------------------------+--------+-----------------------------------------------------------------+
|    123 | test    | sbtest1    | add index | public       |         2 |      112 | 100000000 | 2025-04-28 23:12:15.904000 | 2025-04-28 23:12:15.904000 | 2025-04-28 23:23:27.054000 | synced | ingest, max_write_speed=41943040, max_node_count=3              |
|    121 | test    | sbtest1    | add index | public       |         2 |      112 | 100000000 | 2025-04-28 23:05:16.405000 | 2025-04-28 23:05:16.455000 | 2025-04-28 23:11:37.205000 | synced | ingest, max_write_speed=104857600, max_node_count=3             |
|    119 | test    | sbtest1    | add index | public       |         2 |      112 | 100000000 | 2025-04-28 23:00:31.805000 | 2025-04-28 23:00:31.805000 | 2025-04-28 23:03:58.655000 | synced | ingest, DXF, cloud, max_write_speed=104857600, max_node_count=3 |
|    117 | test    | sbtest1    | add index | public       |         2 |      112 | 100000000 | 2025-04-28 22:52:54.754000 | 2025-04-28 22:52:54.754000 | 2025-04-28 22:57:58.555000 | synced | ingest, DXF, max_write_speed=104857600, max_node_count=3        |

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

codecov · 2025-04-28T12:27:30Z

Codecov Report

Attention: Patch coverage is 13.15789% with 33 lines in your changes missing coverage. Please review.

Project coverage is 74.6163%. Comparing base (e8fb24a) to head (b52d8bc).
Report is 452 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #60904        +/-   ##
================================================
+ Coverage   73.1316%   74.6163%   +1.4846%     
================================================
  Files          1719       1768        +49     
  Lines        476215     488790     +12575     
================================================
+ Hits         348264     364717     +16453     
+ Misses       106583     101188      -5395     
- Partials      21368      22885      +1517

Flag	Coverage Δ
integration	`48.7142% <14.7058%> (?)`
unit	`72.3369% <13.1578%> (-0.0144%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.7804% <ø> (+0.0685%)`	⬆️
parser	`∅ <ø> (∅)`
br	`61.4002% <ø> (+14.0459%)`	⬆️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

pkg/metrics/ddl.go

Co-authored-by: tangenta <tangenta@126.com>

tangenta · 2025-04-29T06:44:53Z

pkg/metrics/ddl.go

@@ -86,6 +95,10 @@ var (
 	AddIndexScanRate      *prometheus.HistogramVec
 )

+func init() {
+	registeredJob = make(map[int64]*metric.Common, 64)


Could you add a check to CheckIngestLeakageForTest to ensure the metrics are unregistered?

pkg/ddl/backfilling_import_cloud.go

CbcWestwolf · 2025-04-29T12:40:25Z

/retest

CbcWestwolf · 2025-04-30T07:37:59Z

/retest

pkg/ddl/backfilling_import_cloud.go

pkg/metrics/ddl.go

D3Hunter · 2025-04-30T09:22:10Z

pkg/ddl/backfilling_read_index.go

@@ -61,6 +62,8 @@ type readIndexStepExecutor struct {
 	backend        *local.Backend
 	// pipeline of current running subtask, it's nil when no subtask is running.
 	currPipe atomic.Pointer[operator.AsyncPipeline]
+
+	m *lightningmetric.Common


D3Hunter · 2025-04-30T09:27:09Z

tests/realtikvtest/addindextest2/global_sort_test.go

+	tk.MustExec("set @@global.tidb_cloud_storage_uri = '';")
+	tk.MustExec("alter table t add index idx(a);")
+	tk.MustExec("alter table t drop index idx;")
+


what's this test for? if you set tk.MustExec("set @@global.tidb_cloud_storage_uri = '';"). it might break later cases

Let me remove it

Co-authored-by: D3Hunter <jujj603@gmail.com>

joechenrh

Rest LGTM

pkg/metrics/ddl.go

ti-chi-bot · 2025-05-06T02:40:46Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: joechenrh, tangenta

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [tangenta]
~~pkg/ddl/OWNERS~~ [tangenta]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2025-05-06T02:40:50Z

[LGTM Timeline notifier]

Timeline:

2025-04-30 08:09:39.372635589 +0000 UTC m=+1034323.184425970: ☑️ agreed by tangenta.
2025-05-06 02:40:49.82637982 +0000 UTC m=+322897.009283076: ☑️ agreed by joechenrh.

Co-authored-by: Ruihao Chen <joechenrh@gmail.com>

CbcWestwolf · 2025-05-06T03:26:49Z

/retest

CbcWestwolf · 2025-06-25T03:10:40Z

/cherrypick release-8.1
/cherrypick release-8.5

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2025-06-25T03:11:24Z

@CbcWestwolf: new pull request created to branch release-8.1: #61980.
But this PR has conflicts, please resolve them!

In response to this:

/cherrypick release-8.1
/cherrypick release-8.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2025-06-25T03:12:07Z

@CbcWestwolf: new pull request created to branch release-8.5: #61981.
But this PR has conflicts, please resolve them!

In response to this:

/cherrypick release-8.1
/cherrypick release-8.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

close #60925

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2025-07-11T09:54:31Z

In response to a cherrypick label: new pull request created to branch release-7.5: #62379.
But this PR has conflicts, please resolve them!

ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Apr 28, 2025

CbcWestwolf force-pushed the ddl_metrics branch from 7b1f41f to 08c4a38 Compare April 28, 2025 12:07

ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Apr 28, 2025

ti-chi-bot bot removed the do-not-merge/needs-tests-checked label Apr 28, 2025

init

296f815

CbcWestwolf force-pushed the ddl_metrics branch from 08c4a38 to 296f815 Compare April 29, 2025 03:37

ti-chi-bot bot removed the do-not-merge/needs-linked-issue label Apr 29, 2025

CbcWestwolf requested a review from tangenta April 29, 2025 06:21

tangenta reviewed Apr 29, 2025

View reviewed changes

pkg/metrics/ddl.go Outdated Show resolved Hide resolved

CbcWestwolf changed the title ~~metrics: add some metrics for adding index~~ metrics: add import speed metric for adding index Apr 29, 2025

CbcWestwolf force-pushed the ddl_metrics branch from c6d9b16 to 04855c2 Compare April 29, 2025 06:28

Update pkg/metrics/ddl.go

967aaa8

Co-authored-by: tangenta <tangenta@126.com>

CbcWestwolf force-pushed the ddl_metrics branch from 04855c2 to 967aaa8 Compare April 29, 2025 06:29

tangenta reviewed Apr 29, 2025

View reviewed changes

D3Hunter reviewed Apr 29, 2025

View reviewed changes

pkg/ddl/backfilling_import_cloud.go Outdated Show resolved Hide resolved

D3Hunter changed the title ~~metrics: add import speed metric for adding index~~ addindex: add import speed metric Apr 29, 2025

CbcWestwolf added 2 commits April 29, 2025 17:40

introduce CheckDDLMetricsLeakageForTest

4aefb38

rename the receiver of cloudImportExecutor

a3e4c9b

remove CheckDDLMetricsLeakageForTest

74ee9ce

CbcWestwolf force-pushed the ddl_metrics branch from a2177bf to 74ee9ce Compare April 30, 2025 06:56

rename

612a7b0

CbcWestwolf requested review from D3Hunter and tangenta April 30, 2025 08:00

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Apr 30, 2025

D3Hunter reviewed Apr 30, 2025

View reviewed changes

Apply suggestions from code review

3e3ba5b

Co-authored-by: D3Hunter <jujj603@gmail.com>

CbcWestwolf force-pushed the ddl_metrics branch from 8cc88f6 to 3e3ba5b Compare April 30, 2025 09:37

joechenrh approved these changes May 6, 2025

View reviewed changes

pkg/metrics/ddl.go Outdated Show resolved Hide resolved

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels May 6, 2025

Update pkg/metrics/ddl.go

b52d8bc

Co-authored-by: Ruihao Chen <joechenrh@gmail.com>

CbcWestwolf force-pushed the ddl_metrics branch from 47105b2 to b52d8bc Compare May 6, 2025 02:50

ti-chi-bot bot merged commit e217a01 into pingcap:master May 6, 2025
27 checks passed

CbcWestwolf deleted the ddl_metrics branch May 6, 2025 04:25

joechenrh mentioned this pull request May 8, 2025

Update description of MAX_WRITE_SPEED pingcap/docs#20852

Draft

16 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jun 25, 2025

This is an automated cherry-pick of pingcap#60904

3c0c664

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Jun 25, 2025

addindex: add import speed metric (#60904) #61980

Merged

13 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jun 25, 2025

This is an automated cherry-pick of pingcap#60904

42bebf1

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Jun 25, 2025

addindex: add import speed metric (#60904) #61981

Merged

13 tasks

CbcWestwolf added a commit to ti-chi-bot/tidb that referenced this pull request Jun 25, 2025

This is an automated cherry-pick of pingcap#60904

c46edd5

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

CbcWestwolf mentioned this pull request Jul 3, 2025

addindex: add import speed metric #62173

Merged

13 tasks

ti-chi-bot bot pushed a commit that referenced this pull request Jul 4, 2025

addindex: add import speed metric (#60904) (#61981)

8a4b853

close #60925

ti-chi-bot bot pushed a commit that referenced this pull request Jul 4, 2025

addindex: add import speed metric (#60904) (#61980)

7b9479e

close #60925

ti-chi-bot bot added the needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. label Jul 11, 2025

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jul 11, 2025

This is an automated cherry-pick of pingcap#60904

703296c

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Jul 11, 2025

addindex: add import speed metric (#60904) #62379

Closed

13 tasks

addindex: add import speed metric #60904

addindex: add import speed metric #60904

Uh oh!

Conversation

CbcWestwolf commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

Uh oh!

codecov bot commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

tangenta Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

CbcWestwolf commented Apr 29, 2025

Uh oh!

CbcWestwolf commented Apr 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

D3Hunter Apr 30, 2025

Choose a reason for hiding this comment

Uh oh!

D3Hunter Apr 30, 2025

Choose a reason for hiding this comment

Uh oh!

CbcWestwolf Apr 30, 2025

Choose a reason for hiding this comment

Uh oh!

joechenrh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ti-chi-bot bot commented May 6, 2025

Uh oh!

ti-chi-bot bot commented May 6, 2025

[LGTM Timeline notifier]

Uh oh!

CbcWestwolf commented May 6, 2025

Uh oh!

Uh oh!

CbcWestwolf commented Jun 25, 2025

Uh oh!

ti-chi-bot commented Jun 25, 2025

Uh oh!

ti-chi-bot commented Jun 25, 2025

Uh oh!

ti-chi-bot commented Jul 11, 2025

Uh oh!

Uh oh!

CbcWestwolf commented Apr 28, 2025 •

edited

Loading

codecov bot commented Apr 28, 2025 •

edited

Loading