Skip to content

coprocessor: handle paging result even if not enabled#63670

Merged
ti-chi-bot[bot] merged 3 commits into
pingcap:masterfrom
tangenta:handle-copr-paging-result
Sep 24, 2025
Merged

coprocessor: handle paging result even if not enabled#63670
ti-chi-bot[bot] merged 3 commits into
pingcap:masterfrom
tangenta:handle-copr-paging-result

Conversation

@tangenta

@tangenta tangenta commented Sep 22, 2025

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: ref #61702

Problem Summary:

tikv-server may return paging range even if paging is not enabled. Then if the range is not handled, the result would be wrong.

What changed and how does it work?

Cherry-pick https://github.com/tidbcloud/tidb-cse/pull/1194.

handle paging result if copResp has Range, even if paging is not enabled.

Check List

Tests

[ERROR] [reporter.go:274] ["admin check found data inconsistency"] [keyspaceName=10708255509240425689] [conn=2936078708] [session_alias=] [table_name=t] [index_name=k] [row_id=28802665] [index=] [row="handle: 28802665, values: [KindInt64 83349]"] [row_mvcc="{\"decoded\":{\"460711990272720897\":{\"bigint_0\":\"63763\",\"bigint_1\":\"98976\",\"bigint_2\":\"41844\",\"c\":\"l6dHRhjDH2ZqOedf\",\"int_0\":\"83231\",\"int_1\":\"72771\",\"int_2\":\"72693\",\"k\":\"83349\",\"pad\":\"YNHPrCdJ6imopFHm\",\"text_0\":\"dgfmKbqpOFjARcR5\",\"varchar_0\":\"M7tghkSBWtsoJQzt\"}},\"key\":\"74800000000000000B5F728000000001B77E69\",\"mvcc\":{\"info\":{\"writes\":[{\"start_ts\":460711990272720897,\"commit_ts\":460711990272720897,\"short_value\":\"gAALAAAAAgMEBQYHCAkKCwwEABQAJAAoACwAMAA0ADgAPABMAFwAlUUBAGw2ZEhSaGpESDJacU9lZGZZTkhQckNkSjZpbW9wRkhtH0UBAEMcAQD1GwEAE/kAAKCCAQB0owAATTd0Z2hrU0JXdHNvSlF6dGRnZm1LYnFwT0ZqQVJjUjU=\"}]}},\"regionID\":1883}"]

[INFO] [index_cop.go:110] [fetchTableScanResult] [keyspaceName=SYSTEM] [rows=224] [columns=2] [firstKey=74800000000000000b5f728000000001bc0e36] [lastKey=74800000000000000b5f728000000001bc0f15]
[INFO] [coprocessor.go:1515] ["[TIME_COP_WAIT] resp_time:1.525922019s txnStartTS:460942369916452865 region_id:692 store_addr:db-dff78b4c-tikv-oq64hd.db-cluster.tidb1633599162411516:20160 stats:Cop:{num_rpc:1, total_time:1.53s} kv_process_ms:5 kv_wait_ms:0 kv_read_ms:490 processed_versions:908367 total_versions:0 rocksdb_delete_skipped_count:0 rocksdb_key_skipped_count:0 rocksdb_cache_hit_count:0 rocksdb_read_count:0 rocksdb_read_byte:0"] [keyspaceName=SYSTEM] [task-id=2] [task-key=10560875421529241256/ddl/backfill/13] [subtaskID=4] [step=read-index]
[INFO] [index_cop.go:110] [fetchTableScanResult] [keyspaceName=SYSTEM] [rows=1024] [columns=2] [firstKey=74800000000000000b5f728000000001b702aa] [lastKey=74800000000000000b5f728000000001b706a9]

[INFO] [backfilling_operators.go:679] ["write external storage operator total count"] [keyspaceName=SYSTEM] [task-id=2] [task-key=10708255509240425689/ddl/backfill/13] [subtaskID=4] [step=read-index] [count=90262661]

TiKV log:

[INFO] [tracker.rs:271] [slow-query] [perf_stats.internal_delete_skipped_count=0] [perf_stats.internal_key_skipped_count=0] [perf_stats.block_read_byte=0] [perf_stats.block_read_count=0] [perf_stats.block_cache_hit_count=0] [scan.range.first="Some(start: 7800000174800000000000000B5F728000000001970ECA end: 7800000174800000000000000B5F728000000001BC0E36)"] [scan.ranges=1] [scan.total=2095073] [scan.processed_size=314414976] [scan.processed=2095072] [scan.is_desc=false] [tag=select] [table_id=0] [txn_start_ts=460969938578833409] [total_suspend_time=6.96508ms] [total_process_time=1.542507015s] [handler_build_time=20.046µs] [wait_time.snapshot=533.886µs] [wait_time.schedule=15.957µs] [wait_time=549.843µs] [total_lifetime=1.550048905s] [remote_host=ipv4:10.0.24.17:47534] [region_id=692] [query_digest=] [session_alias=] [connection_id=0]
[INFO] [runner.rs:538] ["reach max response size, row count 2095072"]

After this PR:

mysql> alter table t add index idx(k);
Query OK, 0 rows affected (6 min 0.72 sec)

mysql> admin check table t;
Query OK, 0 rows affected (39.77 sec)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

Signed-off-by: tangenta <tangenta@126.com>
@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Sep 22, 2025
@tiprow

tiprow Bot commented Sep 22, 2025

Copy link
Copy Markdown

Hi @tangenta. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@codecov

codecov Bot commented Sep 22, 2025

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.8577%. Comparing base (140d1d1) to head (43fd2b2).
⚠️ Report is 19 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #63670        +/-   ##
================================================
+ Coverage   72.7621%   74.8577%   +2.0956%     
================================================
  Files          1842       1889        +47     
  Lines        498001     509982     +11981     
================================================
+ Hits         362356     381761     +19405     
+ Misses       113627     104746      -8881     
- Partials      22018      23475      +1457     
Flag Coverage Δ
integration 48.5698% <100.0000%> (?)
unit 72.3649% <100.0000%> (+0.0627%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.8700% <ø> (ø)
parser ∅ <ø> (∅)
br 63.2729% <ø> (+16.8989%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: tangenta <tangenta@126.com>
@tangenta

Copy link
Copy Markdown
Contributor Author

/retest

@tiprow

tiprow Bot commented Sep 24, 2025

Copy link
Copy Markdown

@tangenta: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Comment thread pkg/store/copr/coprocessor.go Outdated
@ti-chi-bot ti-chi-bot Bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 24, 2025
Signed-off-by: tangenta <tangenta@126.com>
@ti-chi-bot

ti-chi-bot Bot commented Sep 24, 2025

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: D3Hunter, fzzf678

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 24, 2025
@ti-chi-bot

ti-chi-bot Bot commented Sep 24, 2025

Copy link
Copy Markdown

[LGTM Timeline notifier]

Timeline:

  • 2025-09-24 03:35:11.975391641 +0000 UTC m=+414122.045885323: ☑️ agreed by D3Hunter.
  • 2025-09-24 04:11:52.072736897 +0000 UTC m=+416322.143230574: ☑️ agreed by fzzf678.

@hawkingrei

Copy link
Copy Markdown
Contributor

/retest

1 similar comment
@fzzf678

fzzf678 commented Sep 24, 2025

Copy link
Copy Markdown
Contributor

/retest

@tiprow

tiprow Bot commented Sep 24, 2025

Copy link
Copy Markdown

@fzzf678: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot Bot merged commit f13d659 into pingcap:master Sep 24, 2025
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants