Skip to content

WIP: *: batching analyze requests#69686

Open
0xPoe wants to merge 2 commits into
pingcap:masterfrom
0xPoe:poe-batch-analyze
Open

WIP: *: batching analyze requests#69686
0xPoe wants to merge 2 commits into
pingcap:masterfrom
0xPoe:poe-batch-analyze

Conversation

@0xPoe

@0xPoe 0xPoe commented Jul 6, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: close TBD

Problem Summary:

We want to enable a batching mechanism for analyze requests to reduce network traffic and memory usage across different regions on the same TiKV node. This also prepares for the future sample-based NDV project.

What changed and how does it work?

Make the analyze requests batchable by using tidb_store_batch_size when constructing them. Also, try to remove unnecessary keep order settings from analyze requests, as Analyze V2 no longer relies on them.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

Summary by CodeRabbit

  • New Features

    • Analyze requests can now use store batching for more efficient execution.
    • Store batch size can be configured through a new request setting, giving finer control over request processing.
  • Bug Fixes

    • Improved task handling so eligible internal analyze requests are processed correctly with batching enabled.
    • Updated batching behavior for small tasks to better match request type and execution conditions.

0xPoe added 2 commits July 6, 2026 12:20
Signed-off-by: 0xPoe <poe.liu@pm.me>
Signed-off-by: 0xPoe <poe.liu@pm.me>
@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. labels Jul 6, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign cfzjywxk, time-and-fate for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jul 6, 2026
@0xPoe 0xPoe changed the title *: batching analyze requests WIP: *: batching analyze requests Jul 6, 2026
@ti-chi-bot ti-chi-bot Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 6, 2026
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 02013478-841d-4654-bae2-86476ccbe345

📥 Commits

Reviewing files that changed from the base of the PR and between ab1e197 and 7bed095.

📒 Files selected for processing (4)
  • pkg/distsql/request_builder.go
  • pkg/executor/analyze_col.go
  • pkg/store/copr/copr_test/coprocessor_test.go
  • pkg/store/copr/coprocessor.go

📝 Walkthrough

Walkthrough

This PR extends coprocessor store-batching support to Analyze requests. A new SetStoreBatchSize method is added to RequestBuilder, wired into AnalyzeColumnsExec, and coprocessor eligibility logic (canUseStoreBatchBuilder, canStoreBatchTask, checkStoreBatchCopr) is updated to permit batching for Analyze requests, including internal ones, with a new test verifying the behavior.

Changes

Analyze Request Store Batching

Layer / File(s) Summary
RequestBuilder setter and Analyze wiring
pkg/distsql/request_builder.go, pkg/executor/analyze_col.go
Adds SetStoreBatchSize to RequestBuilder and uses it in AnalyzeColumnsExec.buildResp, removing the explicit SetKeepOrder(true) call.
Coprocessor batching eligibility
pkg/store/copr/coprocessor.go
Introduces canUseStoreBatchBuilder and canStoreBatchTask helpers, updates buildCopTasks and batchStoreTaskBuilder.handle to use them, and updates checkStoreBatchCopr to allow Analyze requests (including internal ones) for store batching.
Test coverage
pkg/store/copr/copr_test/coprocessor_test.go
Adds assertions in TestBuildCopIteratorWithBatchStoreCopr verifying an internal Analyze request with StoreBatchSize=3 produces one task with 3 batch subtasks and RowCountHint=-1.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: rleungx, cfzjywxk, yudongusa

Poem

A rabbit hops through cop task queues,
Batching Analyze with joyful news,
No more KeepOrder standing guard,
StoreBatchSize now works so hard,
One task, three batches, tests all pass —
Hop, hop, hooray, the checks amass! 🐇✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description matches the template overall, but the required Issue Number is still a placeholder rather than a real linked issue. Replace "close TBD" with a valid issue reference like "close #12345" and ensure the problem statement points to that issue.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly matches the main change: batching analyze requests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="Running error: context loading failed: failed to load packages: failed to load packages: failed to load with go/packages: context deadline exceeded"
level=error msg="Timeout exceeded: try increasing it by passing --timeout option"


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot

ti-chi-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown

[FORMAT CHECKER NOTIFICATION]

Notice: To remove the do-not-merge/needs-linked-issue label, please provide the linked issue number on one line in the PR body, for example: Issue Number: close #123 or Issue Number: ref #456.

📖 For more info, you can check the "Contribute Code" section in the development guide.

@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.3484%. Comparing base (ab1e197) to head (7bed095).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #69686        +/-   ##
================================================
- Coverage   76.3237%   74.3484%   -1.9754%     
================================================
  Files          2041       2050         +9     
  Lines        560490     574733     +14243     
================================================
- Hits         427787     427305       -482     
- Misses       131802     147217     +15415     
+ Partials        901        211       -690     
Flag Coverage Δ
integration 40.8936% <100.0000%> (+1.1883%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 60.4471% <ø> (ø)
parser ∅ <ø> (∅)
br 47.4650% <ø> (-15.2563%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ti-chi-bot

ti-chi-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown

@0xPoe: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
idc-jenkins-ci-tidb/check_dev_2 7bed095 link true /test check-dev2

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/needs-linked-issue do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant