Skip to content

br: check and compact for br#63419

Merged
ti-chi-bot[bot] merged 10 commits into
pingcap:masterfrom
Leavrth:check_and_compact_for_br
Sep 22, 2025
Merged

br: check and compact for br#63419
ti-chi-bot[bot] merged 10 commits into
pingcap:masterfrom
Leavrth:check_and_compact_for_br

Conversation

@Leavrth

@Leavrth Leavrth commented Sep 9, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #63176

Problem Summary:
After split large regions, there may be some TiKV having a SST containing both region meta kv and data kv.

What changed and how does it work?

compact this SST before ingest

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@Leavrth
check and compact for br
03e5644 
Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>
@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed do-not-merge/needs-tests-checked labels Sep 9, 2025
@tiprow

tiprow Bot commented Sep 9, 2025

Copy link
Copy Markdown

Hi @Leavrth. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@codecov

codecov Bot commented Sep 9, 2025 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 59.18367% with 40 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.5329%. Comparing base (6f7a47a) to head (d5e88e0).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files 
@@               Coverage Diff                @@
##             master     #63419        +/-   ##
================================================
+ Coverage   72.7763%   74.5329%   +1.7566%     
================================================
  Files          1842       1858        +16     
  Lines        497890     498527       +637     
================================================
+ Hits         362346     371567      +9221     
+ Misses       113543     104210      -9333     
- Partials      22001      22750       +749
Flag Coverage Δ
integration 45.8149% <59.1836%> (?)
unit 72.3028% <0.0000%> (-0.0170%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.8700% <ø> (ø)
parser ∅ <ø> (∅)
br 63.2956% <59.1836%> (+16.7978%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Leavrth
udpate kvproto
ea23bc9 
Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>
@ti-chi-bot ti-chi-bot Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Sep 17, 2025
@Leavrth
udpate kvproto
a47500b 
Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>
@Leavrth
fix typos
3afad30 
Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>
@Leavrth

Leavrth commented Sep 19, 2025

Copy link
Copy Markdown
Contributor Author

/retest

@tiprow

tiprow Bot commented Sep 19, 2025

Copy link
Copy Markdown

@Leavrth: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@Leavrth
skip if the start key is larger than or equal to the end key
4bbd6fd 
Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>
@Leavrth Leavrth added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Sep 19, 2025
@ti-chi-bot ti-chi-bot Bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Sep 19, 2025
3pointer
3pointer reviewed Sep 19, 2025
View reviewed changes
Comment thread br/pkg/restore/snap_client/tikv_sender.go
Start: startKey,
End: endKey,
},
TtlSeconds: 7200,

@3pointer 3pointer Sep 19, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what will happen if ttl expired(2 hour)?

@3pointer 3pointer Sep 19, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can ingest enough sst during 2hour, seems no problem.

3pointer
3pointer approved these changes Sep 19, 2025
View reviewed changes

@3pointer 3pointer left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 19, 2025
@ti-chi-bot

ti-chi-bot Bot commented Sep 19, 2025

Copy link
Copy Markdown

[LGTM Timeline notifier]

Timeline:

  • 2025-09-19 08:42:35.337240043 +0000 UTC m=+565.407733726: ☑️ agreed by YuJuncen.
  • 2025-09-19 10:07:51.76164464 +0000 UTC m=+5681.832138321: ☑️ agreed by 3pointer.

@BornChanger

BornChanger commented Sep 19, 2025

Copy link
Copy Markdown
Contributor

/retest

@tiprow

tiprow Bot commented Sep 19, 2025

Copy link
Copy Markdown

@BornChanger: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@BornChanger

BornChanger commented Sep 19, 2025

Copy link
Copy Markdown
Contributor

/ok-to-test

@ti-chi-bot ti-chi-bot Bot added the ok-to-test Indicates a PR is ready to be tested. label Sep 19, 2025
@BornChanger

BornChanger commented Sep 20, 2025

Copy link
Copy Markdown
Contributor

/cc @yudongusa

@ti-chi-bot

ti-chi-bot Bot commented Sep 20, 2025

Copy link
Copy Markdown

@BornChanger: GitHub didn't allow me to request PR reviews from the following users: yudongusa.

Note that only pingcap members and repo collaborators can review this PR, and authors cannot review their own PRs.

Details

In response to this:

/cc @yudongusa

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot

ti-chi-bot Bot commented Sep 20, 2025

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 3pointer, bb7133, BornChanger, yudongusa, YuJuncen

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [3pointer,BornChanger,YuJuncen,bb7133]
  • br/OWNERS [3pointer,BornChanger,YuJuncen]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the approved label Sep 20, 2025
@Leavrth
Merge branch 'master' into check_and_compact_for_br
cd50af4
@Leavrth
debug tikv
8f402a0 
Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>
@Leavrth
debug tikv
9f95cd7 
Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>
@Leavrth
debug tikv
427fb30 
Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>
@Leavrth
debug tikv
d5e88e0 
Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>
@Leavrth

Leavrth commented Sep 22, 2025

Copy link
Copy Markdown
Contributor Author

/retest

2 similar comments
@Leavrth

Leavrth commented Sep 22, 2025

Copy link
Copy Markdown
Contributor Author

/retest

@0xPoe

0xPoe commented Sep 22, 2025

Copy link
Copy Markdown
Member

/retest

@Leavrth Leavrth changed the title br: check and compact for br br: check and compact for br | tikv=release-9.0-beta.2 Sep 22, 2025
@D3Hunter D3Hunter mentioned this pull request Sep 22, 2025
Merged
13 tasks
@Leavrth

Leavrth commented Sep 22, 2025

Copy link
Copy Markdown
Contributor Author

/retest

@Leavrth Leavrth changed the title br: check and compact for br | tikv=release-9.0-beta.2 br: check and compact for br | tikv=release-8.5 | tidb=release-8.5 | pd=release-8.5 Sep 22, 2025
@Leavrth

Leavrth commented Sep 22, 2025

Copy link
Copy Markdown
Contributor Author

/retest

@Leavrth Leavrth changed the title br: check and compact for br | tikv=release-8.5 | tidb=release-8.5 | pd=release-8.5 br: check and compact for br | | tikv-test=release-8.5.2 Sep 22, 2025
@Leavrth

Leavrth commented Sep 22, 2025

Copy link
Copy Markdown
Contributor Author

/retest

@Leavrth Leavrth changed the title br: check and compact for br | | tikv-test=release-8.5.2 br: check and compact for br | tikv-test=release-8.5.2 Sep 22, 2025
@Leavrth

Leavrth commented Sep 22, 2025

Copy link
Copy Markdown
Contributor Author

/retest

@Leavrth Leavrth changed the title br: check and compact for br | tikv-test=release-8.5.2 br: check and compact for br Sep 22, 2025
@Leavrth

Leavrth commented Sep 22, 2025

Copy link
Copy Markdown
Contributor Author

/retest

1 similar comment
@BornChanger

BornChanger commented Sep 22, 2025

Copy link
Copy Markdown
Contributor

/retest

@ti-chi-bot ti-chi-bot Bot merged commit 724cd2f into pingcap:master Sep 22, 2025
29 checks passed
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Sep 22, 2025
@Leavrth @ti-chi-bot
This is an automated cherry-pick of pingcap#63419
753882c 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot

ti-chi-bot commented Sep 22, 2025

Copy link
Copy Markdown
Member

In response to a cherrypick label: new pull request created to branch release-8.5: #63667.
But this PR has conflicts, please resolve them!

@ti-chi-bot ti-chi-bot mentioned this pull request Sep 22, 2025
Merged
13 tasks
ti-chi-bot Bot pushed a commit that referenced this pull request Sep 23, 2025
@ti-chi-bot
br: check and compact for br (#63419) (#63667)
24be24d 
close #63176
Leavrth added a commit to Leavrth/tidb that referenced this pull request Dec 9, 2025
@Leavrth
Revert "br: check and compact for br (pingcap#63419) (pingcap#63667)"
95ae036 
This reverts commit 24be24d.
Leavrth added a commit to Leavrth/tidb that referenced this pull request Dec 12, 2025
@Leavrth
br: check and compact for br (pingcap#63419)
82150c6 
close pingcap#63176
Leavrth added a commit to Leavrth/tidb that referenced this pull request Dec 16, 2025
@Leavrth
br: check and compact for br (pingcap#63419)
74babac 
close pingcap#63176
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bb7133 bb7133 bb7133 approved these changes
@3pointer 3pointer 3pointer approved these changes
@YuJuncen YuJuncen YuJuncen approved these changes
@BornChanger BornChanger BornChanger approved these changes
+1 more reviewer
@yudongusa yudongusa yudongusa approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved lgtm needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. ok-to-test Indicates a PR is ready to be tested. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

BR supports to split the large overlapping SSTs before ingest SSTs

8 participants

@Leavrth @BornChanger @0xPoe @ti-chi-bot @bb7133 @3pointer @YuJuncen @yudongusa