Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Take more than 5 hours to merge empty region after drop 6T data #12421

Closed
mayjiang0203 opened this issue Apr 24, 2022 · 4 comments · Fixed by #13384
Closed

Take more than 5 hours to merge empty region after drop 6T data #12421

mayjiang0203 opened this issue Apr 24, 2022 · 4 comments · Fixed by #13384
Assignees
@tabokie
Labels
affects-5.4 affects-6.1 severity/major type/bug The issue is confirmed as a bug.

Comments

@mayjiang0203
Copy link

mayjiang0203 commented Apr 24, 2022 •
edited
Loading

Bug Report

What version of TiKV are you using?

[2022/04/23 23:53:15.874 +08:00] [INFO] [client.go:376] ["Cluster version information"] [type=tikv] [version=6.0.0-alpha] [git_hash=ed7e1aa945d9ada9dc3ad3bd6420a9879d49ed44]

What operating system and CPU are you using?

Steps to reproduce

What did you expect?

Since there are only less than 50G data during region merge, should not cost so much time.

What did happened?

image
image
image
@mayjiang0203
Copy link
Author

/type bug
/severity major

@tabokie
Copy link
Member

tabokie commented Apr 24, 2022

Flamegraph shows TiKV is blocked on the RocksDB seeks in clean_stale_ranges, which is very possibly caused by tombstones.

image

Possible optimizations:

  • Set the iter upper bound tighter in delete_all_in_range_cf_by_ingest (don't use the max_end_key)
  • Use other deletion methods like delete_range when max_skippable_internal_keys is too high

@Lily2025
Copy link

/assign tabokie

@mayjiang0203 mayjiang0203 changed the title Take more than 5 hours to merge import region after drop 6T data Take more than 5 hours to merge empty region after drop 6T data May 20, 2022
@tabokie tabokie mentioned this issue Aug 31, 2022
Merged
@AkiraXie
Copy link

/label affects-5.4

ti-chi-bot pushed a commit that referenced this issue Sep 6, 2022
@tabokie
raftstore: optimize region destroy (#13384)
cbf85c1 
close #12421

Optimize the performance of merging empty regions

Signed-off-by: tabokie <xy.tao@outlook.com>
@ti-srebot ti-srebot mentioned this issue Sep 6, 2022
Closed
This was referenced Sep 6, 2022
Closed
Closed
tabokie added a commit to tabokie/tikv that referenced this issue Sep 6, 2022
@tabokie
raftstore: optimize region destroy (tikv#13384)
a3680a9 
close tikv#12421

Optimize the performance of merging empty regions

Signed-off-by: tabokie <xy.tao@outlook.com>
@tabokie tabokie mentioned this issue Sep 6, 2022
Closed
tabokie added a commit to tabokie/tikv that referenced this issue Sep 6, 2022
@tabokie
raftstore: optimize region destroy (tikv#13384)
fe164b6 
close tikv#12421

Optimize the performance of merging empty regions

Signed-off-by: tabokie <xy.tao@outlook.com>
@tabokie tabokie mentioned this issue Sep 6, 2022
Closed
tabokie added a commit to tabokie/tikv that referenced this issue Sep 6, 2022
@tabokie
raftstore: optimize region destroy (tikv#13384)
4bdf5f3 
close tikv#12421

Optimize the performance of merging empty regions

Signed-off-by: tabokie <xy.tao@outlook.com>
@tabokie tabokie mentioned this issue Sep 6, 2022
Merged
ti-chi-bot bot pushed a commit that referenced this issue Aug 15, 2023
@tabokie @tonyxuqqi
raftstore: optimize region destroy (#13384) (#13420)
0632369 
close #12421

Optimize the performance of merging empty regions

Signed-off-by: tabokie <xy.tao@outlook.com>

Co-authored-by: tonyxuqqi <tonyxuqi@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tabokie tabokie

Labels
affects-5.4 affects-6.1 severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

raftstore: optimize region destroy tabokie/tikv
6 participants
@VelocityLight @mayjiang0203 @tabokie @AkiraXie @ti-chi-bot @Lily2025