Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TiFlash OOM with high throughput write #6407

Closed
flowbehappy opened this issue Dec 2, 2022 · 1 comment · Fixed by #6446
Closed

TiFlash OOM with high throughput write #6407

flowbehappy opened this issue Dec 2, 2022 · 1 comment · Fixed by #6446

Comments

@flowbehappy
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

  1. A cluster with many TiKV and 4 TiFlash
  2. A table with shard_row_id_bits
  3. Each TiFlash instance has over 300GB
  4. The GC interval is set to 5 days
  5. Write throughput of each TiFlash is over 7MB/s

2. What did you expect to see? (Required)

TiFlash works normally.

3. What did you see instead (Required)

TiFlash OOM.
image
image

4. What is your TiFlash version? (Required)

v6.1.x

@flowbehappy
Copy link
Contributor Author

It is very likely caused by delta compact works slowly. And the root cause is that ~Snapshot of Storage V2 is slow in the case of many page entries.
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment