Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Snapshot deletion is slow #2156

Open
piyushdaftary opened this issue Feb 17, 2022 · 0 comments
Open

[BUG] Snapshot deletion is slow #2156

piyushdaftary opened this issue Feb 17, 2022 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@piyushdaftary
Copy link
Contributor

Describe the bug
Currently during snapshot delete, deletion of unlinked shard level blob is single threaded using SNAPSHOT threadpool. Hence if there is huge number of unlinked shard level blob flies, it will take considerable amount of time to clean them.

Hence I propose to make unlinked shard level blob deletion multi threaded delete the same way we do for cleaning up of stale indices, to speedup the overall snapshot deletion process.

To Reproduce
Steps to reproduce the behavior:

  1. Create an Index index1
  2. ingest documents to index1
  3. Create a snapshot (lets call it snapshot1)
  4. Wait for snapshot snap1 to compete
  5. ingest lots of document to index1
  6. Create a new snapshot (lets call it snapshot2)
  7. Wait for snapshot snap2 to compete
  8. delete snap2
  9. You will see deletion of unlinked shard level blobs are getting deleted in only one thread even though snapshot thread pool have lots of thread available

Expected behavior
Deletion of unlinked shard level blobs should be multithreaded to speedup overall snapshot deletion.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS] CentOs
  • Version [e.g. 22] 8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants