Skip to content

[8.3] Avoid capturing SnapshotsInProgress$Entry in queue (#88707) #88724

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 22, 2022
Merged

[8.3] Avoid capturing SnapshotsInProgress$Entry in queue (#88707) #88724

merged 1 commit into from
Jul 22, 2022

Conversation

DaveCTurner
Copy link
Contributor

Backports the following commits to 8.3:

@DaveCTurner
Avoid capturing SnapshotsInProgress$Entry in queue (elastic#88707)
b46ac10 
Today each time there's shards to snapshot we enqueue a lambda which
captures the current `SnapshotsInProgress$Entry`. This is a pretty
heavyweight object, possibly several MB in size, most of which is not
necessary to capture, and with concurrent snapshots across thousands of
shards we may enqueue many hundreds of slightly different such objects.
With this commit we compute a more efficient representation of the work
to be done by each task in the queue instead.

Relates elastic#77466
@DaveCTurner DaveCTurner added :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >bug auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. labels Jul 22, 2022
@elasticsearchmachine elasticsearchmachine mentioned this pull request Jul 22, 2022
Merged
@elasticsearchmachine elasticsearchmachine merged commit 8c118e1 into elastic:8.3 Jul 22, 2022
@DaveCTurner DaveCTurner deleted the backport/8.3/pr-88707 branch July 22, 2022 13:40
@mark-vieira mark-vieira added v8.3.3 and removed v8.3.4 labels Jul 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport >bug :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. v8.3.3
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@DaveCTurner @mark-vieira @elasticsearchmachine