Handle snapshot restore more robustly #2735
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It is possible that a snapshot is incomplete, even though its lock file was removed. When listing available snapshots, ignore those without a metadata file.
Mirroring `max_committed_offset`.
Similar to `Iterator::size_hint`, `History` gains `tx_range_hint` that returns both a lower and (optional) upper bound.
There is always a possibility that a snapshot is somehow invalid or corrupted. When starting up a `RelationalDB`, don't fail immediately if that is the case, but instead try older snapshots if the available commitlog permits replaying.
kim
changed the title
Shubham8287
reviewed
May 15, 2025
Co-authored-by: Shubham Mishra <shubham@clockworklabs.io> Signed-off-by: Kim Altintop <kim@eagain.io>
Shubham8287
reviewed
May 15, 2025
Signed-off-by: Shubham Mishra <shivam828787@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Snapshots may be corrupted on disk or incomplete where the lock file is gone,
but the snapshot file was not written (it is written last after all objects).
This patch handles both cases by ignoring snapshots without a snapshot file when
listing the available snapshots, and trying older snapshots if the latest one
cannot be restored from (for any reason).
It also takes into account that the prefix of the local commitlog could have
been archived.
API and ABI breaking changes
No
Expected complexity level and risk
2
Testing