Skip to content
This repository has been archived by the owner on Oct 18, 2023. It is now read-only.

ignore remote change counters on database restore if local db file exists #718

Merged
merged 1 commit into from
Oct 3, 2023

Conversation

Horusiath
Copy link
Contributor

This PR skips process of validating local and remote change counters between local database file and remote snapshot if we found a DB file. In such case it will always call for a new snapshot.

Concerns

  1. This means that every time machine is restarted, a new generation will be created - we cannot reuse them if we're going to ignore the remote counters.
  2. @psarna what if during restoration process we had a non empty WAL besides db file? Can we assume that xFrames callback will happen and that we'll pass info about frames to be send to S3 before possible checkpoint?

@Horusiath Horusiath requested review from haaawk and psarna October 3, 2023 06:17
@haaawk
Copy link
Collaborator

haaawk commented Oct 3, 2023

Sounds reasonable to me. Just to confirm, old generations will still be present on S3 and we will be able to use them if it turns out the local data is corrupted, right?

@psarna
Copy link
Contributor

psarna commented Oct 3, 2023

@psarna what if during restoration process we had a non empty WAL besides db file?
let's checkpoint that before we create a new generation. It's part of the boot process, so there are no writers yet and we can just checkpoint the WAL and have a proper checkpointed db to snapshot and start a new generation from

@Horusiath
Copy link
Contributor Author

Horusiath commented Oct 3, 2023

I've checked that we have this scenario covered here: https://github.com/libsql/sqld/blob/55a5c641abccf25b4176f0d47925081ffe65fdab/bottomless/src/replicator.rs#L671C31-L671C31 At the end of snapshot db request we also try to replicate WAL if it has been found.

@Horusiath Horusiath marked this pull request as ready for review October 3, 2023 06:50
@penberg penberg added this pull request to the merge queue Oct 3, 2023
Merged via the queue into main with commit 2346a2b Oct 3, 2023
9 checks passed
@penberg penberg deleted the bottomless-always-accept-local-db branch October 3, 2023 07:36
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants