Skip to content
This repository was archived by the owner on Oct 18, 2023. It is now read-only.

Conversation

@Horusiath
Copy link
Contributor

This solves the issue when SQLite files (DB + WAL) are being modified before we confirm that they were backed up.

PS: don't merge yet, there's still an ongoing issue with tokio task never being scheduled causing a deadlock on subsequent checkpoints:

let handle = tokio::spawn(async move {
let start = Instant::now();
let body = match Self::maybe_compress_main_db_file(db_file, compression).await {
Ok(file) => file,
Err(e) => {
tracing::error!(
"Failed to compress db file (generation {}): {}",
generation,
e
);
let _ = snapshot_notifier.send(Err(e));
return;
}
};
let mut result = snapshot_req.body(body).send().await;
if let Err(e) = result {
tracing::error!(
"Failed to upload snapshot for generation {}: {:?}",
generation,
e
);
let _ = snapshot_notifier.send(Err(e.into()));
return;
}
result = change_counter_req.send().await;
if let Err(e) = result {
tracing::error!(
"Failed to upload change counter for generation {}: {:?}",
generation,
e
);
let _ = snapshot_notifier.send(Err(e.into()));
return;
}
let _ = snapshot_notifier.send(Ok(Some(generation)));
let elapsed = Instant::now() - start;
tracing::debug!("Snapshot upload finished (took {:?})", elapsed);
let _ = tokio::fs::remove_file(format!("db.{}", compression)).await;
});

@MarinPostma MarinPostma force-pushed the bottomless-flush-frames-before-checkpoint branch from c33b15c to ac9f0d5 Compare September 26, 2023 15:55
@Horusiath Horusiath marked this pull request as ready for review September 26, 2023 16:04
@Horusiath Horusiath added this pull request to the merge queue Sep 26, 2023
Merged via the queue into main with commit 0a42e6e Sep 26, 2023
@Horusiath Horusiath deleted the bottomless-flush-frames-before-checkpoint branch September 26, 2023 16:48
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants