Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable write-ahead log, recover in-progress TXes at startup. #2518

Merged
merged 11 commits into from
Jun 16, 2022

Conversation

mystenmark
Copy link
Contributor

@mystenmark mystenmark commented Jun 11, 2022

A few items are still TODO:

  • Tests (coming soon, want to make sure there's consensus on the approach before I work on this)
  • Offline retries - right now we only process the log at startup. The main motivation is for crash recovery, so that's ok. But we'd also like to attempt recovery periodically while running in case txes fail transiently due to db write errors.
  • After tests are done I will:
    • Remove object critical section.
    • Break up the big atomic batch write for better performance.

@mystenmark mystenmark force-pushed the mlogan-storage branch 2 times, most recently from da148b1 to b9bfabb Compare June 11, 2022 03:00
Copy link
Collaborator

@gdanezis gdanezis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one -- I have a few questions around the "pop" in the recovery logic; the conditions under which we might retry; as well as deleting from the retry table.

crates/sui-core/src/authority.rs Outdated Show resolved Hide resolved
crates/sui-core/src/authority.rs Show resolved Hide resolved
crates/sui-core/src/authority.rs Show resolved Hide resolved
crates/sui-core/src/authority.rs Show resolved Hide resolved
crates/sui-core/src/authority.rs Show resolved Hide resolved
crates/sui-core/src/authority.rs Outdated Show resolved Hide resolved
crates/sui-core/src/authority.rs Outdated Show resolved Hide resolved
crates/sui-core/src/authority.rs Show resolved Hide resolved
crates/sui-storage/src/write_ahead_log.rs Show resolved Hide resolved
crates/sui-storage/src/write_ahead_log.rs Show resolved Hide resolved
@mystenmark
Copy link
Contributor Author

@gdanezis thanks for the excellent comments, will address the ones that require edits!

Copy link
Contributor

@velvia velvia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left lots of comments.... good work @mystenmark

crates/sui-core/src/authority.rs Outdated Show resolved Hide resolved
crates/sui-core/src/authority.rs Show resolved Hide resolved
crates/sui-core/src/authority.rs Outdated Show resolved Hide resolved
crates/sui-core/src/authority.rs Show resolved Hide resolved
crates/sui-core/src/authority.rs Show resolved Hide resolved
crates/sui-core/src/authority.rs Outdated Show resolved Hide resolved
crates/sui-core/src/authority.rs Outdated Show resolved Hide resolved
crates/sui-core/src/authority.rs Show resolved Hide resolved
crates/sui-core/src/authority/authority_store.rs Outdated Show resolved Hide resolved
crates/sui-storage/src/write_ahead_log.rs Show resolved Hide resolved
@mystenmark mystenmark enabled auto-merge (squash) June 16, 2022 16:09
@mystenmark mystenmark merged commit 6f33388 into main Jun 16, 2022
@mystenmark mystenmark deleted the mlogan-storage branch June 16, 2022 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants