Skip to content

Add state checksums #1658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Jun 28, 2023
Merged

Add state checksums #1658

merged 28 commits into from
Jun 28, 2023

Conversation

StephenButtolph
Copy link
Contributor

@StephenButtolph StephenButtolph commented Jun 26, 2023

Why this should be merged

Enables integrity checks to prevent issues such as the v1.9.12 X-chain state regression.

How this works

If enabled (can be enabled on both the P-chain and/or the X-chain independently using chain configs):

  • On startup, iterate over the existing UTXO set and generate an accumulator to cryptographically summarize the UTXOs.
  • Upon the addition or removal of a UTXO from the UTXO set - update the accumulator.
  • Perform this same logic for accepted transactions on the X-chain
  • Upon the acceptance of a block log the checksum(s)

How this was tested

Manually and as part of verifying the X-chain DB pruning.

@StephenButtolph StephenButtolph added the DO NOT MERGE This PR must not be merged in its current state label Jun 26, 2023
@StephenButtolph StephenButtolph changed the base branch from dev to remove-statuses June 27, 2023 22:23
@StephenButtolph StephenButtolph changed the title Maintain X-chain state checksums Add state checksums Jun 28, 2023
Comment on lines +95 to +97
type Config struct {
ChecksumsEnabled bool `json:"checksums-enabled"`
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I felt like now was as good of a time as any to introduce P-chain specific configs. We'll probably use this more in the future for APIs and other VM level configs

@@ -563,6 +579,8 @@ func (s *state) writeTxs() error {
txID := txID
txBytes := tx.Bytes()

s.updateTxChecksum(txID)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We write all txs we are accepting to disk. Even transactions that were already written when they were processing. While the following db.Put will be a noop in this case - the DB will also end up deleting the status entry... Which marks this tx as accepted. Regardless - it is correct to update the checksum here.

Comment on lines +881 to +882
statusIt := s.statusDB.NewIterator()
defer statusIt.Release()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't wait for statuses to be gone 🙏

@@ -176,10 +104,20 @@ func (a *acceptor) ApricotAtomicBlock(b *blocks.ApricotAtomicBlock) error {
err,
)
}

a.ctx.Log.Trace(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I also changed these to Trace so that we could better observe these values (without all the p2p messages)

Comment on lines -37 to -43
a.ctx.Log.Debug(
"accepting block",
zap.String("blockType", "banff abort"),
zap.Stringer("blkID", b.ID()),
zap.Uint64("height", b.Height()),
zap.Stringer("parentID", b.Parent()),
)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like reducing all these super similar logs was in-and-of itself a good change.

@StephenButtolph StephenButtolph added enhancement New feature or request vm This involves virtual machines storage This involves storage primitives stability and removed DO NOT MERGE This PR must not be merged in its current state labels Jun 28, 2023
@StephenButtolph StephenButtolph added this to the v1.10.4 milestone Jun 28, 2023
@StephenButtolph StephenButtolph marked this pull request as ready for review June 28, 2023 03:23
@StephenButtolph StephenButtolph added monitoring This primarily focuses on logs, metrics, and/or tracing and removed stability labels Jun 28, 2023
@StephenButtolph StephenButtolph self-assigned this Jun 28, 2023
Base automatically changed from remove-statuses to dev June 28, 2023 19:59
@StephenButtolph StephenButtolph merged commit c97ae07 into dev Jun 28, 2023
@StephenButtolph StephenButtolph deleted the add-x-chain-checksums branch June 28, 2023 21:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request monitoring This primarily focuses on logs, metrics, and/or tracing storage This involves storage primitives vm This involves virtual machines
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants