Storage Component For Simplex

[samliok]

Why this should be merged

This PR introduces the storage component for Simplex. The three functions of the simplex.Storage interface are

• Index which indexes a block and finalization into the storage. Because indexing a block is already handled when a block is accepted, all this component needs to do is index the finalization.
• Retrieve grabs the block and finalization from storage
• Height tells simplex how many blocks have been indexed. The next sequence to commit is always equal to the height.

How this works

How this was tested

Need to be documented in RELEASES.md?

[Copilot]

Pull Request Overview

This PR introduces a storage component for the Simplex consensus protocol that provides block indexing and retrieval functionality. The storage component maintains a sequential chain of blocks with their associated finalizations, backed by a database for persistence.

Key changes include:

• Introduction of a new Storage struct that implements simplex.Storage interface for block management
• Addition of database support to the Config struct for storage operations
• Modification of test utilities to support the new storage architecture with wrapped blocks

Reviewed Changes

Copilot reviewed 6 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
simplex/storage.go	Core storage implementation with block indexing, retrieval, and finalization management
simplex/storage_test.go	Comprehensive test suite covering storage creation, block indexing, and retrieval scenarios
simplex/util_test.go	Updated test utilities with database support and wrapped block implementation for testing
simplex/config.go	Added VM and database fields to Config struct
simplex/block_test.go	Updated existing tests to work with wrapped block structure
go.mod	Updated simplex dependency version

[StephenButtolph]

This is a pretty confusing name. height being equal to the lastAcceptedHeight+1. Could we either:

• align the two (have it equal lastAcceptedHeight and perhaps name it that)
• change the name of this variable to better convey what it is?

[yacovm]

It's simply the height of the chain. It's fairly a common term, we use it extensively in Simplex and it's always the next sequence to commit (e.g if the genesis block is sequence 0, then the first block to commit will have the index of 1, as it is committed when the chain is at height of 1).

It's not equal to the last accepted height + 1, but equal to the last accepted sequence + 1.

[StephenButtolph]

It's not equal to the last accepted height + 1, but equal to the last accepted sequence + 1.

Well - it is currently set to the last accepted height + 1 not the last accepted sequence + 1.

s.height.Store(lastAcceptedBlock.Height() + 1)

no?

[samliok]

i thought block heights should always match to sequences

[yacovm]

Well - it is currently set to the last accepted height + 1 not the last accepted sequence + 1.

If we only have the genesis block, then the height of the chain should be 1, no?

[StephenButtolph]

Perhaps to just make it very clear, we can call this variable nextHeight?

Suggested change

	// height represents the number of blocks indexed in storage.
	height atomic.Uint64
	// nextHeight is the height of the next block to index.
	nextHeight atomic.Uint64

Or we could name it blocksIndexed? I think length could also make sense.

From everything I have seen, the height of a blockchain is equal to the monotonically increasing number included in the last block. (For ethereum they call this the block number, in avalanche we call it the block height).

So, the height (from what I can see) is defined as the number of blocks since genesis. The height of a chain with only the genesis block would be 0.

[yacovm]

To me, blocksIndexed is the definition of height.

From everything I have seen, the height of a blockchain is equal to the monotonically increasing number included in the last block. (For ethereum they call this the block number, in avalanche we call it the block height).

There is the problem - the number in the block cannot be the height, but should be the number (what in Simplex we call the sequence).

In my opinion, a height is something that is defined outside of the block, based on the blocks in the ledger:

https://www.investopedia.com/terms/b/block-height.asp

The block height of a particular block is defined as the number of blocks preceding it in the blockchain.

So, the height (from what I can see) is defined as the number of blocks since genesis. The height of a chain with only the genesis block would be 0.

I think we should treat the genesis block as a regular block when counting the number of blocks, and therefore a chain with only the genesis block, is of height 1.

[samliok]

I am fine with either definition, I will keep it as height for now

.

[yacovm]

If we crash in between, is there any recovery logic needed? Do we need to roll-forward the VM and call indexBlock?

finalizationBytes := finalizationToBytes(finalization)
	if err := s.db.Put(seqBuff, finalizationBytes); err != nil {
		return fmt.Errorf("failed to store finalization: %w", err)
	}
    <-------------------------
	err := s.blockTracker.indexBlock(ctx, bh.Digest)
	if err != nil {
		return fmt.Errorf("failed to index block: %w", err)
	}

[samliok]

so if we put seq x in the database and don't call accept, then when we startup we will set the height to the last accepted block(which is x-1). The next time we index, we will just overwrite the finalization x with a new x'

However, i think we need to have a sanity check in retrieve to ensure we do not try and retrieve a block >= height

[yacovm]

then when we startup we will set the height to the last accepted block(which is x-1)

The reason is that we rely on the VM to find out what is the latest block and not on the finalization DB, right?

However, i think we need to have a sanity check in retrieve to ensure we do not try and retrieve a block >= height

Why? won't we just return database.ErrNotFound ?

[samliok]

Yep correct on both questions

[yacovm]

LGTM modulo two questions above

[StephenButtolph]

2 trivial comments then lgtm