feat: Enable Firewood in the EVM

[alarso16]

Why this should be merged

This is the first step to enable Firewood on the C-Chain. This provides all the necessary EVM constructions to function during consensus.

To review, I recommend you start with:

• Following the initialization path for the Firewood triedb using core/state/database.go
• If you're unfamiliar with Firewood, reviewing the interface for the Firewood FFI may be beneficial. Specifically the firewood.go, proposal.go and revision.go files, as well as the tests.

Next steps for integration

• Integrating firewood metrics/logging - see this comment
• E2E testing
• State Sync
• Performance testing to find optimizations/weak points

How this works

This interfaces with the Go FFI for Firewood and hooks into triedb to enable normal operations of the EVM. The operation is very similar to that of HashDB, with a few differences:

• We need a custom Trie implementation (see core/state/database.go in coreth+libevm) since the geth implementation requires tracking intermediate nodes (which is unnecessary with firewood).
  • Because Firewood uses a single tree structure instead of a two-tier structure like HashDB and PathDB, it is unnecessary to produce storage root hashes.
  • We want each storage trie to be strongly tied to the account trie to easily track changes. Creating a proposal is relatively lightweight, so we can retrieve the root on demand.
  • There's several assumptions about how the trie can be used (namely, no changes will be made to if it's a historical root, and it will not be used to retrieve dirty state). These rely on the specific use case within StateDB and the APIs.
  • To use the custom Trie, a wrapper is used to intercept requests for the trie in the state database. It simply checks whether we're using Firewood currently to
  • This is tested in core/state/database_test.go, which compares the hash output to HashDB
• The path in which the firewood database is stored is provided by AvalancheGo, and this may change the way clients and load testers copy the state at any given point (i.e., they need to copy .avalanchego/chainData as well)
  • Unfortunately, to avoid changes to the geth code, I decided to store the database path in ethdb. This is required to be able to transfer the string from vm.go to blockchain.go. This could also be loaded from the chain context, but this is much clearer.
  • Another unique way the ethdb is used is to track genesis. Since HashDB stores this state permanently, it can simply check whether the root is available. Firewood will clean this from disk after X revisions, so we track the genesis root in ethdb, so we can easily check later if using the correct genesis root.

To run a node, you need a custom config:

{
        "state-sync-enabled": false,
        "state-scheme": "firewood",
        "pruning-enabled": true
}

You also need to run go get github.com/ava-labs/coreth@<most-recent-hash> && go mod tidy to prep avalanchego to build. However, if performance is a concern, you should build Firewood locally. The Firewood team is finishing up some more performance improvements before pushing another version.

Heres what still needs done/is a little funky and should be especially reviewed:

• The database path is stored in ethdb so it can be accessed without any additional config changes. This is not my favorite solution, but it's good enough to work for now. This could similarly be retrieved from the chain context, but this would be extremely difficult for tests.
• Proposals are "lazily" dropped, meaning calls from the TrieWriter to dereference a proposal are ignored. This is documented with a comment on the function, but it is due to being unable to safely recognize when a proposal can be dropped. This could be changed to dropping at recognition only if there is a single proposal with the given root, but I think this is likely more efficient.
• There's likely optimization to be made (feel free to point them out), but some of these ideas cannot be tested until there's some benchmarking framework ready.
• The following APIs are currently unavailable. If any of these are critical to be implemented, we can attempt to implement them. However, in general iterating across the FFI barrier must be avoided. In their current form, some will throw errors and others will return garbage results. Before this is used in production, these should be addressed formally.
  • GetModifiedAccountsByNumber
  • GetModifiedAccountsByHash
  • StorageRangeAt
  • eth_getProof
• Firewood does not support creating proposals from historical revisions, as they can never be committed. If this is required for users in APIs, this may be implementable in Firewood, but is not currently supported.
• API actions WILL pollute the Firewood cache.
• AvalancheGo E2E tests are not yet created

How this was tested

Borrowed a simple fuzzing test for Trie usage from Firewood to ensure hash-compatibility and integrated Firewood into all tests outside of APIs. Additionally, currently bootstrapping on mainnet and Fuji (executed 14 million and 8 million blocks, respectively)

Need to be documented?

Yes, but this should wait until it's ready to be used in production.

Need to update RELEASES.md?

No.

[alarso16]

This file was adapted from Firewood's fuzz testing for the ethhasher. This could contain additional databases (e.g. PathDB) if necessary.

[alarso16]

Closed in favor of multiple PRs (specifically #1029 and #1041)