Skip to content

[feature] Offline generation of Parity Warp Sync Snapshots #21186

Closed
@riptl

Description

@riptl

Request: Add a geth warp subcommand that generates a Parity-compatible "v2" snapshot of the last N blocks and a single state trie, while the node is offline.

The Warp Sync Snapshots used by OpenEthereum are fairly simple to generate, ignoring parallelization for now.
https://openethereum.github.io/wiki/Warp-Sync-Snapshot-Format

The major implementation steps:

  1. Collect blocks with uncles and receipts and PoW metadata.
  2. Iterate the accounts trie, save accounts, code and the storage trie.
  3. A set of writers that do chunking, compression and RLP streaming.
  4. Port OpenEthereum tests.

This issue does not cover

  • building the snapshots during node operation (overlap with Dynamic State Snapshots)
  • "seeding" the snapshots via P2P
  • restoration of warp snapshots

I'm fairly sure that no changes to the other packages (eth/core, trie) are required. The only external dependency, snappy compression, is already pulled in.

I'd like to work out my proof of concept into a PR if this feature has any chance getting merged.
If not (maybe GPLv3 vs LGPLv3 licensing issues?), I'm going to keep it out-of-tree instead.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions