PeerDAS - Tracking Issue

[AgeManning]

Description

This is a tracking issue for the work that is being undertaken on building out Data Availability Sampling.

This will be updated and edited as we go.

Instructions for running a DAS devnet can be found in this gist.

Main PR to unstable:

• PeerDAS implementation #5683

Devnet 5

• Implement getBlobSidecars support for PeerDAS #6251
• Improve range sync with PeerDAS #6258
• Reconstruct data columns without blocking processing and import #5990
• Improving blob propagation post-PeerDAS with Decentralized Blob Building #6268
• Subscribe to column subnets from PeerDAS (Fulu) epoch #5894
• Verify kzg_commitments_inclusion_proof on data column by range responses #6111
• Add additional PeerDAS metrics #6018
• Add option to select KZG library for PeerDAS #6107
• Implement validator custody (spec) Implement basic validator data column custody #6767
• Unsubscribe from blob_sidecars gossip topic from the Fulu fork #6854
• Address all TODO(das) comments
• Batch verify KZG proofs where possible Use data column batch verification consistently #6851
• Don't lookup sync individual columns received from gossip. WIP dapplion:dedup-columns-by-root-requests

TODO(das)

• Collection of small / easy TODOs Fix misc PeerDAS todos #6862
• Generalize block available time metric for PeerDAS #6850

Initial implementation

Data format and distribution (#5050)

• Add ColumnSidecar type
• Extend blobs horizontally with dummy data (0x{random_uniq_bits}...{col_id}) and distribute to column subnets
• Create 32 column subnets NUM_COLUMN_SUBNETS
• Compute subnets to subscribe to based on node_id (see att example here)
• Node to subscribe to only CUSTODY_REQUIREMENT column subnets

Data Column Custody

• Add DataColumnSidecarsByRoot req/resp protocol #5196
• Advertise extra custodied columns through ENR (@jacobkaufmann)
• Implement PEERDAS_ACTIVATION_EPOCH to transition from deneb blobs to data columns (@jimmygchen)
• Send custody data column to DataAvailabilityChecker for determining block importability #5570
• PeerDAS custody lookup sync #5684
• ❌ Proposer to store all data columns to prevent data loss (See this comment) - increase peer count and custody requirement instead

Sampling

• Add DataColumnSidecarsByRoot req/resp protocol #5196
• DAS sampling on sync #5616
• Re-process early sampling requests #5569
• Don't attempt downloading sample for custody columns already in DB
• Respond with first byte to extend timeout for early sampling requests #6929
• Network service update to maintain peers on all custody columns #6930

Syncing

• DataColumnByRange boilerplate #5353
• Fetch custody columns in range sync #5747
• Trigger sampling on sync events #5776

Misc

• feat: add DAS KZG in data col construction #5210
• Add data column kzg verification and update c-kzg. #5701
• ❌ Add EIP-7594 boilerplate  #5734
• PeerDAS spec tests #5772
• Rename PEER_DAS_EPOCH to EIP7594_FORK_EPOCH #5750
• Implement unconditional reconstruction for supernodes #5781
• Add column gossip verification and handle unknown parent block #5783
• PeerDAS parameter changes for devnet-0 #5779
• Multithread compute_cells_and_proofs #5805
• Move PeerDAS type-level config to configurable ChainSpec #5828
• Store data columns individually in store and caches #5890

peerdas-devnet-1

• Update custody subnet count format in ENR #5950

peerdas-devnet-2

• Implement MetaDataV3 with cusotdy_subnet_count #6233
• Update PeerDAS network parameters for peerdas-devnet-2 #6289

peerdas-devnet-3

• Progressively merge das to unstable #6072
• Make Checkpoint sync work for PeerDAS #6026
• Extend simulator tests to support PeerDAS #6109

Post-Pectra (Sampling, 2D PeerDAS etc)

• Update fork choice with sampling results (spec)
• Allow retrying sampling of a data column on a peer after certain amount of time #6110

Current behaviour not specified by consensus-specs

• Only supernode performs reconsturctions once >50% of columns received
• LossyDAS implementation
• Re-process DataColumnsByRoot requests and respond when columns are ready