Implement PeerDAS

[nalepae]

Reading:

Peerdas from scratch

Remaining tasks must have:

Features:

• Backfill: Backfill data columns when checkpoint syncing and when validator custody increases (in progress @kasey). Don't forget to update the earliest available slot.
• Initial sync: Implement robust initial sync.
• Web3Signer: support fulu types

Misc:

• Grandine (not PeerDAS related): See why Prysm cannot peer with Grandine nodes.
• Metrics: Add PeerDAS specific metrics (In progress @KatyaRyazantseva)
• Peer downscoring: Uncomment / modify peer downscoring when a peer requests too many data columns.
• Peer downscoring: Downscore properly peers when responding invalid data columns sidecars by RPC.
• Spectests: Implement all.
• GetBlobsV2: Try as soon as we have a block or a column, and retry periodically (every 200 ms?) until we have what we want, by any way possible. And also call it as soon as we receive the first data column sidecar or block.
• Attestations subnets (Not peerdas related): Check why we are subscribed to so many attestations subnets. (It may be normal.)
• Metadata: Check why in devnets we have Metadata is nil, defaulting to the ENR value.
• Blob limit updated: Understand / fix the spam of logs and its root cause.
• Reconstruction: Delay reconstruction instead reconstructing immediately and delay reseed.
• Earliest available slot: Update when pruning.
• maybe getBlobs endpoint Add getBlobs endpoint ethereum/beacon-APIs#546 Get blob fulu #15610
• update E2E for basic Fulu testing

Remaining tasks nice to have have:

• KZG library: Replace the cKZG library by the goKZG one. (In progress @nalepae @kevaundray)
• Per root/range data columns request: Implement a rescue mode where trying, for full nodes, if no way to retrieve needed columns, to retrieve any of 64 columns then reconstruct the rest.
• prysmctl: Implement byRoot and byRange data columns requests.
• Chaos testing: Indroduce a nasty peer flag that holds AND/OR (both are important) alterate data columns on RPC responses. (In progress 0xPC02)
• ENR (not peerDAS related) Find why sometimes ENR records are not correctly updated via discovery.
• Trusted setup: Add a unit test to pull the upstream JSON and compare it against ours to ensure there’s no discrepancy.
• Blob sidecars beacon API: If 64 <= cgc < 128 and requiring not all the blobs, avoid reconstructing all of them (while respecting the VerifiedRoDataColumnSidecar paradigm.
• KZG: Avoid the power switch.
• Verification: Implement tests in a different package.
• Blocks: Implement tests in a different package.
• Libp2p (Not peerDAS related): Remove all the libp2p fails without this delay comments in tests.
• ENR seq bootnode (Not peerDAS related) ensure that ENR seq never goes backward on bootnode
• SSZ by default for validator client for expensive endpoints (Not PeerDAS related), particularly the get and submit block endpoints. get is done
• Block fetcher: Use blockFetcherConfig directly in blockFetcher
• Rate limiter: Carefully check that the value of rateLimiter.Remaining is non-negative before converting to uint64.
• ROBlock: In s.pendingBlobsRequestForBlock, use ROBlock instead signed block + root all the way down.

Remaining open questions:

• Subnet unsubscription (not peerDAS related): Handle the same way dynamic (fork epoch) and non dynamic (fork epoch + 1) unsubscriptions.
• Peer ban: What happens if a peer announces less custody groups via Metadata/ENR than the minimal requirement? Ban?