From 65d3c6aeb68ecd5b168929ccedc877bad197988e Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Tue, 7 Feb 2023 10:55:51 +0100 Subject: [PATCH 01/22] Free the blobs This PR reintroduces and further decouples blocks and blobs in EIP-4844, so as to improve network and processing performance. Block and blob processing, for the purpose of gossip validation, are independent: they can both be propagated and gossip-validated in parallel - the decoupled design allows 4 important optimizations (or, if you are so inclined, removes 4 unnecessary pessimizations): * Blocks and blobs travel on independent meshes allowing for better parallelization and utilization of high-bandwidth peers * Re-broadcasting after validation can start earlier allowing more efficient use of upload bandwidth - blocks for example can be rebroadcast to peers while blobs are still being downloaded * bandwidth-reduction techniques such as per-peer deduplication are more efficient because of the smaller message size * gossip verification happens independently for blocks and blobs, allowing better sharing / use of CPU and I/O resources in clients With growing block sizes and additional blob data to stream, the network streaming time becomes a dominant factor in propagation times - on a 100mbit line, streaming 1mb to 8 peers takes ~1s - this process is repeated for each hop in both incoming and outgoing directions. This design in particular sends each blob on a separate subnet, thus maximising the potential for parallelisation and providing a natural path for growing the number of blobs per block should the network be judged to be able to handle it. Changes compared to the current design include: * `BlobsSidecar` is split into individual `BlobSidecar` containers - each container is signed individually by the proposer * the signature is used during gossip validation but later dropped. * KZG commitment verification is moved out of the gossip pipeline and instead done before fork choice addition, when both block and sidecars have arrived * clients may verify individual blob commitments earlier * more generally and similar to block verification, gossip propagation is performed solely based on trivial consistency checks and proposer signature verification * by-root blob requests are done per-blob, so as to retain the ability to fill in blobs one-by-one assuming clients generally receive blobs from gossip * by-range blob requests are done per-block, so as to simplify historical sync * range and root requests are limited to `128` entries for both blocks and blobs - practically, the current higher limit of `1024` for blocks does not get used and keeping the limits consistent simplifies implementation - with the merge, block sizes have grown significantly and clients generally fetch smaller chunks. --- specs/eip4844/beacon-chain.md | 5 +- specs/eip4844/fork-choice.md | 39 +++------ specs/eip4844/p2p-interface.md | 147 ++++++++++++++++++++------------- specs/eip4844/validator.md | 33 ++++---- 4 files changed, 124 insertions(+), 100 deletions(-) diff --git a/specs/eip4844/beacon-chain.md b/specs/eip4844/beacon-chain.md index f681ab951e..145710a890 100644 --- a/specs/eip4844/beacon-chain.md +++ b/specs/eip4844/beacon-chain.md @@ -44,6 +44,7 @@ This upgrade adds blobs to the beacon chain as part of EIP-4844. This is an exte | Name | SSZ equivalent | Description | | - | - | - | | `VersionedHash` | `Bytes32` | | +| `BlobIndex` | `uint64` | | ## Constants @@ -52,7 +53,7 @@ This upgrade adds blobs to the beacon chain as part of EIP-4844. This is an exte | Name | Value | | - | - | | `BLOB_TX_TYPE` | `uint8(0x05)` | -| `VERSIONED_HASH_VERSION_KZG` | `Bytes1('0x01')` | +| `VERSIONED_HASH_VERSION_KZG` | `Bytes1('0x01')` | ## Preset @@ -249,7 +250,7 @@ def process_blob_kzg_commitments(state: BeaconState, body: BeaconBlockBody) -> N *Note*: The function `initialize_beacon_state_from_eth1` is modified for pure EIP-4844 testing only. -The `BeaconState` initialization is unchanged, except for the use of the updated `eip4844.BeaconBlockBody` type +The `BeaconState` initialization is unchanged, except for the use of the updated `eip4844.BeaconBlockBody` type when initializing the first body-root. ```python diff --git a/specs/eip4844/fork-choice.md b/specs/eip4844/fork-choice.md index 8dea28dedc..3e909423e9 100644 --- a/specs/eip4844/fork-choice.md +++ b/specs/eip4844/fork-choice.md @@ -7,9 +7,8 @@ - [Introduction](#introduction) - [Containers](#containers) - - [`BlobsSidecar`](#blobssidecar) - [Helpers](#helpers) - - [`validate_blobs_sidecar`](#validate_blobs_sidecar) + - [`validate_blob_sidecars`](#validate_blob_sidecars) - [`is_data_available`](#is_data_available) - [Updated fork-choice handlers](#updated-fork-choice-handlers) - [`on_block`](#on_block) @@ -23,54 +22,42 @@ This is the modification of the fork choice accompanying the EIP-4844 upgrade. ## Containers -### `BlobsSidecar` - -```python -class BlobsSidecar(Container): - beacon_block_root: Root - beacon_block_slot: Slot - blobs: List[Blob, MAX_BLOBS_PER_BLOCK] - kzg_aggregated_proof: KZGProof -``` - ## Helpers -#### `validate_blobs_sidecar` +#### `validate_blob_sidecars` ```python -def validate_blobs_sidecar(slot: Slot, +def validate_blob_sidecars(slot: Slot, beacon_block_root: Root, expected_kzg_commitments: Sequence[KZGCommitment], - blobs_sidecar: BlobsSidecar) -> None: + blob_sidecars: Sequence[BlobSidecar]) -> None: assert slot == blobs_sidecar.beacon_block_slot assert beacon_block_root == blobs_sidecar.beacon_block_root - blobs = blobs_sidecar.blobs - kzg_aggregated_proof = blobs_sidecar.kzg_aggregated_proof - assert len(expected_kzg_commitments) == len(blobs) - - assert verify_aggregate_kzg_proof(blobs, expected_kzg_commitments, kzg_aggregated_proof) + assert len(expected_kzg_commitments) == len(blob_sidecars) + # TODO validate commitments individually or aggregate first? + # assert verify_aggregate_kzg_proof(blobs, expected_kzg_commitments, kzg_aggregated_proof) ``` #### `is_data_available` The implementation of `is_data_available` will become more sophisticated during later scaling upgrades. -Initially, verification requires every verifying actor to retrieve the matching `BlobsSidecar`, -and validate the sidecar with `validate_blobs_sidecar`. +Initially, verification requires every verifying actor to retrieve all matching `BlobSidecar`s, +and validate the sidecar with `validate_blob_sidecars`. -The block MUST NOT be considered valid until a valid `BlobsSidecar` has been downloaded. Blocks that have been previously validated as available SHOULD be considered available even if the associated `BlobsSidecar` has subsequently been pruned. +The block MUST NOT be considered valid until all valid `BlobSidecar`s have been downloaded. Blocks that have been previously validated as available SHOULD be considered available even if the associated `BlobSidecar`s has subsequently been pruned. ```python def is_data_available(slot: Slot, beacon_block_root: Root, blob_kzg_commitments: Sequence[KZGCommitment]) -> bool: # `retrieve_blobs_sidecar` is implementation and context dependent, raises an exception if not available. # Note: the p2p network does not guarantee sidecar retrieval outside of `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` - sidecar = retrieve_blobs_sidecar(slot, beacon_block_root) + sidecars = retrieve_blob_sidecars(slot, beacon_block_root) # For testing, `retrieve_blobs_sidecar` returns "TEST". # TODO: Remove it once we have a way to inject `BlobsSidecar` into tests. if isinstance(sidecar, str): return True - validate_blobs_sidecar(slot, beacon_block_root, blob_kzg_commitments, sidecar) + validate_blob_sidecars(slot, beacon_block_root, blob_kzg_commitments, sidecars) return True ``` @@ -102,7 +89,7 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: # [New in EIP-4844] # Check if blob data is available # If not, this block MAY be queued and subsequently considered when blob data becomes available - assert is_data_available(block.slot, hash_tree_root(block), block.body.blob_kzg_commitments) + assert is_data_available(block.slot, hash_tree_root(block), block.body.blob_kzg_commitments) # Check the block is valid and compute the post-state state = pre_state.copy() diff --git a/specs/eip4844/p2p-interface.md b/specs/eip4844/p2p-interface.md index ae9380f7a8..94821a82cb 100644 --- a/specs/eip4844/p2p-interface.md +++ b/specs/eip4844/p2p-interface.md @@ -10,23 +10,22 @@ The specification of these changes continues in the same format as the network s - - [Configuration](#configuration) - - [Containers](#containers) - - [`SignedBeaconBlockAndBlobsSidecar`](#signedbeaconblockandblobssidecar) - - [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) - - [Topics and messages](#topics-and-messages) - - [Global topics](#global-topics) - - [`beacon_block`](#beacon_block) - - [`beacon_block_and_blobs_sidecar`](#beacon_block_and_blobs_sidecar) - - [Transitioning the gossip](#transitioning-the-gossip) - - [The Req/Resp domain](#the-reqresp-domain) - - [Messages](#messages) - - [BeaconBlocksByRange v2](#beaconblocksbyrange-v2) - - [BeaconBlocksByRoot v2](#beaconblocksbyroot-v2) - - [BeaconBlockAndBlobsSidecarByRoot v1](#beaconblockandblobssidecarbyroot-v1) - - [BlobsSidecarsByRange v1](#blobssidecarsbyrange-v1) -- [Design decision rationale](#design-decision-rationale) - - [Why are blobs relayed as a sidecar, separate from beacon blocks?](#why-are-blobs-relayed-as-a-sidecar-separate-from-beacon-blocks) +- [Configuration](#configuration) +- [Containers](#containers) + - [`BlobSidecar`](#blobsidecar) + - [`SignedBlobSidecar`](#signedblobsidecar) +- [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) + - [Topics and messages](#topics-and-messages) + - [Global topics](#global-topics) + - [`beacon_block`](#beacon_block) + - [`blob_sidecar_{index}`](#blob_sidecar_index) + - [Transitioning the gossip](#transitioning-the-gossip) +- [The Req/Resp domain](#the-reqresp-domain) + - [Messages](#messages) + - [BeaconBlocksByRange v2](#beaconblocksbyrange-v2) + - [BeaconBlocksByRoot v2](#beaconblocksbyroot-v2) + - [BlobSidecarsByRoot v1](#blobsidecarsbyroot-v1) + - [BlobsSidecarsByRange v1](#blobssidecarsbyrange-v1) @@ -35,17 +34,31 @@ The specification of these changes continues in the same format as the network s | Name | Value | Description | |------------------------------------------|-----------------------------------|---------------------------------------------------------------------| -| `MAX_REQUEST_BLOBS_SIDECARS` | `2**7` (= 128) | Maximum number of blobs sidecars in a single request | +| `MAX_REQUEST_BLOCKS_EIP4844` | `2**7` (= 128) | Maximum number of blocks in a single request | | `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` | `2**12` (= 4096 epochs, ~18 days) | The minimum epoch range over which a node must serve blobs sidecars | ## Containers -### `SignedBeaconBlockAndBlobsSidecar` +### `BlobSidecar` ```python -class SignedBeaconBlockAndBlobsSidecar(Container): - beacon_block: SignedBeaconBlock - blobs_sidecar: BlobsSidecar +class BlobSidecar(Container): + block_root: Root + index: BlobIndex # Index of blob in block + slot: Slot + block_parent_root: Root # Proposer shuffling determinant + proposer_index: ValidatorIndex + blob: Blob + kzg_commitment: KZGCommitment + kzg_proof: KZGProof # Allows for quick verification of kzg_commitment +``` + +### `SignedBlobSidecar` + +```python +class SignedBlobSidecar(Container): + message: BlobSidecar + signature: Signature ``` ## The gossip domain: gossipsub @@ -65,34 +78,35 @@ The new topics along with the type of the `data` field of a gossipsub message ar | Name | Message Type | | - | - | -| `beacon_block_and_blobs_sidecar` | `SignedBeaconBlockAndBlobsSidecar` (new) | +| `blob_sidecar_{index}` | `SignedBlobSidecar` (new) | #### Global topics -EIP-4844 introduces a new global topic for beacon block and blobs-sidecars. +EIP-4844 introduces new global topics for blob sidecars. ##### `beacon_block` -This topic is deprecated and clients **MUST NOT** expose in their topic set to any peer. Implementers do not need to do -anything beyond simply skip implementation, and it is explicitly called out as it is a departure from previous versioning -of this topic. +The *type* of the payload of this topic changes to the (modified) `SignedBeaconBlock` found in EIP4844. -Refer to [the section below](#transitioning-the-gossip) for details on how to transition the gossip. +##### `blob_sidecar_{index}` -##### `beacon_block_and_blobs_sidecar` +This topic is used to propagate signed blob sidecars, one for each sidecar index. -This topic is used to propagate new signed and coupled beacon blocks and blobs sidecars to all nodes on the networks. +The following validations MUST pass before forwarding the `sidecar` on the network, assuming the alias `sidecar = signed_blob_sidecar.message`: -In addition to the gossip validations for the `beacon_block` topic from prior specifications, the following validations MUST pass before forwarding the `signed_beacon_block_and_blobs_sidecar` on the network. -Alias `signed_beacon_block = signed_beacon_block_and_blobs_sidecar.beacon_block`, `block = signed_beacon_block.message`, `execution_payload = block.body.execution_payload`. -- _[REJECT]_ The KZG commitments correspond to the versioned hashes in the transactions list - -- i.e. `verify_kzg_commitments_against_transactions(block.body.execution_payload.transactions, block.body.blob_kzg_commitments)` - -Alias `sidecar = signed_beacon_block_and_blobs_sidecar.blobs_sidecar`. -- _[IGNORE]_ the `sidecar.beacon_block_slot` is for the current slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) - -- i.e. `sidecar.beacon_block_slot == block.slot`. -- _[REJECT]_ The KZG commitments in the block are valid against the provided blobs sidecar - -- i.e. `validate_blobs_sidecar(block.slot, hash_tree_root(block), block.body.blob_kzg_commitments, sidecar)` +- _[REJECT]_ The sidecar is for the correct topic -- + i.e. `sidecar.index` matches the topic `{index}`. +- _[IGNORE]_ The sidecar is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) +- _[IGNORE]_ The sidecar is from a slot greater than the latest finalized slot -- + i.e. validate that `sidecar.slot > compute_start_slot_at_epoch(state.finalized_checkpoint.epoch)` +- _[REJECT]_ The proposer signature, `signed_blob_sidecar.signature`, is valid with respect to the `sidecar.proposer_index` pubkey. +- _[IGNORE]_ The sidecar is the only sidecar with valid signature received for the tuple `(sidecar.slot, sidecar.proposer_index, sidecar.index)`. + -- Clients MUST discard blocks where multiple sidecars for the same proposer and index have been observed. +- _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot + in the context of the current shuffling (defined by `parent_root`/`slot`). + If the `proposer_index` cannot immediately be verified against the expected shuffling, + the sidecar MAY be queued for later processing while proposers for the block's branch are calculated -- + in such a case _do not_ `REJECT`, instead `IGNORE` this message. ### Transitioning the gossip @@ -121,6 +135,8 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: | `CAPELLA_FORK_VERSION` | `capella.SignedBeaconBlock` | | `EIP4844_FORK_VERSION` | `eip4844.SignedBeaconBlock` | +No more than `MAX_REQUEST_BLOCKS_EIP4844` may be requested at a time. + #### BeaconBlocksByRoot v2 **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_root/2/` @@ -139,15 +155,23 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: | `BELLATRIX_FORK_VERSION` | `bellatrix.SignedBeaconBlock` | | `CAPELLA_FORK_VERSION` | `capella.SignedBeaconBlock` | -#### BeaconBlockAndBlobsSidecarByRoot v1 +No more than `MAX_REQUEST_BLOCKS_EIP4844` may be requested at a time. -**Protocol ID:** `/eth2/beacon_chain/req/beacon_block_and_blobs_sidecar_by_root/1/` +#### BlobSidecarsByRoot v1 + +**Protocol ID:** `/eth2/beacon_chain/req/blob_sidecars_by_root/1/` Request Content: +```python +class BlobIdentifier(Container): + block_root: Root + index: uint64 +``` + ``` ( - List[Root, MAX_REQUEST_BLOCKS] + List[BlobIdentifier, MAX_REQUEST_BLOCKS_EIP4844] ) ``` @@ -155,29 +179,32 @@ Response Content: ``` ( - List[SignedBeaconBlockAndBlobsSidecar, MAX_REQUEST_BLOCKS] + List[BlobSidecar, MAX_REQUEST_BLOCKS_EIP4844] ) ``` -Requests blocks by block root (= `hash_tree_root(SignedBeaconBlockAndBlobsSidecar.beacon_block.message)`). -The response is a list of `SignedBeaconBlockAndBlobsSidecar` whose length is less than or equal to the number of requests. +Requests sidecars by block root and index. +The response is a list of `BlobSidecar` whose length is less than or equal to the number of requests. It may be less in the case that the responding peer is missing blocks and sidecars. -No more than `MAX_REQUEST_BLOCKS` may be requested at a time. +The response is unsigned, i.e. `BlobSidecar`, as the signature of the beacon block proposer +may not be available beyond the initial distribution via gossip. + +No more than `MAX_REQUEST_BLOCKS_EIP4844` may be requested at a time. -`BeaconBlockAndBlobsSidecarByRoot` is primarily used to recover recent blocks and sidecars (e.g. when receiving a block or attestation whose parent is unknown). +`BlobSidecarsByRoot` is primarily used to recover recent blocks and sidecars (e.g. when receiving a block or attestation whose parent is unknown). The response MUST consist of zero or more `response_chunk`. -Each _successful_ `response_chunk` MUST contain a single `SignedBeaconBlockAndBlobsSidecar` payload. +Each _successful_ `response_chunk` MUST contain a single `BlobSidecar` payload. -Clients MUST support requesting blocks and sidecars since `minimum_request_epoch`, where `minimum_request_epoch = max(finalized_epoch, current_epoch - MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS, EIP4844_FORK_EPOCH)`. If any root in the request content references a block earlier than `minimum_request_epoch`, peers SHOULD respond with error code `3: ResourceUnavailable`. +Clients MUST support requesting sidecars since `minimum_request_epoch`, where `minimum_request_epoch = max(finalized_epoch, current_epoch - MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS, EIP4844_FORK_EPOCH)`. If any root in the request content references a block earlier than `minimum_request_epoch`, peers MAY respond with error code `3: ResourceUnavailable` or not include the blob in the response. -Clients MUST respond with at least one block and sidecar, if they have it. +Clients MUST respond with at least one sidecar, if they have it. Clients MAY limit the number of blocks and sidecars in the response. #### BlobsSidecarsByRange v1 -**Protocol ID:** `/eth2/beacon_chain/req/blobs_sidecars_by_range/1/` +**Protocol ID:** `/eth2/beacon_chain/req/blob_sidecars_by_range/1/` Request Content: ``` @@ -188,16 +215,22 @@ Request Content: ``` Response Content: + +```python +class BlobSidecars(Container): + block_root: Root + List[BlobSidecar, MAX_BLOBS_PER_BLOCK] + ``` ( - List[BlobsSidecar, MAX_REQUEST_BLOBS_SIDECARS] + List[BlobSidecars, MAX_REQUEST_BLOCKS_EIP4844] ) ``` -Requests blobs sidecars in the slot range `[start_slot, start_slot + count)`, +Requests blob sidecars in the slot range `[start_slot, start_slot + count)`, leading up to the current head block as selected by fork choice. -The response is unsigned, i.e. `BlobsSidecarsByRange`, as the signature of the beacon block proposer +The response is unsigned, i.e. `BlobSidecarsByRange`, as the signature of the beacon block proposer may not be available beyond the initial distribution via gossip. Before consuming the next response chunk, the response reader SHOULD verify the blobs sidecar is well-formatted and @@ -215,7 +248,7 @@ Clients MUST keep a record of signed blobs sidecars seen on the epoch range where `current_epoch` is defined by the current wall-clock time, and clients MUST support serving requests of blobs on this range. -Peers that are unable to reply to blobs sidecars requests within the `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` +Peers that are unable to reply to blob sidecar requests within the `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` epoch range SHOULD respond with error code `3: ResourceUnavailable`. Such peers that are unable to successfully reply to this range of requests MAY get descored or disconnected at any time. @@ -229,7 +262,7 @@ participating in the networking immediately, other peers MAY disconnect and/or temporarily ban such an un-synced or semi-synced client. Clients MUST respond with at least the first blobs sidecar that exists in the range, if they have it, -and no more than `MAX_REQUEST_BLOBS_SIDECARS` sidecars. +and no more than `MAX_REQUEST_BLOCKS_EIP4844` sidecars. The following blobs sidecars, where they exist, MUST be sent in consecutive order. diff --git a/specs/eip4844/validator.md b/specs/eip4844/validator.md index bfdd69370a..619444351d 100644 --- a/specs/eip4844/validator.md +++ b/specs/eip4844/validator.md @@ -79,27 +79,30 @@ def validate_blobs_and_kzg_commitments(execution_payload: ExecutionPayload, 3. If valid, set `block.body.blob_kzg_commitments = blob_kzg_commitments`. -#### Constructing the `SignedBeaconBlockAndBlobsSidecar` -To construct a `SignedBeaconBlockAndBlobsSidecar`, a `signed_beacon_block_and_blobs_sidecar` is defined with the necessary context for block and sidecar proposal. - -##### Block -Set `signed_beacon_block_and_blobs_sidecar.beacon_block = block` where `block` is obtained above. +#### Constructing the `SignedBlobSidecar` +To construct a `SignedBlobSidecar`, a `signed_blob_sidecar` is defined with the necessary context for block and sidecar proposal. ##### Sidecar -Coupled with block, the corresponding blobs are packaged into a sidecar object for distribution to the network. -Set `signed_beacon_block_and_blobs_sidecar.blobs_sidecar = sidecar` where `sidecar` is obtained from: +Coupled with block, the corresponding blobs are packaged into sidecar objects for distribution to the network. + +Each `sidecar` is obtained from: ```python -def get_blobs_sidecar(block: BeaconBlock, blobs: Sequence[Blob]) -> BlobsSidecar: - return BlobsSidecar( - beacon_block_root=hash_tree_root(block), - beacon_block_slot=block.slot, - blobs=blobs, - kzg_aggregated_proof=compute_aggregate_kzg_proof(blobs), - ) +def get_blob_sidecar(block: BeaconBlock, blobs: Sequence[Blob]) -> Sequence[BlobsSidecar]: + return [ + BlobsSidecar( + block_root=hash_tree_root(block), + index=idx + slot=block.slot, + block_parent_root=block.parent_root, + blob=blob, + kzg_commitment=block.body.blob_kzg_commitments[idx], + kzg_aggregated_proof=compute_kzg_proof(blob),) + for idx, blob in enumerate(blobs) + ] ``` -This `signed_beacon_block_and_blobs_sidecar` is then published to the global `beacon_block_and_blobs_sidecar` topic. +Each `sidecar` is then published to the global `blob_sidecar_{index}` topics according to its index. After publishing the peers on the network may request the sidecar through sync-requests, or a local user may be interested. The validator MUST hold on to sidecars for `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` epochs and serve when capable, From 1e07685f74bf9d508d2acbb8c6b82aace53142d7 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Tue, 7 Feb 2023 11:14:59 +0100 Subject: [PATCH 02/22] doctoc --- specs/eip4844/validator.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/specs/eip4844/validator.md b/specs/eip4844/validator.md index 619444351d..4ea7051e93 100644 --- a/specs/eip4844/validator.md +++ b/specs/eip4844/validator.md @@ -16,8 +16,7 @@ - [Block and sidecar proposal](#block-and-sidecar-proposal) - [Constructing the `BeaconBlockBody`](#constructing-the-beaconblockbody) - [Blob KZG commitments](#blob-kzg-commitments) - - [Constructing the `SignedBeaconBlockAndBlobsSidecar`](#constructing-the-signedbeaconblockandblobssidecar) - - [Block](#block) + - [Constructing the `SignedBlobSidecar`](#constructing-the-signedblobsidecar) - [Sidecar](#sidecar) From deb82e2f26e9a7cf3062bcce544e842fcfdb72b5 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Tue, 7 Feb 2023 11:23:18 +0100 Subject: [PATCH 03/22] fix member --- specs/eip4844/validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/eip4844/validator.md b/specs/eip4844/validator.md index 4ea7051e93..a4ff8b8f20 100644 --- a/specs/eip4844/validator.md +++ b/specs/eip4844/validator.md @@ -96,7 +96,7 @@ def get_blob_sidecar(block: BeaconBlock, blobs: Sequence[Blob]) -> Sequence[Blob block_parent_root=block.parent_root, blob=blob, kzg_commitment=block.body.blob_kzg_commitments[idx], - kzg_aggregated_proof=compute_kzg_proof(blob),) + kzg_proof=compute_kzg_proof(blob),) for idx, blob in enumerate(blobs) ] ``` From ffc78e99282e65cc4da92c4dba0743be434d367b Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Fri, 10 Feb 2023 10:40:43 +0100 Subject: [PATCH 04/22] fixes * separate constant for blob requests * pedantry --- specs/eip4844/p2p-interface.md | 62 +++++++++++++++++----------------- specs/eip4844/validator.md | 32 ++++++++++-------- 2 files changed, 48 insertions(+), 46 deletions(-) diff --git a/specs/eip4844/p2p-interface.md b/specs/eip4844/p2p-interface.md index 94821a82cb..98365f8238 100644 --- a/specs/eip4844/p2p-interface.md +++ b/specs/eip4844/p2p-interface.md @@ -10,22 +10,24 @@ The specification of these changes continues in the same format as the network s -- [Configuration](#configuration) -- [Containers](#containers) - - [`BlobSidecar`](#blobsidecar) - - [`SignedBlobSidecar`](#signedblobsidecar) -- [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) - - [Topics and messages](#topics-and-messages) - - [Global topics](#global-topics) - - [`beacon_block`](#beacon_block) - - [`blob_sidecar_{index}`](#blob_sidecar_index) - - [Transitioning the gossip](#transitioning-the-gossip) -- [The Req/Resp domain](#the-reqresp-domain) - - [Messages](#messages) - - [BeaconBlocksByRange v2](#beaconblocksbyrange-v2) - - [BeaconBlocksByRoot v2](#beaconblocksbyroot-v2) - - [BlobSidecarsByRoot v1](#blobsidecarsbyroot-v1) - - [BlobsSidecarsByRange v1](#blobssidecarsbyrange-v1) + - [Configuration](#configuration) + - [Containers](#containers) + - [`BlobSidecar`](#blobsidecar) + - [`SignedBlobSidecar`](#signedblobsidecar) + - [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) + - [Topics and messages](#topics-and-messages) + - [Global topics](#global-topics) + - [`beacon_block`](#beacon_block) + - [`blob_sidecar_{index}`](#blob_sidecar_index) + - [Transitioning the gossip](#transitioning-the-gossip) + - [The Req/Resp domain](#the-reqresp-domain) + - [Messages](#messages) + - [BeaconBlocksByRange v2](#beaconblocksbyrange-v2) + - [BeaconBlocksByRoot v2](#beaconblocksbyroot-v2) + - [BlobSidecarsByRoot v1](#blobsidecarsbyroot-v1) + - [BlobSidecarsByRange v1](#blobsidecarsbyrange-v1) +- [Design decision rationale](#design-decision-rationale) + - [Why are blobs relayed as a sidecar, separate from beacon blocks?](#why-are-blobs-relayed-as-a-sidecar-separate-from-beacon-blocks) @@ -35,6 +37,7 @@ The specification of these changes continues in the same format as the network s | Name | Value | Description | |------------------------------------------|-----------------------------------|---------------------------------------------------------------------| | `MAX_REQUEST_BLOCKS_EIP4844` | `2**7` (= 128) | Maximum number of blocks in a single request | +| `MAX_REQUEST_BLOB_SIDECARS` | `2**7` (= 128) | Maximum number of blob sidecars in a single request | | `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` | `2**12` (= 4096 epochs, ~18 days) | The minimum epoch range over which a node must serve blobs sidecars | ## Containers @@ -68,7 +71,8 @@ Some gossip meshes are upgraded in the fork of EIP-4844 to support upgraded type ### Topics and messages Topics follow the same specification as in prior upgrades. -The `beacon_block` topic is deprecated and replaced by the `beacon_block_and_blobs_sidecar` topic. All other topics remain stable. + +The `beacon_block` topic is modified to also support EIP4844 blocks and new topics are added per table below. All other topics remain stable. The specification around the creation, validation, and dissemination of messages has not changed from the Capella document unless explicitly noted here. @@ -94,19 +98,14 @@ This topic is used to propagate signed blob sidecars, one for each sidecar index The following validations MUST pass before forwarding the `sidecar` on the network, assuming the alias `sidecar = signed_blob_sidecar.message`: -- _[REJECT]_ The sidecar is for the correct topic -- - i.e. `sidecar.index` matches the topic `{index}`. +- _[REJECT]_ The sidecar is for the correct topic -- i.e. `sidecar.index` matches the topic `{index}`. - _[IGNORE]_ The sidecar is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- _[IGNORE]_ The sidecar is from a slot greater than the latest finalized slot -- - i.e. validate that `sidecar.slot > compute_start_slot_at_epoch(state.finalized_checkpoint.epoch)` +- _[IGNORE]_ The sidecar is from a slot greater than the latest finalized slot -- i.e. validate that `sidecar.slot > compute_start_slot_at_epoch(state.finalized_checkpoint.epoch)` - _[REJECT]_ The proposer signature, `signed_blob_sidecar.signature`, is valid with respect to the `sidecar.proposer_index` pubkey. - _[IGNORE]_ The sidecar is the only sidecar with valid signature received for the tuple `(sidecar.slot, sidecar.proposer_index, sidecar.index)`. -- Clients MUST discard blocks where multiple sidecars for the same proposer and index have been observed. -- _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot - in the context of the current shuffling (defined by `parent_root`/`slot`). - If the `proposer_index` cannot immediately be verified against the expected shuffling, - the sidecar MAY be queued for later processing while proposers for the block's branch are calculated -- - in such a case _do not_ `REJECT`, instead `IGNORE` this message. +- _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `parent_root`/`slot`). + If the `proposer_index` cannot immediately be verified against the expected shuffling, the sidecar MAY be queued for later processing while proposers for the block's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message. ### Transitioning the gossip @@ -171,7 +170,7 @@ class BlobIdentifier(Container): ``` ( - List[BlobIdentifier, MAX_REQUEST_BLOCKS_EIP4844] + List[BlobIdentifier, MAX_REQUEST_BLOBS_SIDECARS * MAX_BLOBS_PER_BLOCK] ) ``` @@ -179,7 +178,7 @@ Response Content: ``` ( - List[BlobSidecar, MAX_REQUEST_BLOCKS_EIP4844] + List[BlobSidecar, MAX_REQUEST_BLOBS_SIDECARS * MAX_BLOBS_PER_BLOCK] ) ``` @@ -190,7 +189,7 @@ It may be less in the case that the responding peer is missing blocks and sideca The response is unsigned, i.e. `BlobSidecar`, as the signature of the beacon block proposer may not be available beyond the initial distribution via gossip. -No more than `MAX_REQUEST_BLOCKS_EIP4844` may be requested at a time. +No more than `MAX_REQUEST_BLOBS_SIDECARS * MAX_BLOBS_PER_BLOCK` may be requested at a time. `BlobSidecarsByRoot` is primarily used to recover recent blocks and sidecars (e.g. when receiving a block or attestation whose parent is unknown). @@ -202,7 +201,7 @@ Clients MUST support requesting sidecars since `minimum_request_epoch`, where `m Clients MUST respond with at least one sidecar, if they have it. Clients MAY limit the number of blocks and sidecars in the response. -#### BlobsSidecarsByRange v1 +#### BlobSidecarsByRange v1 **Protocol ID:** `/eth2/beacon_chain/req/blob_sidecars_by_range/1/` @@ -220,10 +219,11 @@ Response Content: class BlobSidecars(Container): block_root: Root List[BlobSidecar, MAX_BLOBS_PER_BLOCK] +``` ``` ( - List[BlobSidecars, MAX_REQUEST_BLOCKS_EIP4844] + List[BlobSidecars, MAX_REQUEST_BLOB_SIDECARS] ) ``` diff --git a/specs/eip4844/validator.md b/specs/eip4844/validator.md index a4ff8b8f20..8c3fc6faf3 100644 --- a/specs/eip4844/validator.md +++ b/specs/eip4844/validator.md @@ -16,7 +16,7 @@ - [Block and sidecar proposal](#block-and-sidecar-proposal) - [Constructing the `BeaconBlockBody`](#constructing-the-beaconblockbody) - [Blob KZG commitments](#blob-kzg-commitments) - - [Constructing the `SignedBlobSidecar`](#constructing-the-signedblobsidecar) + - [Constructing the `SignedBlobSidecar`s](#constructing-the-signedblobsidecars) - [Sidecar](#sidecar) @@ -78,27 +78,29 @@ def validate_blobs_and_kzg_commitments(execution_payload: ExecutionPayload, 3. If valid, set `block.body.blob_kzg_commitments = blob_kzg_commitments`. -#### Constructing the `SignedBlobSidecar` +#### Constructing the `SignedBlobSidecar`s + To construct a `SignedBlobSidecar`, a `signed_blob_sidecar` is defined with the necessary context for block and sidecar proposal. ##### Sidecar -Coupled with block, the corresponding blobs are packaged into sidecar objects for distribution to the network. +Blobs associated with a block are packaged into sidecar objects for distribution to the network. Each `sidecar` is obtained from: ```python -def get_blob_sidecar(block: BeaconBlock, blobs: Sequence[Blob]) -> Sequence[BlobsSidecar]: - return [ - BlobsSidecar( - block_root=hash_tree_root(block), - index=idx - slot=block.slot, - block_parent_root=block.parent_root, - blob=blob, - kzg_commitment=block.body.blob_kzg_commitments[idx], - kzg_proof=compute_kzg_proof(blob),) - for idx, blob in enumerate(blobs) - ] +def get_blob_sidecars(block: BeaconBlock, blobs: Sequence[Blob]) -> Sequence[BlobsSidecar]: + return [ + BlobsSidecar( + block_root=hash_tree_root(block), + index=index, + slot=block.slot, + block_parent_root=block.parent_root, + blob=blob, + kzg_commitment=block.body.blob_kzg_commitments[idx], + kzg_proof=compute_kzg_proof(blob), + ) + for index, blob in enumerate(blobs) + ] ``` Each `sidecar` is then published to the global `blob_sidecar_{index}` topics according to its index. From 8bc19d99aea32897be7eace506b1b344b33b0de1 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Fri, 10 Feb 2023 11:16:51 +0100 Subject: [PATCH 05/22] fixes * expand sidecar gossip conditions * editing * add spec text for `BlobSidecar` signatures --- specs/deneb/fork-choice.md | 2 +- specs/deneb/p2p-interface.md | 49 +++++++++++++++++++----------------- specs/deneb/validator.md | 19 +++++++++++--- 3 files changed, 43 insertions(+), 27 deletions(-) diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index 89eac22020..ea235c0553 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -53,7 +53,7 @@ def is_data_available(slot: Slot, beacon_block_root: Root, blob_kzg_commitments: sidecars = retrieve_blob_sidecars(slot, beacon_block_root) # For testing, `retrieve_blobs_sidecar` returns "TEST". - # TODO: Remove it once we have a way to inject `BlobsSidecar` into tests. + # TODO: Remove it once we have a way to inject `BlobSidecar` into tests. if isinstance(sidecar, str): return True diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index f023657952..41af2f9f14 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -10,22 +10,22 @@ The specification of these changes continues in the same format as the network s - - [Configuration](#configuration) - - [Containers](#containers) - - [`BlobSidecar`](#blobsidecar) - - [`SignedBlobSidecar`](#signedblobsidecar) - - [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) - - [Topics and messages](#topics-and-messages) - - [Global topics](#global-topics) - - [`beacon_block`](#beacon_block) - - [`blob_sidecar_{index}`](#blob_sidecar_index) - - [Transitioning the gossip](#transitioning-the-gossip) - - [The Req/Resp domain](#the-reqresp-domain) - - [Messages](#messages) - - [BeaconBlocksByRange v2](#beaconblocksbyrange-v2) - - [BeaconBlocksByRoot v2](#beaconblocksbyroot-v2) - - [BlobSidecarsByRoot v1](#blobsidecarsbyroot-v1) - - [BlobSidecarsByRange v1](#blobsidecarsbyrange-v1) +- [Configuration](#configuration) +- [Containers](#containers) + - [`BlobSidecar`](#blobsidecar) + - [`SignedBlobSidecar`](#signedblobsidecar) +- [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) + - [Topics and messages](#topics-and-messages) + - [Global topics](#global-topics) + - [`beacon_block`](#beacon_block) + - [`blob_sidecar_{index}`](#blob_sidecar_index) + - [Transitioning the gossip](#transitioning-the-gossip) +- [The Req/Resp domain](#the-reqresp-domain) + - [Messages](#messages) + - [BeaconBlocksByRange v2](#beaconblocksbyrange-v2) + - [BeaconBlocksByRoot v2](#beaconblocksbyroot-v2) + - [BlobSidecarsByRoot v1](#blobsidecarsbyroot-v1) + - [BlobSidecarsByRange v1](#blobsidecarsbyrange-v1) - [Design decision rationale](#design-decision-rationale) - [Why are blobs relayed as a sidecar, separate from beacon blocks?](#why-are-blobs-relayed-as-a-sidecar-separate-from-beacon-blocks) @@ -99,8 +99,9 @@ This topic is used to propagate signed blob sidecars, one for each sidecar index The following validations MUST pass before forwarding the `sidecar` on the network, assuming the alias `sidecar = signed_blob_sidecar.message`: - _[REJECT]_ The sidecar is for the correct topic -- i.e. `sidecar.index` matches the topic `{index}`. -- _[IGNORE]_ The sidecar is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) +- _[IGNORE]_ The sidecar is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- i.e. validate that `sidecar.slot <= current_slot` (a client MAY queue future blocks for processing at the appropriate slot). - _[IGNORE]_ The sidecar is from a slot greater than the latest finalized slot -- i.e. validate that `sidecar.slot > compute_start_slot_at_epoch(state.finalized_checkpoint.epoch)` +- _[IGNORE]_ The blob's block's parent defined by `sidecar.block_parent_root`) has been seen (via both gossip and non-gossip sources) (a client MAY queue blocks for processing once the parent block is retrieved). - _[REJECT]_ The proposer signature, `signed_blob_sidecar.signature`, is valid with respect to the `sidecar.proposer_index` pubkey. - _[IGNORE]_ The sidecar is the only sidecar with valid signature received for the tuple `(sidecar.slot, sidecar.proposer_index, sidecar.index)`. -- Clients MUST discard blocks where multiple sidecars for the same proposer and index have been observed. @@ -140,9 +141,6 @@ No more than `MAX_REQUEST_BLOCKS_DENEB` may be requested at a time. **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_root/2/` -After `DENEB_FORK_EPOCH`, `BeaconBlocksByRootV2` is replaced by `BeaconBlockAndBlobsSidecarByRootV1`. -Clients MUST support requesting blocks by root for pre-fork-epoch blocks. - Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: [1]: # (eth2spec: skip) @@ -153,6 +151,7 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: | `ALTAIR_FORK_VERSION` | `altair.SignedBeaconBlock` | | `BELLATRIX_FORK_VERSION` | `bellatrix.SignedBeaconBlock` | | `CAPELLA_FORK_VERSION` | `capella.SignedBeaconBlock` | +| `DENEB_FORK_VERSION` | `deneb.SignedBeaconBlock` | No more than `MAX_REQUEST_BLOCKS_DENEB` may be requested at a time. @@ -160,6 +159,8 @@ No more than `MAX_REQUEST_BLOCKS_DENEB` may be requested at a time. **Protocol ID:** `/eth2/beacon_chain/req/blob_sidecars_by_root/1/` +New in deneb. + Request Content: ```python @@ -191,7 +192,7 @@ may not be available beyond the initial distribution via gossip. No more than `MAX_REQUEST_BLOBS_SIDECARS * MAX_BLOBS_PER_BLOCK` may be requested at a time. -`BlobSidecarsByRoot` is primarily used to recover recent blocks and sidecars (e.g. when receiving a block or attestation whose parent is unknown). +`BlobSidecarsByRoot` is primarily used to recover recent blobs (e.g. when receiving a block with a transaction whose corresponding blob is missing). The response MUST consist of zero or more `response_chunk`. Each _successful_ `response_chunk` MUST contain a single `BlobSidecar` payload. @@ -205,6 +206,8 @@ Clients MAY limit the number of blocks and sidecars in the response. **Protocol ID:** `/eth2/beacon_chain/req/blob_sidecars_by_range/1/` +New in deneb. + Request Content: ``` ( @@ -282,9 +285,9 @@ Clients MUST respond with blobs sidecars that are consistent from a single chain After the initial blobs sidecar, clients MAY stop in the process of responding if their fork choice changes the view of the chain in the context of the request. -# Design decision rationale +## Design decision rationale -## Why are blobs relayed as a sidecar, separate from beacon blocks? +### Why are blobs relayed as a sidecar, separate from beacon blocks? This "sidecar" design provides forward compatibility for further data increases by black-boxing `is_data_available()`: with full sharding `is_data_available()` can be replaced by data-availability-sampling (DAS) diff --git a/specs/deneb/validator.md b/specs/deneb/validator.md index 1b6995ad80..bd05c31d2d 100644 --- a/specs/deneb/validator.md +++ b/specs/deneb/validator.md @@ -88,9 +88,9 @@ Blobs associated with a block are packaged into sidecar objects for distribution Each `sidecar` is obtained from: ```python -def get_blob_sidecars(block: BeaconBlock, blobs: Sequence[Blob]) -> Sequence[BlobsSidecar]: +def get_blob_sidecars(block: BeaconBlock, blobs: Sequence[Blob]) -> Sequence[BlobSidecar]: return [ - BlobsSidecar( + BlobSidecar( block_root=hash_tree_root(block), index=index, slot=block.slot, @@ -101,11 +101,24 @@ def get_blob_sidecars(block: BeaconBlock, blobs: Sequence[Blob]) -> Sequence[Blo ) for index, blob in enumerate(blobs) ] + ``` -Each `sidecar` is then published to the global `blob_sidecar_{index}` topics according to its index. +Then `signed_sidecar = SignedBlobSidecar(message=sidecar, signature=signature)` is constructed and to the global `blob_sidecar_{index}` topics according to its index. + +`signature` is obtained from: + +```python +def get_blob_sidecar_signature(state: BeaconState, + sidecar: BlobSidecar, + privkey: int) -> BLSSignature: + domain = get_domain(state, DOMAIN_BEACON_PROPOSER, compute_epoch_at_slot(sidecar.slot)) + signing_root = compute_signing_root(sidecar, domain) + return bls.Sign(privkey, signing_root) +``` After publishing the peers on the network may request the sidecar through sync-requests, or a local user may be interested. + The validator MUST hold on to sidecars for `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` epochs and serve when capable, to ensure the data-availability of these blobs throughout the network. From c8719f85246a64383bd8cd0a977f88d7a3a255bb Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Tue, 14 Feb 2023 13:32:18 +0100 Subject: [PATCH 06/22] Apply suggestions from code review Co-authored-by: Danny Ryan --- specs/deneb/p2p-interface.md | 12 ++++++------ specs/deneb/validator.md | 4 ++-- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 41af2f9f14..d77e4f371d 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -47,13 +47,13 @@ The specification of these changes continues in the same format as the network s ```python class BlobSidecar(Container): block_root: Root - index: BlobIndex # Index of blob in block + index: BlobIndex # Index of blob in block slot: Slot - block_parent_root: Root # Proposer shuffling determinant + block_parent_root: Root # Proposer shuffling determinant proposer_index: ValidatorIndex blob: Blob kzg_commitment: KZGCommitment - kzg_proof: KZGProof # Allows for quick verification of kzg_commitment + kzg_proof: KZGProof # Allows for quick verification of kzg_commitment ``` ### `SignedBlobSidecar` @@ -101,11 +101,11 @@ The following validations MUST pass before forwarding the `sidecar` on the netwo - _[REJECT]_ The sidecar is for the correct topic -- i.e. `sidecar.index` matches the topic `{index}`. - _[IGNORE]_ The sidecar is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- i.e. validate that `sidecar.slot <= current_slot` (a client MAY queue future blocks for processing at the appropriate slot). - _[IGNORE]_ The sidecar is from a slot greater than the latest finalized slot -- i.e. validate that `sidecar.slot > compute_start_slot_at_epoch(state.finalized_checkpoint.epoch)` -- _[IGNORE]_ The blob's block's parent defined by `sidecar.block_parent_root`) has been seen (via both gossip and non-gossip sources) (a client MAY queue blocks for processing once the parent block is retrieved). +- _[IGNORE]_ The blob's block's parent (defined by `sidecar.block_parent_root`) has been seen (via both gossip and non-gossip sources) (a client MAY queue blocks for processing once the parent block is retrieved). - _[REJECT]_ The proposer signature, `signed_blob_sidecar.signature`, is valid with respect to the `sidecar.proposer_index` pubkey. - _[IGNORE]_ The sidecar is the only sidecar with valid signature received for the tuple `(sidecar.slot, sidecar.proposer_index, sidecar.index)`. -- Clients MUST discard blocks where multiple sidecars for the same proposer and index have been observed. -- _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `parent_root`/`slot`). +- _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `block_parent_root`/`slot`). If the `proposer_index` cannot immediately be verified against the expected shuffling, the sidecar MAY be queued for later processing while proposers for the block's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message. ### Transitioning the gossip @@ -185,7 +185,7 @@ Response Content: Requests sidecars by block root and index. The response is a list of `BlobSidecar` whose length is less than or equal to the number of requests. -It may be less in the case that the responding peer is missing blocks and sidecars. +It may be less in the case that the responding peer is missing blocks or sidecars. The response is unsigned, i.e. `BlobSidecar`, as the signature of the beacon block proposer may not be available beyond the initial distribution via gossip. diff --git a/specs/deneb/validator.md b/specs/deneb/validator.md index bd05c31d2d..a415fbf438 100644 --- a/specs/deneb/validator.md +++ b/specs/deneb/validator.md @@ -96,7 +96,7 @@ def get_blob_sidecars(block: BeaconBlock, blobs: Sequence[Blob]) -> Sequence[Blo slot=block.slot, block_parent_root=block.parent_root, blob=blob, - kzg_commitment=block.body.blob_kzg_commitments[idx], + kzg_commitment=block.body.blob_kzg_commitments[index], kzg_proof=compute_kzg_proof(blob), ) for index, blob in enumerate(blobs) @@ -104,7 +104,7 @@ def get_blob_sidecars(block: BeaconBlock, blobs: Sequence[Blob]) -> Sequence[Blo ``` -Then `signed_sidecar = SignedBlobSidecar(message=sidecar, signature=signature)` is constructed and to the global `blob_sidecar_{index}` topics according to its index. +Then `signed_sidecar = SignedBlobSidecar(message=sidecar, signature=signature)` is constructed and published to the `blob_sidecar_{index}` topics according to its index. `signature` is obtained from: From e6b8324e25f4e2151346629cd4535b6fde3ca083 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Tue, 14 Feb 2023 13:39:59 +0100 Subject: [PATCH 07/22] sidecar domain --- specs/deneb/beacon-chain.md | 6 ++++++ specs/deneb/validator.md | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/specs/deneb/beacon-chain.md b/specs/deneb/beacon-chain.md index 97b426f5d6..c06e44f39b 100644 --- a/specs/deneb/beacon-chain.md +++ b/specs/deneb/beacon-chain.md @@ -48,6 +48,12 @@ This upgrade adds blobs to the beacon chain as part of Deneb. This is an extensi ## Constants +### Domain types + +| Name | Value | +| - | - | +| `DOMAIN_BLOB_SIDECAR` | `DomainType('0x0B000000')` | + ### Blob | Name | Value | diff --git a/specs/deneb/validator.md b/specs/deneb/validator.md index a415fbf438..1d4a7287e9 100644 --- a/specs/deneb/validator.md +++ b/specs/deneb/validator.md @@ -112,7 +112,7 @@ Then `signed_sidecar = SignedBlobSidecar(message=sidecar, signature=signature)` def get_blob_sidecar_signature(state: BeaconState, sidecar: BlobSidecar, privkey: int) -> BLSSignature: - domain = get_domain(state, DOMAIN_BEACON_PROPOSER, compute_epoch_at_slot(sidecar.slot)) + domain = get_domain(state, DOMAIN_BLOB_SIDECAR, compute_epoch_at_slot(sidecar.slot)) signing_root = compute_signing_root(sidecar, domain) return bls.Sign(privkey, signing_root) ``` From 58207c1c05b494b97bf01176547e2ed8749783f5 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Tue, 14 Feb 2023 14:18:29 +0100 Subject: [PATCH 08/22] Upper limit on indices --- specs/deneb/fork-choice.md | 2 +- specs/deneb/p2p-interface.md | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index ea235c0553..29d59048b3 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -44,7 +44,7 @@ The implementation of `is_data_available` will become more sophisticated during Initially, verification requires every verifying actor to retrieve all matching `BlobSidecar`s, and validate the sidecar with `validate_blob_sidecars`. -The block MUST NOT be considered valid until all valid `BlobSidecar`s have been downloaded. Blocks that have been previously validated as available SHOULD be considered available even if the associated `BlobSidecar`s has subsequently been pruned. +The block MUST NOT be considered valid until all valid `BlobSidecar`s have been downloaded. Blocks that have been previously validated as available SHOULD be considered available even if the associated `BlobSidecar`s have subsequently been pruned. ```python def is_data_available(slot: Slot, beacon_block_root: Root, blob_kzg_commitments: Sequence[KZGCommitment]) -> bool: diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index d77e4f371d..3cbc000529 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -94,7 +94,7 @@ The *type* of the payload of this topic changes to the (modified) `SignedBeaconB ##### `blob_sidecar_{index}` -This topic is used to propagate signed blob sidecars, one for each sidecar index. +This topic is used to propagate signed blob sidecars, one for each sidecar index. The number of indices is defined by `MAX_BLOBS_PER_BLOCK`. The following validations MUST pass before forwarding the `sidecar` on the network, assuming the alias `sidecar = signed_blob_sidecar.message`: @@ -108,6 +108,7 @@ The following validations MUST pass before forwarding the `sidecar` on the netwo - _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `block_parent_root`/`slot`). If the `proposer_index` cannot immediately be verified against the expected shuffling, the sidecar MAY be queued for later processing while proposers for the block's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message. + ### Transitioning the gossip See gossip transition details found in the [Altair document](../altair/p2p-interface.md#transitioning-the-gossip) for From 3a37c3c4978467f8909f66d8899b80b9e1373e72 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Wed, 15 Feb 2023 08:00:39 +0100 Subject: [PATCH 09/22] Allow clients to orphan blocks from spammy proposers Proposers that spam the blob topic with multiple blob versions, some of which are invalid, MAY see their block orphaned. --- specs/deneb/p2p-interface.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 3cbc000529..6f64e5514e 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -104,7 +104,7 @@ The following validations MUST pass before forwarding the `sidecar` on the netwo - _[IGNORE]_ The blob's block's parent (defined by `sidecar.block_parent_root`) has been seen (via both gossip and non-gossip sources) (a client MAY queue blocks for processing once the parent block is retrieved). - _[REJECT]_ The proposer signature, `signed_blob_sidecar.signature`, is valid with respect to the `sidecar.proposer_index` pubkey. - _[IGNORE]_ The sidecar is the only sidecar with valid signature received for the tuple `(sidecar.slot, sidecar.proposer_index, sidecar.index)`. - -- Clients MUST discard blocks where multiple sidecars for the same proposer and index have been observed. + -- If full verification of the blob fails at a later processing stage, clients MUST clear the blob from this "seen" cache so as to allow a the valid blob to propagate. Block producers MAY orphan blocks if they have observed multiple blobs signed by the proposer for the same "seen" tuple. - _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `block_parent_root`/`slot`). If the `proposer_index` cannot immediately be verified against the expected shuffling, the sidecar MAY be queued for later processing while proposers for the block's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message. From da34af97d4b6575131e67c2bf22acb6de4f0951d Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Wed, 15 Feb 2023 08:51:57 +0100 Subject: [PATCH 10/22] simplify blob verification, range request * validate blobs using raw types * remove `BlobSidecars` and send flattened list of `BlobSidecar` instances instead --- specs/deneb/fork-choice.md | 35 +++++++++++++++--------------- specs/deneb/p2p-interface.md | 41 ++++++++++++------------------------ 2 files changed, 30 insertions(+), 46 deletions(-) diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index 29d59048b3..58c281a597 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -27,37 +27,36 @@ This is the modification of the fork choice accompanying the Deneb upgrade. #### `validate_blob_sidecars` ```python -def validate_blob_sidecars(slot: Slot, - beacon_block_root: Root, - expected_kzg_commitments: Sequence[KZGCommitment], - blob_sidecars: Sequence[BlobSidecar]) -> None: - assert slot == blobs_sidecar.beacon_block_slot - assert beacon_block_root == blobs_sidecar.beacon_block_root - assert len(expected_kzg_commitments) == len(blob_sidecars) - # TODO validate commitments individually or aggregate first? - # assert verify_aggregate_kzg_proof(blobs, expected_kzg_commitments, kzg_aggregated_proof) +def validate_blobs(expected_kzg_commitments: Sequence[KZGCommitment], + blobs: Sequence[Blob], + proofs: Sequence[KZGProof]) -> None: + assert len(expected_kzg_commitments) == len(blobs) + assert len(blobs) == len(proofs) + + # Clients MAY use `verify_blob_kzg_proof_multi` for efficiency + for commitment, blob, proof in zip(expected_kzg_commitments, blobs, proofs): + assert verify_blob_kzg_proof(commitment, blob, proof) ``` #### `is_data_available` The implementation of `is_data_available` will become more sophisticated during later scaling upgrades. -Initially, verification requires every verifying actor to retrieve all matching `BlobSidecar`s, -and validate the sidecar with `validate_blob_sidecars`. +Initially, verification requires every verifying actor to retrieve all matching `Blob`s and `KZGProof`s, and validate them with `validate_blobs`. -The block MUST NOT be considered valid until all valid `BlobSidecar`s have been downloaded. Blocks that have been previously validated as available SHOULD be considered available even if the associated `BlobSidecar`s have subsequently been pruned. +The block MUST NOT be considered valid until all valid `Blob`s have been downloaded. Blocks that have been previously validated as available SHOULD be considered available even if the associated `Blob`s have subsequently been pruned. ```python -def is_data_available(slot: Slot, beacon_block_root: Root, blob_kzg_commitments: Sequence[KZGCommitment]) -> bool: - # `retrieve_blobs_sidecar` is implementation and context dependent, raises an exception if not available. +def is_data_available(beacon_block_root: Root, blob_kzg_commitments: Sequence[KZGCommitment]) -> bool: + # `retrieve_blobs_and_proofs` is implementation and context dependent, raises an exception if not available. It returns all the blobs for the given block root. # Note: the p2p network does not guarantee sidecar retrieval outside of `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` - sidecars = retrieve_blob_sidecars(slot, beacon_block_root) + blobs, proofs = retrieve_blobs_and_proofs(beacon_block_root) - # For testing, `retrieve_blobs_sidecar` returns "TEST". + # For testing, `retrieve_blobs_and_proofs` returns "TEST". # TODO: Remove it once we have a way to inject `BlobSidecar` into tests. if isinstance(sidecar, str): return True - validate_blob_sidecars(slot, beacon_block_root, blob_kzg_commitments, sidecars) + validate_blobs(expected_kzg_commitments, blobs, proofs) return True ``` @@ -89,7 +88,7 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: # [New in Deneb] # Check if blob data is available # If not, this block MAY be queued and subsequently considered when blob data becomes available - assert is_data_available(block.slot, hash_tree_root(block), block.body.blob_kzg_commitments) + assert is_data_available(hash_tree_root(block), block.body.blob_kzg_commitments) # Check the block is valid and compute the post-state state = pre_state.copy() diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 6f64e5514e..63a63feda6 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -218,34 +218,24 @@ Request Content: ``` Response Content: - -```python -class BlobSidecars(Container): - block_root: Root - List[BlobSidecar, MAX_BLOBS_PER_BLOCK] -``` - ``` ( - List[BlobSidecars, MAX_REQUEST_BLOB_SIDECARS] + List[BlobSidecar, MAX_REQUEST_BLOB_SIDECARS * MAX_BLOBS_PER_BLOCK] ) ``` -Requests blob sidecars in the slot range `[start_slot, start_slot + count)`, -leading up to the current head block as selected by fork choice. +Requests blob sidecars in the slot range `[start_slot, start_slot + count)`, leading up to the current head block as selected by fork choice. -The response is unsigned, i.e. `BlobSidecarsByRange`, as the signature of the beacon block proposer -may not be available beyond the initial distribution via gossip. +The response is unsigned, i.e. `BlobSidecarsByRange`, as the signature of the beacon block proposer may not be available beyond the initial distribution via gossip. -Before consuming the next response chunk, the response reader SHOULD verify the blobs sidecar is well-formatted and -correct w.r.t. the expected KZG commitments through `validate_blobs_sidecar`. +Before consuming the next response chunk, the response reader SHOULD verify the blobs sidecar is well-formatted and correct w.r.t. the expected KZG commitments through `validate_blobs_sidecar`. -`BlobsSidecarsByRange` is primarily used to sync blobs that may have been missed on gossip and to sync within the `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` window. +`BlobSidecarsByRange` is primarily used to sync blobs that may have been missed on gossip and to sync within the `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` window. The request MUST be encoded as an SSZ-container. The response MUST consist of zero or more `response_chunk`. -Each _successful_ `response_chunk` MUST contain a single `BlobsSidecar` payload. +Each _successful_ `response_chunk` MUST contain a single `BlobSidecar` payload. Clients MUST keep a record of signed blobs sidecars seen on the epoch range `[max(current_epoch - MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS, DENEB_FORK_EPOCH), current_epoch]` @@ -265,26 +255,21 @@ to be fully compliant with `BlobsSidecarsByRange` requests. participating in the networking immediately, other peers MAY disconnect and/or temporarily ban such an un-synced or semi-synced client. -Clients MUST respond with at least the first blobs sidecar that exists in the range, if they have it, -and no more than `MAX_REQUEST_BLOCKS_DENEB` sidecars. +Clients MUST respond with at least the first blobs sidecar that exists in the range, if they have it, and no more than `MAX_REQUEST_BLOB_SIDECARS * MAX_BLOBS_PER_BLOCK` sidecars. -The following blobs sidecars, where they exist, MUST be sent in consecutive order. +The following blobs sidecars, where they exist, MUST be sent in consecutive `(slot, index)` order. Clients MAY limit the number of blobs sidecars in the response. -An empty `BlobSidecar` is one that does not contain any blobs, but contains non-zero `beacon_block_root`, `beacon_block_slot` and a valid `kzg_aggregated_proof`. -Clients MAY NOT want to consider empty `BlobSidecar`s in rate limiting logic. - -The response MUST contain no more than `count` blobs sidecars. +The response MUST contain no more than `count * MAX_BLOBS_PER_BLOCK` blob sidecars. -Clients MUST respond with blobs sidecars from their view of the current fork choice --- that is, blobs sidecars as included by blocks from the single chain defined by the current head. +Clients MUST respond with blob sidecars from their view of the current fork choice +-- that is, blob sidecars as included by blocks from the single chain defined by the current head. Of note, blocks from slots before the finalization MUST lead to the finalized block reported in the `Status` handshake. -Clients MUST respond with blobs sidecars that are consistent from a single chain within the context of the request. +Clients MUST respond with blob sidecars that are consistent from a single chain within the context of the request. -After the initial blobs sidecar, clients MAY stop in the process of responding -if their fork choice changes the view of the chain in the context of the request. +After the initial blob sidecar, clients MAY stop in the process of responding if their fork choice changes the view of the chain in the context of the request. ## Design decision rationale From a5f61fc173cc4c8bf350bc72945f5cf3dc39bae5 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Wed, 15 Feb 2023 08:57:23 +0100 Subject: [PATCH 11/22] correct function --- specs/deneb/validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/deneb/validator.md b/specs/deneb/validator.md index 1d4a7287e9..d867098744 100644 --- a/specs/deneb/validator.md +++ b/specs/deneb/validator.md @@ -97,7 +97,7 @@ def get_blob_sidecars(block: BeaconBlock, blobs: Sequence[Blob]) -> Sequence[Blo block_parent_root=block.parent_root, blob=blob, kzg_commitment=block.body.blob_kzg_commitments[index], - kzg_proof=compute_kzg_proof(blob), + kzg_proof=compute_blob_kzg_proof(blob), ) for index, blob in enumerate(blobs) ] From f0dc126602041679ed50f3cc8d6f6efccc0fa0ab Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Wed, 15 Feb 2023 09:10:31 +0100 Subject: [PATCH 12/22] doctoc --- specs/deneb/beacon-chain.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specs/deneb/beacon-chain.md b/specs/deneb/beacon-chain.md index c06e44f39b..ba0148d47f 100644 --- a/specs/deneb/beacon-chain.md +++ b/specs/deneb/beacon-chain.md @@ -11,6 +11,7 @@ - [Introduction](#introduction) - [Custom types](#custom-types) - [Constants](#constants) + - [Domain types](#domain-types) - [Blob](#blob) - [Preset](#preset) - [Execution](#execution) From c39fda19c6bfdb39054c4f3e02d2316886efaa04 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Thu, 16 Feb 2023 08:18:52 +0100 Subject: [PATCH 13/22] Apply suggestions from code review Co-authored-by: Danny Ryan Co-authored-by: Jimmy Chen --- specs/deneb/fork-choice.md | 5 +++-- specs/deneb/p2p-interface.md | 8 ++++---- specs/deneb/validator.md | 2 +- 3 files changed, 8 insertions(+), 7 deletions(-) diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index 58c281a597..46403cbe99 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -24,7 +24,7 @@ This is the modification of the fork choice accompanying the Deneb upgrade. ## Helpers -#### `validate_blob_sidecars` +#### `validate_blobs` ```python def validate_blobs(expected_kzg_commitments: Sequence[KZGCommitment], @@ -47,7 +47,8 @@ The block MUST NOT be considered valid until all valid `Blob`s have been downloa ```python def is_data_available(beacon_block_root: Root, blob_kzg_commitments: Sequence[KZGCommitment]) -> bool: - # `retrieve_blobs_and_proofs` is implementation and context dependent, raises an exception if not available. It returns all the blobs for the given block root. + # `retrieve_blobs_and_proofs` is implementation and context dependent + # It returns all the blobs for the given block root, and raises an exception if not available # Note: the p2p network does not guarantee sidecar retrieval outside of `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` blobs, proofs = retrieve_blobs_and_proofs(beacon_block_root) diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 63a63feda6..258b9e3f1f 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -228,7 +228,7 @@ Requests blob sidecars in the slot range `[start_slot, start_slot + count)`, lea The response is unsigned, i.e. `BlobSidecarsByRange`, as the signature of the beacon block proposer may not be available beyond the initial distribution via gossip. -Before consuming the next response chunk, the response reader SHOULD verify the blobs sidecar is well-formatted and correct w.r.t. the expected KZG commitments through `validate_blobs_sidecar`. +Before consuming the next response chunk, the response reader SHOULD verify the blob sidecar is well-formatted and correct w.r.t. the expected KZG commitments through `validate_blobs`. `BlobSidecarsByRange` is primarily used to sync blobs that may have been missed on gossip and to sync within the `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` window. @@ -255,11 +255,11 @@ to be fully compliant with `BlobsSidecarsByRange` requests. participating in the networking immediately, other peers MAY disconnect and/or temporarily ban such an un-synced or semi-synced client. -Clients MUST respond with at least the first blobs sidecar that exists in the range, if they have it, and no more than `MAX_REQUEST_BLOB_SIDECARS * MAX_BLOBS_PER_BLOCK` sidecars. +Clients MUST respond with at least the first blob sidecar that exists in the range, if they have it, and no more than `MAX_REQUEST_BLOB_SIDECARS * MAX_BLOBS_PER_BLOCK` sidecars. -The following blobs sidecars, where they exist, MUST be sent in consecutive `(slot, index)` order. +The following blob sidecars, where they exist, MUST be sent in consecutive `(slot, index)` order. -Clients MAY limit the number of blobs sidecars in the response. +Clients MAY limit the number of blob sidecars in the response. The response MUST contain no more than `count * MAX_BLOBS_PER_BLOCK` blob sidecars. diff --git a/specs/deneb/validator.md b/specs/deneb/validator.md index d867098744..45a8e5c834 100644 --- a/specs/deneb/validator.md +++ b/specs/deneb/validator.md @@ -104,7 +104,7 @@ def get_blob_sidecars(block: BeaconBlock, blobs: Sequence[Blob]) -> Sequence[Blo ``` -Then `signed_sidecar = SignedBlobSidecar(message=sidecar, signature=signature)` is constructed and published to the `blob_sidecar_{index}` topics according to its index. +Then for each sidecar, `signed_sidecar = SignedBlobSidecar(message=sidecar, signature=signature)` is constructed and published to the `blob_sidecar_{index}` topics according to its index. `signature` is obtained from: From 639ff9b2b0acfbba0e642e04da6efa76ad1f5292 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Thu, 16 Feb 2023 08:30:40 +0100 Subject: [PATCH 14/22] Update specs/deneb/p2p-interface.md Co-authored-by: Jimmy Chen --- specs/deneb/p2p-interface.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 258b9e3f1f..2ba8de409f 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -38,7 +38,7 @@ The specification of these changes continues in the same format as the network s |------------------------------------------|-----------------------------------|---------------------------------------------------------------------| | `MAX_REQUEST_BLOCKS_DENEB` | `2**7` (= 128) | Maximum number of blocks in a single request | | `MAX_REQUEST_BLOB_SIDECARS` | `2**7` (= 128) | Maximum number of blob sidecars in a single request | -| `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` | `2**12` (= 4096 epochs, ~18 days) | The minimum epoch range over which a node must serve blobs sidecars | +| `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` | `2**12` (= 4096 epochs, ~18 days) | The minimum epoch range over which a node must serve blob sidecars | ## Containers From 24a19bb886f7294f8b9c8ff990818936b044f5fa Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Thu, 16 Feb 2023 09:12:34 +0100 Subject: [PATCH 15/22] fixes * fight the test suite * clarify who orphans the block * must supply all blobs of a block in range request --- setup.py | 4 ++-- specs/deneb/fork-choice.md | 6 +++--- specs/deneb/p2p-interface.md | 23 ++++++++++++++--------- 3 files changed, 19 insertions(+), 14 deletions(-) diff --git a/setup.py b/setup.py index f87ed5a6cf..7b1c718b8c 100644 --- a/setup.py +++ b/setup.py @@ -653,9 +653,9 @@ def preparations(cls): @classmethod def sundry_functions(cls) -> str: return super().sundry_functions() + '\n\n' + ''' -def retrieve_blobs_sidecar(slot: Slot, beacon_block_root: Root) -> PyUnion[BlobsSidecar, str]: +def retrieve_blobs_and_proofs(slot: Slot, beacon_block_root: Root) -> PyUnion[BlobsSidecar, str]: # pylint: disable=unused-argument - return "TEST"''' + return ("TEST", "TEST")''' @classmethod def hardcoded_custom_type_dep_constants(cls, spec_object) -> str: diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index 46403cbe99..a2866092d2 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -8,7 +8,7 @@ - [Introduction](#introduction) - [Containers](#containers) - [Helpers](#helpers) - - [`validate_blob_sidecars`](#validate_blob_sidecars) + - [`validate_blobs`](#validate_blobs) - [`is_data_available`](#is_data_available) - [Updated fork-choice handlers](#updated-fork-choice-handlers) - [`on_block`](#on_block) @@ -52,9 +52,9 @@ def is_data_available(beacon_block_root: Root, blob_kzg_commitments: Sequence[KZ # Note: the p2p network does not guarantee sidecar retrieval outside of `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` blobs, proofs = retrieve_blobs_and_proofs(beacon_block_root) - # For testing, `retrieve_blobs_and_proofs` returns "TEST". + # For testing, `retrieve_blobs_and_proofs` returns ("TEST", "TEST"). # TODO: Remove it once we have a way to inject `BlobSidecar` into tests. - if isinstance(sidecar, str): + if isinstance(blobs, str): return True validate_blobs(expected_kzg_commitments, blobs, proofs) diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 2ba8de409f..5e0b162070 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -14,6 +14,7 @@ The specification of these changes continues in the same format as the network s - [Containers](#containers) - [`BlobSidecar`](#blobsidecar) - [`SignedBlobSidecar`](#signedblobsidecar) + - [`BlobIdentifier`](#blobidentifier) - [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) - [Topics and messages](#topics-and-messages) - [Global topics](#global-topics) @@ -61,7 +62,15 @@ class BlobSidecar(Container): ```python class SignedBlobSidecar(Container): message: BlobSidecar - signature: Signature + signature: BLSSignature +``` + +### `BlobIdentifier` + +```python +class BlobIdentifier(Container): + block_root: Root + index: uint64 ``` ## The gossip domain: gossipsub @@ -104,7 +113,7 @@ The following validations MUST pass before forwarding the `sidecar` on the netwo - _[IGNORE]_ The blob's block's parent (defined by `sidecar.block_parent_root`) has been seen (via both gossip and non-gossip sources) (a client MAY queue blocks for processing once the parent block is retrieved). - _[REJECT]_ The proposer signature, `signed_blob_sidecar.signature`, is valid with respect to the `sidecar.proposer_index` pubkey. - _[IGNORE]_ The sidecar is the only sidecar with valid signature received for the tuple `(sidecar.slot, sidecar.proposer_index, sidecar.index)`. - -- If full verification of the blob fails at a later processing stage, clients MUST clear the blob from this "seen" cache so as to allow a the valid blob to propagate. Block producers MAY orphan blocks if they have observed multiple blobs signed by the proposer for the same "seen" tuple. + -- If full verification of the blob fails at a later processing stage, clients MUST clear the blob from this "seen" cache so as to allow a the valid blob to propagate. The next block producer MAY orphan the block if they have observed multiple blobs signed by the proposer for the same "seen" tuple. - _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `block_parent_root`/`slot`). If the `proposer_index` cannot immediately be verified against the expected shuffling, the sidecar MAY be queued for later processing while proposers for the block's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message. @@ -164,12 +173,6 @@ New in deneb. Request Content: -```python -class BlobIdentifier(Container): - block_root: Root - index: uint64 -``` - ``` ( List[BlobIdentifier, MAX_REQUEST_BLOBS_SIDECARS * MAX_BLOBS_PER_BLOCK] @@ -255,7 +258,9 @@ to be fully compliant with `BlobsSidecarsByRange` requests. participating in the networking immediately, other peers MAY disconnect and/or temporarily ban such an un-synced or semi-synced client. -Clients MUST respond with at least the first blob sidecar that exists in the range, if they have it, and no more than `MAX_REQUEST_BLOB_SIDECARS * MAX_BLOBS_PER_BLOCK` sidecars. +Clients MUST respond with at least the blob sidecars of the first blob-carrying block that exists in the range, if they have it, and no more than `MAX_REQUEST_BLOB_SIDECARS * MAX_BLOBS_PER_BLOCK` sidecars. + +Clients MUST include all blob sidecars of each block from which they include blob sidecars. The following blob sidecars, where they exist, MUST be sent in consecutive `(slot, index)` order. From 5fe857b2096c6a903ac0c822cae66e2236f03feb Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Thu, 16 Feb 2023 09:20:40 +0100 Subject: [PATCH 16/22] fixes --- setup.py | 2 +- specs/deneb/p2p-interface.md | 2 +- specs/deneb/validator.md | 1 - 3 files changed, 2 insertions(+), 3 deletions(-) diff --git a/setup.py b/setup.py index 7b1c718b8c..666ba8afc1 100644 --- a/setup.py +++ b/setup.py @@ -653,7 +653,7 @@ def preparations(cls): @classmethod def sundry_functions(cls) -> str: return super().sundry_functions() + '\n\n' + ''' -def retrieve_blobs_and_proofs(slot: Slot, beacon_block_root: Root) -> PyUnion[BlobsSidecar, str]: +def retrieve_blobs_and_proofs(slot: Slot, beacon_block_root: Root) -> PyUnion[(Blob, KZGProof), (str, str)]: # pylint: disable=unused-argument return ("TEST", "TEST")''' diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 5e0b162070..163522ec3a 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -284,5 +284,5 @@ This "sidecar" design provides forward compatibility for further data increases with full sharding `is_data_available()` can be replaced by data-availability-sampling (DAS) thus avoiding all blobs being downloaded by all beacon nodes on the network. -Such sharding design may introduce an updated `BlobsSidecar` to identify the shard, +Such sharding design may introduce an updated `BlobSidecar` to identify the shard, but does not affect the `BeaconBlock` structure. diff --git a/specs/deneb/validator.md b/specs/deneb/validator.md index 45a8e5c834..b29330ce57 100644 --- a/specs/deneb/validator.md +++ b/specs/deneb/validator.md @@ -52,7 +52,6 @@ def get_blobs_and_kzg_commitments(payload_id: PayloadId) -> Tuple[Sequence[BLSFi ## Beacon chain responsibilities All validator responsibilities remain unchanged other than those noted below. -Namely, the blob handling and the addition of `SignedBeaconBlockAndBlobsSidecar`. ### Block and sidecar proposal From f23ed0cdbc4d7590212653a460eb709350e7ed37 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Thu, 16 Feb 2023 21:11:18 +0800 Subject: [PATCH 17/22] Make linter happy --- setup.py | 2 +- specs/deneb/fork-choice.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/setup.py b/setup.py index 031ac7457c..9c5488f126 100644 --- a/setup.py +++ b/setup.py @@ -653,7 +653,7 @@ def preparations(cls): @classmethod def sundry_functions(cls) -> str: return super().sundry_functions() + '\n\n' + ''' -def retrieve_blobs_and_proofs(slot: Slot, beacon_block_root: Root) -> PyUnion[(Blob, KZGProof), (str, str)]: +def retrieve_blobs_and_proofs(beacon_block_root: Root) -> PyUnion[Tuple[Blob, KZGProof], Tuple[str, str]]: # pylint: disable=unused-argument return ("TEST", "TEST")''' diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index 2757e591ef..8fa08357a7 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -54,7 +54,7 @@ def is_data_available(beacon_block_root: Root, blob_kzg_commitments: Sequence[KZ # For testing, `retrieve_blobs_and_proofs` returns ("TEST", "TEST"). # TODO: Remove it once we have a way to inject `BlobSidecar` into tests. - if isinstance(blobs, str): + if isinstance(blobs, str) or isinstance(proofs, str): return True validate_blobs(blob_kzg_commitments, blobs, proofs) From a7e45db9ac2b60a33e144444969ad3ac0aae3d4c Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Thu, 16 Feb 2023 22:09:57 +0800 Subject: [PATCH 18/22] Fix `verify_kzg_proof_batch` and the tests --- specs/deneb/fork-choice.md | 4 +-- specs/deneb/polynomial-commitments.md | 9 ++++--- ...lobs_sidecar.py => test_validate_blobs.py} | 26 ++++++++++--------- 3 files changed, 21 insertions(+), 18 deletions(-) rename tests/core/pyspec/eth2spec/test/deneb/unittests/fork_choice/{test_validate_blobs_sidecar.py => test_validate_blobs.py} (54%) diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index 8fa08357a7..e93eb54faf 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -33,9 +33,7 @@ def validate_blobs(expected_kzg_commitments: Sequence[KZGCommitment], assert len(expected_kzg_commitments) == len(blobs) assert len(blobs) == len(proofs) - # Clients MAY use `verify_blob_kzg_proof_multi` for efficiency - for commitment, blob, proof in zip(expected_kzg_commitments, blobs, proofs): - assert verify_blob_kzg_proof(commitment, blob, proof) + assert verify_blob_kzg_proof_batch(blobs, expected_kzg_commitments, proofs) ``` #### `is_data_available` diff --git a/specs/deneb/polynomial-commitments.md b/specs/deneb/polynomial-commitments.md index afcf934fc7..76affe6200 100644 --- a/specs/deneb/polynomial-commitments.md +++ b/specs/deneb/polynomial-commitments.md @@ -411,15 +411,18 @@ def verify_kzg_proof_batch(commitments: Sequence[KZGCommitment], # Verify: e(sum r^i proof_i, [s]) == # e(sum r^i (commitment_i - [y_i]) + sum r^i z_i proof_i, [1]) proof_lincomb = g1_lincomb(proofs, r_powers) - proof_z_lincomb = g1_lincomb(proofs, [z * r_power for z, r_power in zip(zs, r_powers)]) + proof_z_lincomb = g1_lincomb( + proofs, + [BLSFieldElement((int(z) * int(r_power)) % BLS_MODULUS) for z, r_power in zip(zs, r_powers)], + ) C_minus_ys = [bls.add(bls.bytes48_to_G1(commitment), bls.multiply(bls.G1, BLS_MODULUS - y)) for commitment, y in zip(commitments, ys)] C_minus_y_as_KZGCommitments = [KZGCommitment(bls.G1_to_bytes48(x)) for x in C_minus_ys] C_minus_y_lincomb = g1_lincomb(C_minus_y_as_KZGCommitments, r_powers) return bls.pairing_check([ - [proof_lincomb, bls.neg(KZG_SETUP_G2[1])], - [bls.add(C_minus_y_lincomb, proof_z_lincomb), bls.G2] + [bls.bytes48_to_G1(proof_lincomb), bls.neg(bls.bytes96_to_G2(KZG_SETUP_G2[1]))], + [bls.add(bls.bytes48_to_G1(C_minus_y_lincomb), bls.bytes48_to_G1(proof_z_lincomb)), bls.G2] ]) ``` diff --git a/tests/core/pyspec/eth2spec/test/deneb/unittests/fork_choice/test_validate_blobs_sidecar.py b/tests/core/pyspec/eth2spec/test/deneb/unittests/fork_choice/test_validate_blobs.py similarity index 54% rename from tests/core/pyspec/eth2spec/test/deneb/unittests/fork_choice/test_validate_blobs_sidecar.py rename to tests/core/pyspec/eth2spec/test/deneb/unittests/fork_choice/test_validate_blobs.py index 87ed9ff8ea..d9934c5ade 100644 --- a/tests/core/pyspec/eth2spec/test/deneb/unittests/fork_choice/test_validate_blobs_sidecar.py +++ b/tests/core/pyspec/eth2spec/test/deneb/unittests/fork_choice/test_validate_blobs.py @@ -16,7 +16,7 @@ ) -def _run_validate_blobs_sidecar_test(spec, state, blob_count): +def _run_validate_blobs(spec, state, blob_count): block = build_empty_block_for_next_slot(spec, state) opaque_tx, blobs, blob_kzg_commitments = get_sample_opaque_tx(spec, blob_count=blob_count) block.body.blob_kzg_commitments = blob_kzg_commitments @@ -24,30 +24,32 @@ def _run_validate_blobs_sidecar_test(spec, state, blob_count): block.body.execution_payload.block_hash = compute_el_block_hash(spec, block.body.execution_payload) state_transition_and_sign_block(spec, state, block) - blobs_sidecar = spec.get_blobs_sidecar(block, blobs) - expected_commitments = [spec.blob_to_kzg_commitment(blobs[i]) for i in range(blob_count)] - spec.validate_blobs_sidecar(block.slot, block.hash_tree_root(), expected_commitments, blobs_sidecar) + # Also test the proof generation in `get_blob_sidecars` + blob_sidecars = spec.get_blob_sidecars(block, blobs) + blobs = [sidecar.blob for sidecar in blob_sidecars] + kzg_proofs = [sidecar.kzg_proof for sidecar in blob_sidecars] + spec.validate_blobs(blob_kzg_commitments, blobs, kzg_proofs) @with_deneb_and_later @spec_state_test -def test_validate_blobs_sidecar_zero_blobs(spec, state): - _run_validate_blobs_sidecar_test(spec, state, blob_count=0) +def test_validate_blobs_zero_blobs(spec, state): + _run_validate_blobs(spec, state, blob_count=0) @with_deneb_and_later @spec_state_test -def test_validate_blobs_sidecar_one_blob(spec, state): - _run_validate_blobs_sidecar_test(spec, state, blob_count=1) +def test_validate_blobs_one_blob(spec, state): + _run_validate_blobs(spec, state, blob_count=1) @with_deneb_and_later @spec_state_test -def test_validate_blobs_sidecar_two_blobs(spec, state): - _run_validate_blobs_sidecar_test(spec, state, blob_count=2) +def test_validate_blobs_two_blobs(spec, state): + _run_validate_blobs(spec, state, blob_count=2) @with_deneb_and_later @spec_state_test -def test_validate_blobs_sidecar_max_blobs(spec, state): - _run_validate_blobs_sidecar_test(spec, state, blob_count=spec.MAX_BLOBS_PER_BLOCK) +def test_validate_blobs_max_blobs(spec, state): + _run_validate_blobs(spec, state, blob_count=spec.MAX_BLOBS_PER_BLOCK) From 9dd7d2ba2f5b9bb1ce50494d1971d483764afcb6 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 17 Feb 2023 11:59:56 -0700 Subject: [PATCH 19/22] fix Blob pluralization in a few places --- specs/deneb/p2p-interface.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 163522ec3a..8357776abb 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -201,7 +201,7 @@ No more than `MAX_REQUEST_BLOBS_SIDECARS * MAX_BLOBS_PER_BLOCK` may be requested The response MUST consist of zero or more `response_chunk`. Each _successful_ `response_chunk` MUST contain a single `BlobSidecar` payload. -Clients MUST support requesting sidecars since `minimum_request_epoch`, where `minimum_request_epoch = max(finalized_epoch, current_epoch - MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS, DENEB_FORK_EPOCH)`. If any root in the request content references a block earlier than `minimum_request_epoch`, peers MAY respond with error code `3: ResourceUnavailable` or not include the blob in the response. +Clients MUST support requesting sidecars since `minimum_request_epoch`, where `minimum_request_epoch = max(finalized_epoch, current_epoch - MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS, DENEB_FORK_EPOCH)`. If any root in the request content references a block earlier than `minimum_request_epoch`, peers MAY respond with error code `3: ResourceUnavailable` or not include the blob in the response. Clients MUST respond with at least one sidecar, if they have it. Clients MAY limit the number of blocks and sidecars in the response. @@ -233,7 +233,7 @@ The response is unsigned, i.e. `BlobSidecarsByRange`, as the signature of the be Before consuming the next response chunk, the response reader SHOULD verify the blob sidecar is well-formatted and correct w.r.t. the expected KZG commitments through `validate_blobs`. -`BlobSidecarsByRange` is primarily used to sync blobs that may have been missed on gossip and to sync within the `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` window. +`BlobSidecarsByRange` is primarily used to sync blobs that may have been missed on gossip and to sync within the `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` window. The request MUST be encoded as an SSZ-container. @@ -241,18 +241,18 @@ The response MUST consist of zero or more `response_chunk`. Each _successful_ `response_chunk` MUST contain a single `BlobSidecar` payload. Clients MUST keep a record of signed blobs sidecars seen on the epoch range -`[max(current_epoch - MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS, DENEB_FORK_EPOCH), current_epoch]` +`[max(current_epoch - MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS, DENEB_FORK_EPOCH), current_epoch]` where `current_epoch` is defined by the current wall-clock time, and clients MUST support serving requests of blobs on this range. -Peers that are unable to reply to blob sidecar requests within the `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` +Peers that are unable to reply to blob sidecar requests within the `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` epoch range SHOULD respond with error code `3: ResourceUnavailable`. Such peers that are unable to successfully reply to this range of requests MAY get descored or disconnected at any time. *Note*: The above requirement implies that nodes that start from a recent weak subjectivity checkpoint -MUST backfill the local blobs database to at least epoch `current_epoch - MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` -to be fully compliant with `BlobsSidecarsByRange` requests. +MUST backfill the local blobs database to at least epoch `current_epoch - MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` +to be fully compliant with `BlobSidecarsByRange` requests. *Note*: Although clients that bootstrap from a weak subjectivity checkpoint can begin participating in the networking immediately, other peers MAY From 54d2559eb54db217844d30098662e0be3c8d9e5c Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Sat, 18 Feb 2023 17:45:16 +0100 Subject: [PATCH 20/22] remove producer reorg on multi-blob * also, use root/index for uniqueness --- specs/deneb/p2p-interface.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 8357776abb..b11e61ef65 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -112,8 +112,7 @@ The following validations MUST pass before forwarding the `sidecar` on the netwo - _[IGNORE]_ The sidecar is from a slot greater than the latest finalized slot -- i.e. validate that `sidecar.slot > compute_start_slot_at_epoch(state.finalized_checkpoint.epoch)` - _[IGNORE]_ The blob's block's parent (defined by `sidecar.block_parent_root`) has been seen (via both gossip and non-gossip sources) (a client MAY queue blocks for processing once the parent block is retrieved). - _[REJECT]_ The proposer signature, `signed_blob_sidecar.signature`, is valid with respect to the `sidecar.proposer_index` pubkey. -- _[IGNORE]_ The sidecar is the only sidecar with valid signature received for the tuple `(sidecar.slot, sidecar.proposer_index, sidecar.index)`. - -- If full verification of the blob fails at a later processing stage, clients MUST clear the blob from this "seen" cache so as to allow a the valid blob to propagate. The next block producer MAY orphan the block if they have observed multiple blobs signed by the proposer for the same "seen" tuple. +- _[IGNORE]_ The sidecar is the only sidecar with valid signature received for the tuple `(sidecar.block_root, sidecar.index)`. - _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `block_parent_root`/`slot`). If the `proposer_index` cannot immediately be verified against the expected shuffling, the sidecar MAY be queued for later processing while proposers for the block's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message. From 4e2a9920f1e1f1f7496289161fa7d830bb5b1d0e Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Mon, 20 Feb 2023 12:15:53 +0100 Subject: [PATCH 21/22] Update specs/deneb/p2p-interface.md Co-authored-by: g11tech --- specs/deneb/p2p-interface.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index b11e61ef65..531161b4e9 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -70,7 +70,7 @@ class SignedBlobSidecar(Container): ```python class BlobIdentifier(Container): block_root: Root - index: uint64 + index: BlobIndex ``` ## The gossip domain: gossipsub From ac0ec660d39e9d2470c0efb6be8ad53815085833 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Mon, 20 Feb 2023 16:35:11 +0100 Subject: [PATCH 22/22] add parent validation requirement sama as block --- specs/deneb/p2p-interface.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 531161b4e9..040d594dd2 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -111,6 +111,7 @@ The following validations MUST pass before forwarding the `sidecar` on the netwo - _[IGNORE]_ The sidecar is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- i.e. validate that `sidecar.slot <= current_slot` (a client MAY queue future blocks for processing at the appropriate slot). - _[IGNORE]_ The sidecar is from a slot greater than the latest finalized slot -- i.e. validate that `sidecar.slot > compute_start_slot_at_epoch(state.finalized_checkpoint.epoch)` - _[IGNORE]_ The blob's block's parent (defined by `sidecar.block_parent_root`) has been seen (via both gossip and non-gossip sources) (a client MAY queue blocks for processing once the parent block is retrieved). +- _[REJECT]_ The blob's block's parent (defined by `sidecar.block_parent_root`) passes validation. - _[REJECT]_ The proposer signature, `signed_blob_sidecar.signature`, is valid with respect to the `sidecar.proposer_index` pubkey. - _[IGNORE]_ The sidecar is the only sidecar with valid signature received for the tuple `(sidecar.block_root, sidecar.index)`. - _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `block_parent_root`/`slot`).