Merge branch 'dev' into JustinDrake-patch-7

Author: djrtwo

README.md

@@ -43,6 +43,7 @@ while the details are in review and may change.
   * [ethereum.org](https://ethereum.org) high-level description of the merge [here](https://ethereum.org/en/eth2/docking/)
 * Specifications:
   * [Beacon Chain changes](specs/merge/beacon-chain.md)
+  * [Merge fork](specs/merge/fork.md)
   * [Fork Choice changes](specs/merge/fork-choice.md)
   * [Validator additions](specs/merge/validator.md)
 

configs/mainnet.yaml

@@ -32,7 +32,7 @@ SHARDING_FORK_VERSION: 0x03000000
 SHARDING_FORK_EPOCH: 18446744073709551615
 
 # TBD, 2**32 is a placeholder. Merge transition approach is in active R&D.
-TRANSITION_TOTAL_DIFFICULTY: 4294967296
+MIN_ANCHOR_POW_BLOCK_DIFFICULTY: 4294967296
 
 
 # Time parameters

configs/minimal.yaml

@@ -31,7 +31,7 @@ SHARDING_FORK_VERSION: 0x03000001
 SHARDING_FORK_EPOCH: 18446744073709551615
 
 # TBD, 2**32 is a placeholder. Merge transition approach is in active R&D.
-TRANSITION_TOTAL_DIFFICULTY: 4294967296
+MIN_ANCHOR_POW_BLOCK_DIFFICULTY: 4294967296
 
 
 # Time parameters

setup.py

@@ -510,7 +510,7 @@ def sundry_functions(cls) -> str:
 
 def get_pow_block(hash: Bytes32) -> PowBlock:
     return PowBlock(block_hash=hash, is_valid=True, is_processed=True,
-                    total_difficulty=config.TRANSITION_TOTAL_DIFFICULTY)
+                    total_difficulty=uint256(0), difficulty=uint256(0))
 
 
 def get_execution_state(execution_state_root: Bytes32) -> ExecutionState:
@@ -871,6 +871,7 @@ def finalize_options(self):
                     specs/phase0/validator.md
                     specs/phase0/weak-subjectivity.md
                     specs/merge/beacon-chain.md
+                    specs/merge/fork.md
                     specs/merge/fork-choice.md
                     specs/merge/validator.md
                 """

specs/altair/p2p-interface.md

@@ -74,6 +74,23 @@ New topics are added in Altair to support the sync committees and the beacon blo
 
 The specification around the creation, validation, and dissemination of messages has not changed from the Phase 0 document.
 
+The derivation of the `message-id` has changed starting with Altair to incorporate the message `topic` along with the message `data`. These are fields of the `Message` Protobuf, and interpreted as empty byte strings if missing.
+The `message-id` MUST be the following 20 byte value computed from the message:
+* If `message.data` has a valid snappy decompression, set `message-id` to the first 20 bytes of the `SHA256` hash of
+  the concatenation of the following data: `MESSAGE_DOMAIN_VALID_SNAPPY`, the length of the topic byte string (encoded as little-endian `uint64`),
+  the topic byte string, and the snappy decompressed message data:
+  i.e. `SHA256(MESSAGE_DOMAIN_VALID_SNAPPY + uint_to_bytes(uint64(len(message.topic))) + message.topic + snappy_decompress(message.data))[:20]`.
+* Otherwise, set `message-id` to the first 20 bytes of the `SHA256` hash of
+  the concatenation of the following data: `MESSAGE_DOMAIN_INVALID_SNAPPY`, the length of the topic byte string (encoded as little-endian `uint64`),
+  the topic byte string, and the raw message data:
+  i.e. `SHA256(MESSAGE_DOMAIN_INVALID_SNAPPY + uint_to_bytes(uint64(len(message.topic))) + message.topic + message.data)[:20]`.
+
+Implementations may need to carefully handle the function that computes the `message-id`. In particular, messages on topics with the Phase 0
+fork digest should use the `message-id` procedure specified in the Phase 0 document.
+Messages on topics with the Altair fork digest should use the `message-id` procedure defined here.
+If an implementation only supports a single `message-id` function, it can define a switch inline;
+for example, `if topic in phase0_topics: return phase0_msg_id_fn(message) else return altair_msg_id_fn(message)`.
+
 The new topics along with the type of the `data` field of a gossipsub message are given in this table:
 
 | Name | Message Type |
@@ -144,6 +161,7 @@ The following validations MUST pass before forwarding the `sync_committee_messag
 - _[IGNORE]_ The signature's slot is for the current slot (with a MAXIMUM_GOSSIP_CLOCK_DISPARITY allowance), i.e. `sync_committee_message.slot == current_slot`.
 - _[IGNORE]_ The block being signed over (`sync_committee_message.beacon_block_root`) has been seen (via both gossip and non-gossip sources).
 - _[IGNORE]_ There has been no other valid sync committee signature for the declared `slot` for the validator referenced by `sync_committee_message.validator_index`.
+  Note this validation is _per topic_ so that for a given `slot`, multiple messages could be forwarded with the same `validator_index` as long as the `subnet_id`s are distinct.
 - _[REJECT]_ The `subnet_id` is valid for the given validator, i.e. `subnet_id in compute_subnets_for_sync_committee(state, sync_committee_message.validator_index)`.
   Note this validation implies the validator is part of the broader current sync committee along with the correct subcommittee.
 - _[REJECT]_ The `signature` is valid for the message `beacon_block_root` for the validator referenced by `validator_index`.

specs/merge/beacon-chain.md

@@ -14,8 +14,6 @@
 - [Custom types](#custom-types)
 - [Constants](#constants)
   - [Execution](#execution)
-- [Configuration](#configuration)
-  - [Merge](#merge)
 - [Containers](#containers)
   - [Extended containers](#extended-containers)
     - [`BeaconBlockBody`](#beaconblockbody)
@@ -31,11 +29,12 @@
   - [Misc](#misc)
     - [`compute_timestamp_at_slot`](#compute_timestamp_at_slot)
 - [Beacon chain state transition function](#beacon-chain-state-transition-function)
-- [Execution engine](#execution-engine)
-  - [`on_payload`](#on_payload)
+  - [Execution engine](#execution-engine)
+    - [`on_payload`](#on_payload)
   - [Block processing](#block-processing)
   - [Execution payload processing](#execution-payload-processing)
     - [`process_execution_payload`](#process_execution_payload)
+- [Initialize state for pure Merge testnets and test vectors](#initialize-state-for-pure-merge-testnets-and-test-vectors)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 <!-- /TOC -->
@@ -63,17 +62,6 @@ This patch adds transaction execution to the beacon chain as part of the merge.
 | `MAX_TRANSACTIONS_PER_PAYLOAD` | `uint64(2**14)` (= 16,384) |
 | `BYTES_PER_LOGS_BLOOM` | `uint64(2**8)` (= 256) |
 
-## Configuration
-
-### Merge
-
-*Note*: The configuration value `MERGE_FORK_EPOCH` is not final.
-
-| Name | Value |
-| - | - |
-| `MERGE_FORK_VERSION` | `Version('0x02000000')` |
-| `MERGE_FORK_EPOCH` | `Epoch(18446744073709551615)` |
-
 ## Containers
 
 ### Extended containers
@@ -234,3 +222,63 @@ def process_execution_payload(state: BeaconState, body: BeaconBlockBody, executi
         transactions_root=hash_tree_root(payload.transactions),
     )
 ```
+
+## Initialize state for pure Merge testnets and test vectors
+
+This helper function is only for initializing the state for pure Merge testnets and tests.
+
+*Note*: The function `initialize_beacon_state_from_eth1` is modified: (1) using `MERGE_FORK_VERSION` as the current fork version, (2) utilizing the Merge `BeaconBlockBody` when constructing the initial `latest_block_header`, and (3) adding initial `latest_execution_payload_header`.
+
+```python
+def initialize_beacon_state_from_eth1(eth1_block_hash: Bytes32,
+                                      eth1_timestamp: uint64,
+                                      deposits: Sequence[Deposit]) -> BeaconState:
+    fork = Fork(
+        previous_version=GENESIS_FORK_VERSION,
+        current_version=MERGE_FORK_VERSION,  # [Modified in Merge]
+        epoch=GENESIS_EPOCH,
+    )
+    state = BeaconState(
+        genesis_time=eth1_timestamp + GENESIS_DELAY,
+        fork=fork,
+        eth1_data=Eth1Data(block_hash=eth1_block_hash, deposit_count=uint64(len(deposits))),
+        latest_block_header=BeaconBlockHeader(body_root=hash_tree_root(BeaconBlockBody())),
+        randao_mixes=[eth1_block_hash] * EPOCHS_PER_HISTORICAL_VECTOR,  # Seed RANDAO with Eth1 entropy
+    )
+
+    # Process deposits
+    leaves = list(map(lambda deposit: deposit.data, deposits))
+    for index, deposit in enumerate(deposits):
+        deposit_data_list = List[DepositData, 2**DEPOSIT_CONTRACT_TREE_DEPTH](*leaves[:index + 1])
+        state.eth1_data.deposit_root = hash_tree_root(deposit_data_list)
+        process_deposit(state, deposit)
+
+    # Process activations
+    for index, validator in enumerate(state.validators):
+        balance = state.balances[index]
+        validator.effective_balance = min(balance - balance % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE)
+        if validator.effective_balance == MAX_EFFECTIVE_BALANCE:
+            validator.activation_eligibility_epoch = GENESIS_EPOCH
+            validator.activation_epoch = GENESIS_EPOCH
+
+    # Set genesis validators root for domain separation and chain versioning
+    state.genesis_validators_root = hash_tree_root(state.validators)
+
+    # [New in Merge] Construct execution payload header
+    # Note: initialized with zero block height
+    state.latest_execution_payload_header = ExecutionPayloadHeader(
+        block_hash=eth1_block_hash,
+        parent_hash=Hash32(),
+        coinbase=Bytes20(),
+        state_root=Bytes32(),
+        number=uint64(0),
+        gas_limit=uint64(0),
+        gas_used=uint64(0),
+        timestamp=eth1_timestamp,
+        receipt_root=Bytes32(),
+        logs_bloom=ByteVector[BYTES_PER_LOGS_BLOOM](),
+        transactions_root=Root(),
+    )
+
+    return state
+```

specs/merge/fork-choice.md

@@ -12,13 +12,13 @@
   - [`ExecutionEngine`](#executionengine)
     - [`set_head`](#set_head)
     - [`finalize_block`](#finalize_block)
-- [Containers](#containers)
-    - [`PowBlock`](#powblock)
-- [Helper functions](#helper-functions)
-    - [`get_pow_block`](#get_pow_block)
-    - [`is_valid_transition_block`](#is_valid_transition_block)
+- [Helpers](#helpers)
+  - [`TransitionStore`](#transitionstore)
+  - [`PowBlock`](#powblock)
+  - [`get_pow_block`](#get_pow_block)
+  - [`is_valid_terminal_pow_block`](#is_valid_terminal_pow_block)
 - [Updated fork-choice handlers](#updated-fork-choice-handlers)
-    - [`on_block`](#on_block)
+  - [`on_block`](#on_block)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 <!-- /TOC -->
@@ -66,44 +66,52 @@ def finalize_block(self: ExecutionEngine, block_hash: Hash32) -> bool:
     ...
 ```
 
-## Containers
+## Helpers
 
-#### `PowBlock`
+### `TransitionStore`
 
 ```python
-class PowBlock(Container):
+@dataclass
+class TransitionStore(object):
+    transition_total_difficulty: uint256
+```
+
+### `PowBlock`
+
+```python
+@dataclass
+class PowBlock(object):
     block_hash: Hash32
     is_processed: boolean
     is_valid: boolean
     total_difficulty: uint256
+    difficulty: uint256
 ```
 
-## Helper functions
-
-#### `get_pow_block`
+### `get_pow_block`
 
 Let `get_pow_block(block_hash: Hash32) -> PowBlock` be the function that given the hash of the PoW block returns its data.
 
 *Note*: The `eth_getBlockByHash` JSON-RPC method does not distinguish invalid blocks from blocks that haven't been processed yet. Either extending this existing method or implementing a new one is required.
 
-#### `is_valid_transition_block`
+### `is_valid_terminal_pow_block`
 
 Used by fork-choice handler, `on_block`.
 
 ```python
-def is_valid_transition_block(block: PowBlock) -> bool:
-    is_total_difficulty_reached = block.total_difficulty >= TRANSITION_TOTAL_DIFFICULTY
+def is_valid_terminal_pow_block(transition_store: TransitionStore, block: PowBlock) -> bool:
+    is_total_difficulty_reached = block.total_difficulty >= transition_store.transition_total_difficulty
     return block.is_valid and is_total_difficulty_reached
 ```
 
 ## Updated fork-choice handlers
 
-#### `on_block`
+### `on_block`
 
 *Note*: The only modification is the addition of the verification of transition block conditions.
 
 ```python
-def on_block(store: Store, signed_block: SignedBeaconBlock) -> None:
+def on_block(store: Store, signed_block: SignedBeaconBlock, transition_store: TransitionStore=None) -> None:
     block = signed_block.message
     # Parent block must be known
     assert block.parent_root in store.block_states
@@ -119,11 +127,11 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None:
     assert get_ancestor(store, block.parent_root, finalized_slot) == store.finalized_checkpoint.root
 
     # [New in Merge]
-    if is_transition_block(pre_state, block):
+    if (transition_store is not None) and is_merge_block(pre_state, block):
         # Delay consideration of block until PoW block is processed by the PoW node
         pow_block = get_pow_block(block.body.execution_payload.parent_hash)
         assert pow_block.is_processed
-        assert is_valid_transition_block(pow_block)
+        assert is_valid_terminal_pow_block(transition_store, pow_block)
 
     # Check the block is valid and compute the post-state
     state = pre_state.copy()