Skip to content

[Tracking] Hive ethereum/engine simulation #1285

New issue
New issue
Open
Open
[Tracking] Hive ethereum/engine simulation#1285
Assignees
fedacking
Labels
L1Ethereum clientef-testsHive tests, execution-spec-tests
@mpaulucci

Description

@mpaulucci
mpaulucci
opened on Nov 26, 2024

These are the failing tests

make run-hive SIMULATION=ethereum/engine TEST_PATTERN="engine-cancun"
cat $(ls -t hive/workspace/logs/*-*.json | head -n1) | jq '.testCases[] | select(.summaryResult.pass == false) | .name'
  • Invalid NewPayload, Incomplete Transactions, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction Signature, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction Signature, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction Nonce, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction Nonce, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction GasPrice, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction GasPrice, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction GasTipCapPrice, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction Gas, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction Gas, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction Value, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction Value, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction ChainID, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction ChainID, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid Missing Ancestor ReOrg, StateRoot, EmptyTxs=False, Invalid P1
  • Invalid Missing Ancestor ReOrg, StateRoot, EmptyTxs=True, Invalid P1
  • Invalid Missing Ancestor ReOrg, StateRoot, EmptyTxs=False, Invalid P9
  • Invalid Missing Ancestor ReOrg, StateRoot, EmptyTxs=True, Invalid P9
  • Sidechain Reorg
  • Re-Org Back to Canonical Chain From Syncing Chain
  • Re-Org Back into Canonical Chain, Depth=10, Execute Side Payload on Re-Org
  • Invalid PayloadAttributes, Missing BeaconRoot, Syncing=True
  • Invalid NewPayload, ParentBeaconBlockRoot, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, BlobGasUsed, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Blob Count on BlobGasUsed, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, ExcessBlobGas, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, VersionedHashes, Syncing=False, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, VersionedHashes, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, VersionedHashes Version, Syncing=False, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, VersionedHashes Version, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Incomplete VersionedHashes, Syncing=False, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Incomplete VersionedHashes, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Extra VersionedHashes, Syncing=False, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Extra VersionedHashes, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Request Blob Pooled Transactions Single
  • Request Blob Pooled Transactions Multiple
  • Bad Hash on NewPayload (Syncing=true, Sidechain=false)
  • Bad Hash on NewPayload (Syncing=true, Sidechain=true)
  • Unknown HeadBlockHash
  • Invalid PayloadAttributes, Zero timestamp, Syncing=True
  • Blob Transaction Ordering, Multiple Clients
  • Invalid PayloadAttributes, Parent timestamp, Syncing=True
  • In-Order Consecutive Payload Execution
  • Valid NewPayload->ForkchoiceUpdated on Syncing Client
  • Invalid NewPayload, ParentHash, Syncing=False, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, ParentHash, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, StateRoot, Syncing=True, EmptyTxs=True, DynFeeTxs=False
  • Invalid NewPayload, StateRoot, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, ReceiptsRoot, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Number, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, GasLimit, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, GasUsed, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Timestamp, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, PrevRandao, Syncing=True, EmptyTxs=False, DynFeeTxs=False
make run-hive SIMULATION=ethereum/engine TEST_PATTERN="engine-withdrawals"
cat $(ls -t hive/workspace/logs/*-*.json | head -n1) | jq '.testCases[] | select(.summaryResult.pass == false) | .name'

Currently failing tests:

  • Empty Withdrawals #1605
  • Corrupted Block Hash Payload (INVALID) #1608
  • Sync after 2 blocks - Withdrawals on Block 1 - Single Withdrawal Account - No Transactions
  • Sync after 2 blocks - Withdrawals on Block 1 - Single Withdrawal Account
  • Sync after 2 blocks - Withdrawals on Block 2 - Multiple Withdrawal Accounts - No Transactions
  • Sync after 2 blocks - Withdrawals on Block 2 - Multiple Withdrawal Accounts
  • Sync after 128 blocks - Withdrawals on Block 2 - Multiple Withdrawal Accounts
  • Withdrawals Fork on Block 1 - 1 Block Re-Org
  • Withdrawals Fork On Genesis
  • Withdrawals Fork on Block 1 - 8 Block Re-Org NewPayload
  • Withdrawals Fork on Block 1 - 8 Block Re-Org, Sync
  • Withdrawals Fork on Block 8 - 10 Block Re-Org NewPayload
  • Withdrawals Fork on Block 8 - 10 Block Re-Org Sync
  • Withdrawals Fork on Canonical Block 8 / Side Block 7 - 10 Block Re-Org
  • Withdrawals Fork on Canonical Block 8 / Side Block 7 - 10 Block Re-Org Sync
  • Withdrawals Fork on Canonical Block 8 / Side Block 9 - 10 Block Re-Org
  • Withdrawals Fork on Canonical Block 8 / Side Block 9 - 10 Block Re-Org Sync
  • GetPayloadBodiesByRange
  • Withdrawals Fork on Block 1
  • GetPayloadBodies After Sync
  • GetPayloadBodiesByRange (Sidechain)
  • GetPayloadBodiesByRange (Empty Transactions/Withdrawals)
  • GetPayloadBodiesByHash
  • GetPayloadBodiesByHash (Empty Transactions/Withdrawals)
  • GetPayloadBodies Parallel
  • Withdrawals Fork on Block 2
  • Withdrawals Fork on Block 3
  • Withdraw to a single account
  • Withdraw to two accounts
  • Withdraw many accounts
  • Withdraw zero amount
make run-hive SIMULATION=ethereum/engine TEST_PATTERN="engine-api"
cat $(ls -t hive/workspace/logs/*-*.json | head -n1) | jq '.testCases[] | select(.summaryResult.pass == false) | .name'

Current failing tests:

  • Invalid Missing Ancestor Syncing ReOrg, Transaction Signature, EmptyTxs=False, CanonicalReOrg=True, Invalid P9
  • Invalid Missing Ancestor Syncing ReOrg, Transaction Nonce, EmptyTxs=False, CanonicalReOrg=True, Invalid P9
  • Invalid Missing Ancestor Syncing ReOrg, Transaction Gas, EmptyTxs=False, CanonicalReOrg=True, Invalid P9
  • Invalid Missing Ancestor Syncing ReOrg, Transaction GasPrice, EmptyTxs=False, CanonicalReOrg=True, Invalid P9
  • Invalid Missing Ancestor Syncing ReOrg, Transaction Value, EmptyTxs=False, CanonicalReOrg=True, Invalid P9
  • Sidechain Reorg
  • Re-Org Back to Canonical Chain From Syncing Chain
  • Re-Org Back into Canonical Chain, Depth=10, Execute Side Payload on Re-Org
  • Unknown HeadBlockHash
  • Invalid PayloadAttributes, Zero timestamp, Syncing=True
  • Invalid PayloadAttributes, Parent timestamp, Syncing=True
  • In-Order Consecutive Payload Execution
  • Valid NewPayload->ForkchoiceUpdated on Syncing Client
  • Invalid NewPayload, ParentHash, Syncing=False, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, ParentHash, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, StateRoot, Syncing=True, EmptyTxs=True, DynFeeTxs=False
  • Invalid NewPayload, StateRoot, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, ReceiptsRoot, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Bad Hash on NewPayload (Syncing=true, Sidechain=false)
  • Invalid NewPayload, Number, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, GasLimit, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, GasUsed, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Timestamp, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, PrevRandao, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Bad Hash on NewPayload (Syncing=true, Sidechain=true)
  • Invalid NewPayload, Incomplete Transactions, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction Signature, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction Signature, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction Nonce, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction Nonce, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction GasPrice, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction GasPrice, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction GasTipCapPrice, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction Gas, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction Gas, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction Value, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction Value, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid NewPayload, Transaction ChainID, Syncing=True, EmptyTxs=False, DynFeeTxs=False
  • Invalid NewPayload, Transaction ChainID, Syncing=True, EmptyTxs=False, DynFeeTxs=True
  • Invalid Missing Ancestor ReOrg, StateRoot, EmptyTxs=False, Invalid P1
  • Invalid Missing Ancestor ReOrg, StateRoot, EmptyTxs=True, Invalid P1
  • Invalid Missing Ancestor ReOrg, StateRoot, EmptyTxs=False, Invalid P9
  • Invalid Missing Ancestor ReOrg, StateRoot, EmptyTxs=True, Invalid P9
  • Invalid Missing Ancestor Syncing ReOrg, StateRoot, EmptyTxs=True, CanonicalReOrg=True, Invalid P9
  • Invalid Missing Ancestor Syncing ReOrg, StateRoot, EmptyTxs=False, CanonicalReOrg=True, Invalid P9
  • Invalid Missing Ancestor Syncing ReOrg, ReceiptsRoot, EmptyTxs=False, CanonicalReOrg=True, Invalid P8
  • Invalid Missing Ancestor Syncing ReOrg, GasLimit, EmptyTxs=False, CanonicalReOrg=True, Invalid P8
  • Invalid Missing Ancestor Syncing ReOrg, GasUsed, EmptyTxs=False, CanonicalReOrg=True, Invalid P8
  • Invalid Missing Ancestor Syncing ReOrg, Timestamp, EmptyTxs=False, CanonicalReOrg=True, Invalid P8
  • Invalid Missing Ancestor Syncing ReOrg, Incomplete Transactions, EmptyTxs=False, CanonicalReOrg=True, Invalid P9

Metadata

Metadata

Assignees

Labels

L1Ethereum clientef-testsHive tests, execution-spec-tests

Type

No type

Projects

ethrex_l1

Status

In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions