Interop Testing #33 | April 14 2025

[marioevz]

Interop Testing #33 | April 14 2025

• April 14, 2025, 14:00 UTC
• 60 Minutes
• Recurring meeting : true
• Call series : Interop Testing
• Occurrence rate : weekly
• Already on Ethereum Calendar : true
• Need YouTube stream links : true

Agenda

• Pectra Testing
• PeerDAS Testing
• EOF Testing

Facilitator email: mario.vega@ethereum.org

[github-actions]

Discourse Topic ID: 23458

• Action: Created at 2025-04-10 20:59 UTC
• URL: https://ethereum-magicians.org/t/23458

Recurring Zoom Meeting Created

YouTube Stream Links:

• Stream 1 (Apr 14, 2025): https://youtube.com/watch?v=qLUYr1QPU_0

Calendar Event

• Meeting already on Ethereum Calendar

RSS Feed Updated

• Zoom details sent to: mario.vega@ethereum.org

Telegram Notification

• Sent message to Ethereum Protocol Updates channel

[spencer-tb]

Note this isn't critical, but an interesting case.

We'd like to quickly discuss the modified withdrawal code contract test cases: https://github.com/ethereum/execution-spec-tests/blob/main/tests/prague/eip7002_el_triggerable_withdrawals/test_modified_withdrawal_contract.py

• Besu/Reth error (with a stack underflow) calling the modified contract code correctly producing an invalid block.
• Geth/Erigon pass calling the modified contract code correctly producing a valid block.

I say both are correct via the system contract changes here:
ethereum/EIPs#9508

1. Do clients care about looking deeper? It's a scenario that can never happen but maybe make things more robust
2. Should we skip the tests in the next release?

[marioevz]

I wanted to bring up negative testing in the execution layer.

We have a branch in EEST that uses an exception mapper per client in order to validate that the errors that are hit by the clients during the consumption of a negative test, and we found 2 client issues after enabling it.

We want to discuss path forward because this entails:

• Creating and maintaining a list of exception messages per client.
• Test failures when the client updates exception messages.

[akashkshirsagar31]

Interop Testing Call 33 Live on X/Twitter: https://twitter.com/i/broadcasts/1RDGlzwjkeDxL

[danceratopz]

JICYMI: The Hive Pectra dashboard is orchestrated by a Github Action, s/o @skylenet!

EL Client teams are welcome to make a PR with their best Pectra release candidate or keep their version as master; whatever is most appropriate.

Dashboard: https://hive.ethpandaops.io/pectra/index.html
Action: https://github.com/ethpandaops/pectra-devnets/blob/master/.github/workflows/hive.yaml
Example version bump: https://github.com/ethpandaops/pectra-devnets/pull/109/files

[enesinnc]

Do you share the meeting notes?

[marioevz]

Meeting notes thanks to @poojaranjan

Pectra Updates

Barnabas

• Using MEV boost and have MEV builders.
• Will begin testing for flow.

Marius

• Re-reviewed Pectra tests due to concerns about insufficient coverage.
• Discovered 23 edge cases for EL and CL; addressing them.
• Confident in test coverage now.
• Developing scripts to test 7702 edge cases and validate pool health.
• Spencer and Justin working on new test cases.
• PR with more tests: #4265

Mario

• Tests once written should be executed.
• On EEST, working on postmortem report of testing.
• There should be a writeup and will include the edge cases so we don't miss them in future. Its WIP.

Enrico

• Raised issues with MEV-Boost v1.9-rc3.
• Working on SSZ compiled test plans.

Justin

• Will support debugging MEV-Boost issue.
• Tests will be available after nightly build merge.
• Asked if CL clients can bypass spec tests?

Terence / Parithosh / Pawan

• No bypass currently, but support is possible or being looked into.
• No clear answer on model-based testing yet.

Modified Withdrawal Contract Test Cases

Spencer

• Felix created a test modifying the withdrawal contract.
• Besu/Reth throw invalid block; Geth/Erigon accept it.
• Wants feedback from clients on whether this is a concern.

Daniel (Besu)

• Considered theoretical; low priority for now.

Som (Erigon)

• Erigon continues block producing silently.

Roman

• Issue with assumption about EIP constraints being handled by the system contract.

Spencer

• Will follow up only if other clients express interest.

Negative Testing in Execution Layer

Mario

• Found a few issues last week. Some exceptions were noticeable and not hitting the expected handler
• There was mismatch in receipt or gas use and they didn't hit the right handler
• Several exceptions weren’t caught as expected.
• Question - do we want to make exception mapping for clients as a mandatory check mandatory?
• Shared link: Invalid Blocks Notes

Lukasz

• Exception list changes infrequent at Nethermind.

Danno

• My concern is this brings exceptions into the protocol now. Before this what mattered was that it was rejected, not why it was rejected. If we are testing error strings we de facto make them part of the protocol
• Prefer flexibility in client behavior.

Jochem

• Warned against restricting implementation optimizations.

Mario / Lukasz

• We don't have to make it part of Protocol but just testing.
• Multiple exceptions can be accepted.
• Shared mapper examples

Attestation Metrics

Nflaig

• 7-day metrics improved but not perfect.
• Asked if another attestation analysis was needed for attestation performace or wait for the stable release?

Enrico (Besu) / Terence (Prysm)

• Current metrics are fine.

Bot Updates

Parithosh

• Introduced a new bot for building client images.
  Discord Link

PeerDAS Testing

Barnabas

• Devnet 5 ongoing; 75% participation.
• Issues with Reth noted.

Roman (Reth)

• Degraded block processing due to resource exhaustion.

Parithosh

• Machines had only 8GB RAM; resource usage expected to grow.

Rafael

• Asked if issues were on supernodes (confirmed yes).

Barnabas

• MEV workflow functional with Fulu.

Justin

• Proposed BPO fork.
• Shared thread

Marius / Justin

• I think they should just be normal hf on EL tbh. Compared to past forks (Glacier).
• EL side manageable, CL more complex. EL thread

Will / Manu

• PeerDAS discussion item for next day.

EOF

General

• Devnet 0 completed with clean fuzzing.
• Devnet 1 pending merge; only one PR left.
• Clients like Besu, EVM1 ready.

Marius

• Awaiting a stable devnet 1 spec.

Parithosh

• Shared devnet spec: EOF Devnet 1

Roman (Reth)

• Shared implementation branch: PR #2377

Barnabas

• Autobuilding the Reth branch moving forward.

Jochem (EthJS)

• Working on implementation; shared plans to publish the branch.

Som (Erigon)

• Implementation WIP; dev unavailable currently.