feat: add support for EEST blockchain test fixtures by jsign · Pull Request #286 · eth-act/zkevm-benchmark-workload

jsign · 2026-05-21T17:10:06Z

This PR:

Adds support in ere-hosts to directly consume EEST tests new format from execution-specs@projects/zkevm. It is the usual fixture format, but for each block contains statelessInputBytes and statelessOutputBytes.
The runner automatically can detect both workload fixture formats: current one (now called Legacy, and new one Eest). i.e. this is not a breaking change.
If Eest fixtures are detected, we directly use stateless{Input|Output}Bytes for the guest program run (i.e. running EEST fixtures can be done directly without a previously required witness-generator-cli).
The zkevm-metrics metadata for an EEST run contains more details to easily map to original EEST file.

Example run output in zkevm-metrics:

{
  "name": "eest__tests_amsterdam_eip8025_optional_proofs_test_witness_state_writes_py_test_witness_state_reverted_sstore_still_contains_storage_proof_fork_Amsterdam-blockchain_test__block0",
  "timestamp_completed": "2026-05-21T17:48:02.645237188Z",
  "metadata": {
    "block_index": 0,
    "block_number": 1,
    "block_used_gas": 26012,
    "chain_id": 1,
    "fixture_format": "eest",
    "network": "Amsterdam",
    "original_test_name": "tests/amsterdam/eip8025_optional_proofs/test_witness_state_writes.py::test_witness_state_reverted_sstore_still_contains_storage_proof[fork_Amsterdam-blockchain_test]",
    "source_path": "blockchain_tests/for_amsterdam/amsterdam/eip8025_optional_proofs/witness_state_writes/witness_state_reverted_sstore_still_contains_storage_proof.json"
  },
  "execution": {
    "crashed": {
      "reason": "zkVM method error: Emulator panicked: Mem::read() section not found for addr: 0=0 with width: 8"
    }
  }
}

…dator

…ing and logging

jsign · 2026-05-21T17:10:51Z

The previous crates/benchmark-runner/src/stateless_validator.rs file was getting a bit loaded. I moved to a module with a bit more organized structure.

I'll simply comment on the new stuff.

jsign · 2026-05-21T17:13:38Z

+#[derive(Debug, Clone)]
+pub(crate) struct EestStatelessFixture {


Now we have an enum for BenchmarkFixture:

#[derive(Debug, Clone)] pub(crate) enum BenchmarkFixture { Legacy(StatelessValidationFixture), Eest(EestStatelessFixture), }

StatelessValidationFixture is the usual format. This new EestStatelssFixture is the one coming from EEST variant. It mainly contains basic data extracted from parsing EEST fixture that will be also used as Metadata for zkevm-metrics outputs.

Example of output:

{ "name": "eest__tests_amsterdam_eip8025_optional_proofs_test_witness_state_writes_py_test_witness_state_reverted_sstore_still_contains_storage_proof_fork_Amsterdam-blockchain_test__block0", "timestamp_completed": "2026-05-21T17:48:02.645237188Z", "metadata": { "block_index": 0, "block_number": 1, "block_used_gas": 26012, "chain_id": 1, "fixture_format": "eest", "network": "Amsterdam", "original_test_name": "tests/amsterdam/eip8025_optional_proofs/test_witness_state_writes.py::test_witness_state_reverted_sstore_still_contains_storage_proof[fork_Amsterdam-blockchain_test]", "source_path": "blockchain_tests/for_amsterdam/amsterdam/eip8025_optional_proofs/witness_state_writes/witness_state_reverted_sstore_still_contains_storage_proof.json" }, "execution": { "crashed": { "reason": "zkVM method error: Emulator panicked: Mem::read() section not found for addr: 0=0 with width: 8" } } }

jsign · 2026-05-21T17:15:14Z

+    pub(crate) stateless_output_bytes: Vec<u8>,
+}
+
+#[derive(Debug, Deserialize)]


EEST fixture parsing structs.

jsign · 2026-05-21T19:55:16Z

+#[derive(Debug, Clone)]
+pub(crate) enum BenchmarkFixture {
+    Legacy(Box<StatelessValidationFixture>),
+    Eest(EestStatelessFixture),
+}


Note now BenchmarkFixture is an enum.

jsign · 2026-05-21T19:56:08Z

+fn load_benchmark_fixtures(path: &Path, input_root: &Path) -> Result<Vec<BenchmarkFixture>> {
+    let content = std::fs::read(path)?;
+    let value: serde_json::Value = serde_json::from_slice(&content)
+        .with_context(|| format!("Failed to parse {}", path.display()))?;
+
+    if value.get("stateless_input").is_some() {
+        let fixture = serde_json::from_value(value)
+            .with_context(|| format!("Failed to parse legacy fixture {}", path.display()))?;
+        return Ok(vec![BenchmarkFixture::Legacy(Box::new(fixture))]);
+    }
+
+    load_eest_benchmark_fixtures(value, path, input_root)
+        .map(|fixtures| fixtures.into_iter().map(BenchmarkFixture::Eest).collect())
+}


The automatic detection if we are in the legacy or eest format.

jsign · 2026-05-21T19:57:15Z

As said, mostly refactoring to make it more modular.

jsign · 2026-05-21T19:58:31Z

+pub(crate) fn stateless_validator_input_from_fixture(
+    fixture: BenchmarkFixture,
+    el: ExecutionClient,
+) -> Result<Box<dyn GuestFixture>> {
+    match fixture {
+        BenchmarkFixture::Legacy(fixture) => match el {
+            ExecutionClient::Reth => reth_input_from_fixture(*fixture),
+            ExecutionClient::Ethrex => ethrex_input_from_fixture(*fixture),
+        },
+        BenchmarkFixture::Eest(fixture) => raw_eest_input_from_fixture(fixture),
+    }
+}


Gist-ish of the PR: depending on the BenchmarkFixture variant, we do the usual for Legacy, or call new raw_eest-input_from_fixture(...).

jsign · 2026-05-21T20:00:18Z

+        block_number: fixture.block_number,
+        block_used_gas: fixture.block_used_gas,
+    };
+    let expected_public_values = Sha256::digest(fixture.stateless_output_bytes).to_vec();


Nit: here we explicitly Sha256 for now. But I think in theory, zkVMs are expected to output raw bytes that the guest program wrote to output.

@han0110, do you know if this is something we could assume in new Zisk v0.18.0? Or maybe we should still keep this Sha256-ing for a while.

But I think in theory, zkVMs are expected to output raw bytes that the guest program wrote to output.

Yes but currently only SP1 and RISC Zero support that, others have restricted size:

Airbender - 32 bytes

OpenVM - 32 bytes (configurable)

ZisK - 256 bytes

So I think we have to stick with the sha256 output for a while, or we drop the support of Airbender in ere-guests and configure OpenVM to support 256 bytes as well (assuming the public values will not exceed 256 bytes, where currently it's 41 bytes)

Cool, let's keep the Sha256, and we can see how things evolve.

@marcinbugaj, is this something Zisk is planning to support? Since I think from zkEVM standards POV it should be required the return exactly what the guest program wrote as output?

han0110

LGTM! (looks like the fixture name is a bit tricky to handle when needing to use it for file name)

han0110 · 2026-05-22T03:27:51Z

+        block_number: fixture.block_number,
+        block_used_gas: fixture.block_used_gas,
+    };
+    let expected_public_values = Sha256::digest(fixture.stateless_output_bytes).to_vec();


But I think in theory, zkVMs are expected to output raw bytes that the guest program wrote to output.

Yes but currently only SP1 and RISC Zero support that, others have restricted size:

Airbender - 32 bytes

OpenVM - 32 bytes (configurable)

ZisK - 256 bytes

So I think we have to stick with the sha256 output for a while, or we drop the support of Airbender in ere-guests and configure OpenVM to support 256 bytes as well (assuming the public values will not exceed 256 bytes, where currently it's 41 bytes)

… fixture formats

jsign · 2026-05-22T12:51:57Z

(looks like the fixture name is a bit tricky to handle when needing to use it for file name)

Yeah, it is a bit tricky not to pass 256 in length. Before, I used a shorter style but had to add a hash to the name to avoid potential collisions, which was ugly. I think it should be fine. If we discover it violates the length max, we should prob switch to that idea too.
For UI-related stuff, it should use the metadata field in the JSON result, which is the original "nice" name (i.e., not rely on the fixture result file name).

jsign · 2026-05-22T13:09:20Z

Might leave this open for some extra mins/hours -- I'm using this branch to test a dashboard automation reg EEST fixtures direct run.

jsign added 4 commits May 21, 2026 14:07

feat: add support for EEST blockchain test fixtures in stateless vali…

8e03615

…dator

lints

20eaf4f

feat: enhance EEST support by adding blockchain tests directory handl…

9907839

…ing and logging

feat: improve EEST fixture naming by ensuring uniqueness and truncation

574404a

jsign commented May 21, 2026

View reviewed changes

jsign marked this pull request as ready for review May 21, 2026 20:00

jsign requested a review from han0110 May 21, 2026 20:00

han0110 approved these changes May 22, 2026

View reviewed changes

jsign added 2 commits May 22, 2026 09:39

improve eest folder-style detection to avoid fillings with non-target…

a8ed377

… fixture formats

review feedback

00a8cb7

jsign merged commit 02758cd into master May 22, 2026
5 checks passed

		#[derive(Debug, Clone)]
		pub(crate) struct EestStatelessFixture {

Conversation

jsign commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsign May 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

han0110 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsign commented May 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jsign commented May 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jsign commented May 21, 2026 •

edited

Loading

jsign May 22, 2026 •

edited

Loading

jsign commented May 22, 2026 •

edited

Loading