Skip to content

Conversation

@LouisTsai-Csie
Copy link
Collaborator

@LouisTsai-Csie LouisTsai-Csie commented Jan 23, 2026

🗒️ Description

The original blobhash benchmark is failing in every mode, this PR supports it under fill mode with --gas-benchmark-values and --fixed-opcode-count flag. But it still needs some update to support execute remote, since now it does not support blob transactions.

🔗 Related Issues or PRs

PR #1695

✅ Checklist

  • All: Ran fast tox checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    uvx tox -e static
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
  • Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.
  • Ported Tests: All converted JSON/YML tests from ethereum/tests or tests/static have been assigned @ported_from marker.

Cute Animal Picture

animal-0

@codecov
Copy link

codecov bot commented Jan 23, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.14%. Comparing base (16316e2) to head (e87ba3a).
⚠️ Report is 10 commits behind head on forks/amsterdam.

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #2067   +/-   ##
================================================
  Coverage            86.14%   86.14%           
================================================
  Files                  599      599           
  Lines                39472    39472           
  Branches              3780     3780           
================================================
  Hits                 34002    34002           
  Misses                4848     4848           
  Partials               622      622           
Flag Coverage Δ
unittests 86.14% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@spencer-tb
Copy link
Contributor

spencer-tb commented Jan 23, 2026

Can we add a pytest skip for execute here for now? With a todo

@pytest.mark.execute(pytest.mark.skip(reason="Blob txs not supported by execute"))  

Copy link
Contributor

@spencer-tb spencer-tb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOBHASH reads from the current tx blob array, not a previous tx, so we can't go the setup_tx route. I added an updated test implementation below.

For the --fixed-opcode-count mode:

  • Prague: No tx gas limit cap. Single tx with effectively unlimited gas (set to 1T). Op.POP keeps the stack clean, enabling unlimited iterations.
  • Osaka+: Has a 16M tx gas limit cap. Single tx capped at 16M gas, no splitting. This only lets us get to a limit of 1200 (x1000) for the count.

For the --gas-benchmark-values mode:

  • Prague: No tx gas limit cap, so no splitting occurs. Single blob tx works fine.
  • Osaka+: Has a 16M tx gas limit cap which causes transaction splitting. Each split tx would be a blob tx, and multiple blob txs per block could exceed the block blob limit.

The test below calculates required_splits = gas_benchmark_value / tx_gas_limit_cap and compares against max_blobs_per_block. If splits exceed the blob limit, the test is skipped for that fork. If we want to hit 150M we need 10 blobs, 9 will only get us to 144M. This means we should probably fill for >BPO1, maybe Amsterdam. If Amsterdam inherits BPO2 then we can reach around 336M.

@pytest.mark.repricing
@pytest.mark.parametrize(
    "blob_present",
    [
        pytest.param(0, id="no_blobs"),
        pytest.param(1, id="one_blob"),
    ],
)
def test_blobhash(
    fork: Fork,
    benchmark_test: BenchmarkTestFiller,
    pre: Alloc,
    blob_present: int,
    fixed_opcode_count: int | None,
    gas_benchmark_value: int,
) -> None:
    """Benchmark BLOBHASH instruction."""
    tx_kwargs: dict = {}
    if blob_present:
        cap = fork.transaction_gas_limit_cap()
        if fixed_opcode_count is None and cap is not None:
            # Check if blob tx splits would exceed block blob limit
            required_splits = math.ceil(gas_benchmark_value / cap)
            max_blobs = fork.max_blobs_per_block()
            if required_splits > max_blobs:
                pytest.skip(
                    f"Blob tx needs {required_splits} splits but fork allows "
                    f"{max_blobs} blobs/block"
                )
        tx_kwargs = {
            "ty": TransactionType.BLOB_TRANSACTION,
            "max_fee_per_blob_gas": fork.min_base_fee_per_blob_gas(),
            "blob_versioned_hashes": add_kzg_version(
                [i.to_bytes(32, "big") for i in range(blob_present)],
                BlobsSpec.BLOB_COMMITMENT_VERSION_KZG,
            ),
        }

    benchmark_test(
        target_opcode=Op.BLOBHASH,
        code_generator=ExtCallGenerator(
            attack_block=Op.POP(Op.BLOBHASH(0)),
            tx_kwargs=tx_kwargs,
        ),
    )

@LouisTsai-Csie LouisTsai-Csie force-pushed the benchmark-migrate-osaka branch from 8c31bb3 to 5976565 Compare January 27, 2026 14:50
@LouisTsai-Csie LouisTsai-Csie self-assigned this Jan 29, 2026
@LouisTsai-Csie LouisTsai-Csie added A-test-benchmark Area: execution_testing.benchmark and tests/benchmark C-refactor Category: refactor labels Jan 29, 2026
@LouisTsai-Csie
Copy link
Collaborator Author

@spencer-tb Thanks a lot!! One small thing: we could remove POP , since the target contract in the fixed-opcode-count would not run more than 1000 iterations, this means it is impossible to stack overflow! Updated accordingly.

Copy link
Contributor

@spencer-tb spencer-tb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<3

@spencer-tb spencer-tb merged commit 1b266ab into ethereum:forks/amsterdam Jan 29, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-test-benchmark Area: execution_testing.benchmark and tests/benchmark C-refactor Category: refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants