[Node Operator Question] Deployment of full node with partial state retention over last 5000000 blocks

[free-cornet]

Are you running the most up to date node software?

• Yes

Did you check the documentation?

• Yes

Did you check for duplicate questions?

• Yes

Issue Description

Due to storage limitations, I’d like to run a full node that retains only the most recent 5,000,000 blocks of historical state. According to the documentation, this should be achievable by setting the flags --history.state=5000000 and/or --history.transactions=5000000, together with --state.scheme=path. I’ve deployed a local full node using these parameters.

My goal is to use the Web3 method debug_traceBlockByNumber on blocks within this 5,000,000-block window to extract internal transactions. However, while it works correctly for the most recent 128 blocks, attempting to trace older blocks results in the following error:

"code": -32000,
"message": "historical state is not available"

I’m wondering if I’ve misunderstood how these flags work. Is it possible to configure a node to retain partial historical state for recent blocks, enabling tracing with debug_traceBlockByNumber, without having to run a full archive node?

Steps to Reproduce

Deploy a local full node, using the following config:

/usr/local/bin/geth \
  --ws \
  --ws.port=8547 \
  --ws.addr=localhost \
  --ws.origins="*" \
  --ws.api=eth,net,web3,debug \
  --http \
  --http.port=8548 \
  --http.addr=localhost \
  --http.vhosts="*" \
  --http.corsdomain="*" \
  --http.api=eth,net,web3,debug \
  --port=30305 \
  --authrpc.addr=localhost \
  --authrpc.jwtsecret=/mnt/crypto/optimism/setup/jwt.txt \
  --authrpc.port=8552 \
  --authrpc.vhosts="*" \
  --datadir=/mnt/optimism/datadir \
  --verbosity=3 \
  --rollup.disabletxpoolgossip=true \
  --rollup.sequencerhttp=https://mainnet-sequencer.optimism.io/ \
  --op-network=op-mainnet \
  --state.scheme=path \
  --history.state=5000000 \
  --history.transactions=5000000

Then try to query the RPC with the Web3 method debug_traceBlockByNumber for a block older than the 128 most recent blocks (but part of the 5000000 most recent blocks):

curl -X POST http://localhost:8548 \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc": "2.0",
    "method": "debug_traceBlockByNumber",
    "params": ["0x88f8a48", {}],
    "id": 1
  }'

Which component is affected?

• op-geth
• op-node
• Other

Expected vs. Actual Behavior

I expect to receive the trace execution of the queried block, containing the internal transactions. Instead I got:

{
    "jsonrpc":"2.0",
    "id":1,
    "error":{
        "code":-32000,
        "message":"historical state is not available"
    }
}

Environment Details

The node is running in a Ubuntu 24.04 LTS VM.

• CPU: 16 cores
• Memory: 64GB
• Disk space: 5TB SATA SSD (mounted over NFS)

The Ethereum L1 node is an archive node deployed locally on the same machine using Erigon and Prysm, using a 2TB NVMe as storage.

Protocol Information

• op-geth version: 1.101603.2-stable-f305011e
• op-node version: v1.14.3-8c183919-1761178868
• network name: op-mainnet
• chain ID: 10
• L1 chain: mainnet (Ethereum, Chain ID=1)

Node Logs

No logs relative to --history flags.

Only relevant log is:

State scheme set by user scheme=path

Troubleshooting Attempts

No response

Additional Information

No response

Feedback

No response

[Chomtana]

You have to launch the node using a full archival snapshot if your node is recently launched

If you use snap sync or full node snapshot, snapshot doesn't have required past blocks

[free-cornet]

So you're telling me to run a archive node instead of a full node, in order to have the complete historical chain of the blockchain, or is full archival snapshot something else?

[Chomtana]

Yeah, you better use op-reth for this case

https://reth.rs/run/opstack/

And use snapshot from https://publicnode.com/snapshots

but SATA disk might be a problem, you better get an SSD or NVME

[free-cornet]

I see. Do you have any idea of the storage requirements if using op-reth (or op-erigon)? I know that those execution client have way more optimized storage performance when used for Ethereum archive node.

Yes, SATA disks might lead to limitations, the disk but the ~20TB NVMe, required for an archive node, are very costly.

Edit: By the way, when I said SATA but I meant SATA SSD, just not NVMe SSD.

[Chomtana]

Since https://publicnode.com/snapshots op-reth archive snapshot is 1.1TB

I think 5 TB disk should be fine but make sure the speed is fast enough. Mounted over NFS might have high overhead especially for random write.

[jsvisa]

@free-cornet you need to start op-geth with --gcmode=archive to enable the state indexing feature.

[xerion12]

so i think the issue you are facing is quite common, and it’s actually not a bug, more like a design thing in ethereum nodes. if you want to use debug_traceBlockByNumber for blocks older than the last8192, then you need to run an archive node, not a full node. even if you set --history.state=5000000, the node will still only keep state for the last few thousand blocks, because the protocol does not allow full historical state on full nodes. you must use --gcmode=archive for archive nodes to get full historical state. this way you can trace any block, but it needs a lot more storage. if you only need recent blocks, full node is fine. sorry for the trouble, but this is how it works.