we must optimize "EnableMemory" in trace

[lispc]

I traced a random recent block (block 14742200, 30M gas).

When EnableMemory: false, debug_traceBlockByNumber finished within 15 seconds, the trace file size is less than 130MiB.

But when EnableMemory: true, debug_traceBlockByNumber never runs successfully even on a 64GiB mem server. The debug_traceBlockByNumber rpc call usually fails(return nothing) after more than 2 mintues, 40-50GiB mem used.

curl -s -H "Content-Type: application/json" -X POST --data '{"jsonrpc":"2.0","method":"debug_traceBlockByNumber", "params": ["0xe0f2b8",{"DisableStack":false,"DisableStorage":false,"EnableMemory":true,"EnableReturnData":true,"Timeout":"10m"}], "id": 99}' 127.0.0.1:8545 > tmp.trace

(I used ethereum/go-ethereum, not scroll-tech/go-ethereum)

[lispc]

use kv(addr:value) rather than a huge full vector to contain mem rw information?

[lispc]

or just reconstruct memory in prover side. We set "EnableMemory" to false.

[0xmountaintop]

what if we use "eth_getBlockResultByHash"? can you re-test it?

[lispc]

not easy. The real block contains some "real world" txs. Our erc20 tx is usually simpler than an average defi tx.

Maybe we can just send many erc20 transfer to l2geth and use "eth_getBlockResultByHash" to fetch a blockResult. I guess the results will be similar, since both trace and "BlockResult" contains memory snapshot of every step.

[lispc]

Long term options:

(1) implement rpc similar to witness protocol, ref: https://github.com/NethermindEth/nethermind/blob/master/src/Nethermind/Nethermind.State/Witnesses/WitnessCollector.cs , and generate traces statelessly on the prover side. (Maybe we can implement this non-invasively using "prestate" tracer and "eth_getProof".

(2) generate WitnessBlock directly in geth side to avoid generating huge full trace. May need to rewrite some codes in bus-mapping in golang

[0xmountaintop]

should we close this issue now?