Deprecating the pending state

[s1na]

This issue is to keep track of the conversation started at the JSON-RPC session at devconnect.

Most state-related methods which take in a blocktag allow users to specify "pending". The semantic difference to "latest" is that the EL keeps track of a pending state by applying all of the pending transactions on top of the current state.

We're questioning the practical use of this tag as it depends heavily on the local view of the txpool. The mempool is fluid, there is no guarantee that the transactions there will be mined. Moreover different nodes within an RPC provider infrastructure have different views on the mempool and some notable ones do not serve pending state.

The path to deprecating "pending" we are proposing is to create an empty pending block. This means the block context fields such as number and timestamp will reflect the "next" block, but state is from that of "latest". This also means that there will be no guarantees that the client can deliver pending logs. The only exception would be eth_getTransactionCount which should return the latest nonce for an account, considering also txes in the mempool.

This will alleviate some load from nodes to build the pending block and maintain its state while keeping most of the current use-cases.

Looking into the future: by adding block context overrides to relevant methods such as eth_call, eth_estimateGas and eth_createAccessList we remove the need for creating an empty pending block. Users can simulate on top of "latest" but pass in overrides for the next block. Furthermore if clients implement eth_multicall, it will be possible to simulate the pending state by passing higher weighted pending txes to multicall.

[protolambda]

In the OP-Stack we've been aliasing the pending RPC label to latest, to:

• Save the cost of computing a pending-block (low block-time of 2 seconds makes recomputing a 30M gas block quite expensive).
• Serve sensible data, when the tx-pool is private.

This has been working well enough for L2 users (although maybe better than L1 due to the lower block time). I support the aliasing of it to latest as deprecation strategy on L1.

And we still support eth_getTransactionCount, since it's attached to the mem-pool, not the actual pending state, so is not as costly to maintain. And since it does not reveal the full transaction data, generally not regarded as privacy issue, and a good UX trade-off.

So far we've left computation of pending state still as opt-in feature for individual node users, which then allows them to inspect the block-building of their local mempool (which just contains a copy of their own transactions, that the node passed on privately to the sequencer endpoint), which they can tune the gas-limit of as well (again to save unnecessary re-computation cost). But from user-issues it appears that nobody is actually using the opt-in feature.

With the removal of Clique from EL clients, removing the pending flag unblocks the cleanup of a lot of legacy pre-engine-API block-building, so I'm very much in favor from the perspective of reducing code complexity.

[s1na]

Adding a note that this means it will not be possible to retrieve pending logs anymore. I personally think if pending state/logs are required they can be computed using eth_multicall over the mempool set.

Link to geth implementation: ethereum/go-ethereum#28440

[lmittmann]

The pending header is very useful for simulating transactions for the pending block when constructing the VM's block context.

[s1na]

The pending header is very useful for simulating transactions for the pending block when constructing the VM's block context.

Interesting. The fields that might be useful are: timestamp, number, baseFeePerGas and gasLimit. They are all computable off-band based on previous header but I see the point that they are consensus logic and better handled within the node.

If my assumption is right I think a path forward is to return a pending header with 0 transactions. It will fill the fields above but every other field will be trash.

[niklasnatter]

The pending header is very useful for simulating transactions for the pending block when constructing the VM's block context.

One relevant example for this is using the eth_createAccessList with the pending block tag. A lot of contracts that use checkpoints (eg. Uniswap v3 for oracle observations) do a binary search that uses the current block number. Falling back to latest will lead to wrong entries in the access list in this case.

A more flexible way to cover these cases could be supporting blockOverrides (like already implemented for eth_call in go-ethereum: ethereum/go-ethereum#26414) for the relevant RPC methods.

[s1na]

Gotcha. I will update the description above to clearly specify: "we will use a pending block with zero transactions" when "pending" is passed. This should cover most cases.

That said, I agree we should push blockOverrides to be implemented across the board by the time we fully remove "pending".

[ryanschneider]

And we still support eth_getTransactionCount

I would advocate against this since I think it gives users a false sense of "correctness", instead I think we should be encouraging users/wallets to do their own "off chain/out of pool" nonce management. In fact I'd be in favor of a new 2718 tx type that supports a different nonce/replay protection scheme.