all: implement EIP-1153 transient storage #26003

tynes · 2022-10-17T18:24:37Z

Implements TSTORE and TLOAD as specified by the following EIP:

https://eips.ethereum.org/EIPS/eip-1153
https://ethereum-magicians.org/t/eip-1153-transient-storage-opcodes/553

This PR includes test coverage of these opcodes.

We are interested in having this included as part of Shanghai, as talked about at the EIP panel at Devcon. There is general community support by application layer developers, see the Ethereum Magicians thread. There are 4 implementations so far, see the links below.

Includes an extensive test suite for ethereum/tests here snreynolds/tests#1

Link to Shanghai planning issue #25820
Replaces #24463

For additional information, see https://www.eip1153.com/

Additional implementations:

The EIP itself takes into account the maximum amount of memory that can be allocated if an entire block's worth of gas is used to TSTORE data, see https://eips.ethereum.org/EIPS/eip-1153#security-considerations.

About 9.15MB can be allocated in theory, plus the runtime overhead. cc @MariusVanDerWijden

30M gas * 1 TSTORE / 100 gas * 32 bytes / 1 TSTORE * 1MB / 2^20 bytes ~= 9.15MB

Co-authored-by: Sara Reynolds snreynolds2506@gmail.com

holiman

A few comments. Generally the implementation looks pretty clean.

Also, I don't think there's any point updating the testdata -- not sure what commit you are updating it to, but it's not the latest. If it's some branch with 1153-tests, then that will have to be done later on once it's made it into develop on tests.

holiman · 2022-10-17T19:13:33Z

core/state/journal.go

+func (ch transientStorageChange) dirtied() *common.Address {
+	return ch.account
+}


We need to look into this: I think it is incorrect to report the account as dirtied by a transient op.

The dirtied address is added to the journal dirties, which is later used to figure out which accounts have been modified -- in the sense that the trie representation of the account is changed. But a transient op does not modify the trie representation.

Putting this note here as a todo to investigate a bit further.

Ah, we do need to put it in journal.dirties after all, because otherwise we won't call finalise at the end of the tx, and if we don't do that, the transient storage doesn't get reset between transactions.

The cost for this is that we add the address to the prefetcher, which starts loading tries, to speed up the commit-phase later on. And if transient is adopted, then that is unnecessary work which we should avoid.

I'm not so sure that a Storage on the stateObject is the best representation of a transient storage. The state-objects live longer than a single transaction. Having the transient on the objects themselves just forces us to iterate through them at the end of each tx, just to clean up.

The transient storage(s) are essentially a transaction-global map of address --> Storage. After each transaction, we could just nuke it out and prepare a fresh one.

This is how we did with the accesslist. In statedb.go:

// Per-transaction access list accessList *accessList

It uses the 'normal' journal, but it is otherwise it's own separate entity, not tied to stateobjects.

This makes sense, it feels like it sits at a similar level of abstraction as the access list since it is applied on a per transaction basis. Thank you for the feedback, will update the design to follow this approach

I am of the opinion that dirtied() should return nil. Setting a transient storage value does not make an account dirty, in the trie-sense.
I'm open to being proven wrong

I agree with this, a dirty account implies that it was mutated in some way

I agree with this, a dirty account implies that it was mutated in some way

98eff44 has dirtied() return nil

core/vm/opcodes.go

tynes · 2022-10-23T21:49:02Z

Thank you for the review @holiman. I've addressed your feedback and moved the API away from the stateObject.

To run the ethereum/tests, I used the following setup:

./retesteth/retesteth -t GeneralStateTests/stEIP1153 -- \
    --testpath <path to ethereum/tests> \
    --all --vmtrace --clients t8ntooleip --singlenet London

Note that I was having trouble with the London+1153 syntax, so I just built evm with 1153 activated in London to run these tests for simplicity. Not sure why it was giving me an error

holiman

LGTM, a minor nit or two

core/state/transient_storage.go

core/vm/instructions.go

core/state/statedb.go

core/state/transient_storage.go

holiman

LGTM

Implements `TSTORE` and `TLOAD` as specified by the following EIP: https://eips.ethereum.org/EIPS/eip-1153 https://ethereum-magicians.org/t/eip-1153-transient-storage-opcodes/553 Co-authored-by: Sara Reynolds <snreynolds2506@gmail.com>

Moves the transient storage API away from the state object based on PR feedback. Confirm that all ethereum/tests still pass using `retesteth`

Moves the actual opcode implementations to `eips.go` which contains opcodes added through EIPs. Also add method docs to the new functions.

Transient storage doesn't mutate the state so return `nil` instead of returning the account that had transient storage modified.

Implements EIP-1153 "Transient Storage" https://eips.ethereum.org/EIPS/eip-1153 Based on ethereum/go-ethereum#26003

Port ethereum/go-ethereum#26003 --------- Co-authored-by: Mark Tyneway <mark.tyneway@gmail.com>

Implements TSTORE and TLOAD as specified by the following EIP: https://eips.ethereum.org/EIPS/eip-1153 https://ethereum-magicians.org/t/eip-1153-transient-storage-opcodes/553 Co-authored-by: Sara Reynolds <snreynolds2506@gmail.com> Co-authored-by: Martin Holst Swende <martin@swende.se> Co-authored-by: Gary Rong <garyrong0905@gmail.com>

* all: implement EIP-1153 transient storage (ethereum#26003) Implements TSTORE and TLOAD as specified by the following EIP: https://eips.ethereum.org/EIPS/eip-1153 https://ethereum-magicians.org/t/eip-1153-transient-storage-opcodes/553 Co-authored-by: Sara Reynolds <snreynolds2506@gmail.com> Co-authored-by: Martin Holst Swende <martin@swende.se> Co-authored-by: Gary Rong <garyrong0905@gmail.com> * core/vm: move TSTORE,TLOAD to correct opcode nums (ethereum#27613) * core/vm: move TSTORE,TLOAD to correct opcode nums * core/vm: cleanup * fix tests, rename * goimports * enable 1153 at Curie * bump version * comment fix * improve test * version * testchainconfig * fix another test that affects newly added * fix previous test to clenaup after --------- Co-authored-by: Mark Tyneway <mark.tyneway@gmail.com> Co-authored-by: Sara Reynolds <snreynolds2506@gmail.com> Co-authored-by: Martin Holst Swende <martin@swende.se> Co-authored-by: Gary Rong <garyrong0905@gmail.com> Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>

tynes requested review from karalabe, holiman and rjl493456442 as code owners October 17, 2022 18:24

tynes mentioned this pull request Oct 17, 2022

Shanghai tracking issue #25820

Closed

holiman added the maybe shanghai label Oct 17, 2022

holiman reviewed Oct 17, 2022

View reviewed changes

tynes force-pushed the feat/eip1153 branch from 347dd7f to 9c9ea83 Compare October 23, 2022 21:43

holiman approved these changes Nov 11, 2022

View reviewed changes

core/state/transient_storage.go Outdated Show resolved Hide resolved

core/vm/instructions.go Outdated Show resolved Hide resolved

holiman reviewed Nov 11, 2022

View reviewed changes

core/state/statedb.go Show resolved Hide resolved

core/state/statedb.go Show resolved Hide resolved

core/state/transient_storage.go Show resolved Hide resolved

core/state/transient_storage.go Show resolved Hide resolved

core/state/transient_storage.go Show resolved Hide resolved

holiman changed the title ~~eip1153: transient storage~~ all: implement EIP-1153 transient storage Nov 11, 2022

This was referenced Nov 11, 2022

Implement in Geth Uniswap/v4-core#133

Closed

Add EIP-1153 to CFI for Shanghai ethereum/execution-specs#642

Merged

holiman approved these changes Nov 15, 2022

View reviewed changes

holiman mentioned this pull request Nov 15, 2022

core: Implement EIP 1153 #24463

Closed

holiman added the status:triage label Nov 15, 2022

fjl removed the status:triage label Nov 15, 2022

tynes and others added 8 commits November 16, 2022 09:57

eip1153: transient storage

e1a3af3

Implements `TSTORE` and `TLOAD` as specified by the following EIP: https://eips.ethereum.org/EIPS/eip-1153 https://ethereum-magicians.org/t/eip-1153-transient-storage-opcodes/553 Co-authored-by: Sara Reynolds <snreynolds2506@gmail.com>

eip1153: move API away from stateObject

87f02f3

Moves the transient storage API away from the state object based on PR feedback. Confirm that all ethereum/tests still pass using `retesteth`

core/state: method-docs

7799a41

vm: move opcode implementations to eips.go

663c1a2

Moves the actual opcode implementations to `eips.go` which contains opcodes added through EIPs. Also add method docs to the new functions.

state: add method docs for transient storage changes

83b558a

state: transient storage changes don't dirty

9763391

Transient storage doesn't mutate the state so return `nil` instead of returning the account that had transient storage modified.

core: reset transient storage after each transaction

c118725

all: refactor state preparation before state transition

5ea11ab

rjl493456442 force-pushed the feat/eip1153 branch from 98eff44 to 5ea11ab Compare November 16, 2022 07:47

rjl493456442 requested review from zsfelfoldi and s1na as code owners November 16, 2022 07:47

rjl493456442 approved these changes Nov 16, 2022

View reviewed changes

holiman added this to the 1.11.0 milestone Nov 16, 2022

holiman merged commit b4ea2bf into ethereum:master Nov 16, 2022

yperbasis mentioned this pull request Nov 16, 2022

EIP 1153 erigontech/erigon#6049

Closed

tynes added a commit to tynes/erigon that referenced this pull request Nov 26, 2022

all: eip1153

0127728

Implements EIP-1153 "Transient Storage" https://eips.ethereum.org/EIPS/eip-1153 Based on ethereum/go-ethereum#26003

This was referenced Nov 26, 2022

all: eip1153 erigontech/erigon#6133

Closed

feat: eip 1153 bluealloy/revm#276

Closed

[ETHSF_HACKATHON_SUBMISSION] (EIP-1153) Transient Storage in EVM ethereum-optimism/op-geth#25

Closed

Tristan-Wilson mentioned this pull request Feb 28, 2023

Merge v1.11.2 OffchainLabs/go-ethereum#205

Merged

13 tasks

snreynolds mentioned this pull request Mar 15, 2023

Execution Layer Meeting 157 ethereum/pm#737

Closed

yperbasis mentioned this pull request Apr 28, 2023

Implement EIP-1153 transient storage erigontech/erigon#7405

Merged

yperbasis added a commit to erigontech/erigon that referenced this pull request May 1, 2023

Implement EIP-1153 transient storage (#7405)

1533674

Port ethereum/go-ethereum#26003 --------- Co-authored-by: Mark Tyneway <mark.tyneway@gmail.com>

holiman added cancun and removed maybe shanghai labels Jul 11, 2023

dustinxie mentioned this pull request Mar 11, 2024

[evm] enable transient storage feature iotexproject/iotex-core#4177

Closed

2 tasks

gzliudan mentioned this pull request Sep 26, 2024

[WIP] implement EIP-1153(transient storage) XinFinOrg/XDPoSChain#647

Open

19 tasks

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Sep 27, 2024

all: implement EIP-1153 transient storage (ethereum#26003)

d52bf79

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Oct 2, 2024

all: implement EIP-1153 transient storage (ethereum#26003)

6acb29d

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Oct 16, 2024

all: implement EIP-1153 transient storage (ethereum#26003)

bbf66f3

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Oct 17, 2024

all: implement EIP-1153 transient storage (ethereum#26003)

83c190f

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Oct 18, 2024

all: implement EIP-1153 transient storage (ethereum#26003)

2c00c30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

all: implement EIP-1153 transient storage #26003

all: implement EIP-1153 transient storage #26003

tynes commented Oct 17, 2022

holiman left a comment

holiman Oct 17, 2022

holiman Oct 17, 2022 •

edited

Loading

holiman Oct 18, 2022

tynes Oct 18, 2022

holiman Nov 14, 2022

tynes Nov 14, 2022

tynes Nov 14, 2022

tynes commented Oct 23, 2022

holiman left a comment

holiman left a comment

all: implement EIP-1153 transient storage #26003

all: implement EIP-1153 transient storage #26003

Conversation

tynes commented Oct 17, 2022

holiman left a comment

Choose a reason for hiding this comment

holiman Oct 17, 2022

Choose a reason for hiding this comment

holiman Oct 17, 2022 • edited Loading

Choose a reason for hiding this comment

holiman Oct 18, 2022

Choose a reason for hiding this comment

tynes Oct 18, 2022

Choose a reason for hiding this comment

holiman Nov 14, 2022

Choose a reason for hiding this comment

tynes Nov 14, 2022

Choose a reason for hiding this comment

tynes Nov 14, 2022

Choose a reason for hiding this comment

tynes commented Oct 23, 2022

holiman left a comment

Choose a reason for hiding this comment

holiman left a comment

Choose a reason for hiding this comment

holiman Oct 17, 2022 •

edited

Loading