Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix for unit tests: do not call NewAccessWitness in NewEVMTxContext #49

Conversation

gballet
Copy link
Owner

@gballet gballet commented Jan 4, 2022

Ensure that the tests pass if I deactivate witnesses in general tests. This includes part of the recommended fix for #47.

⚠️ ☠️ this doesn't change the fact that calculating an address is very slow with the proposed "pedersen hash" scheme.

@gballet gballet requested a review from holiman as a code owner January 4, 2022 09:23
@gballet gballet removed the request for review from holiman January 4, 2022 09:23
@gballet
Copy link
Owner Author

gballet commented Jan 4, 2022

the test keeps failing because of the following issue:

goroutine 79 [runnable]:
github.com/crate-crypto/go-ipa/bandersnatch/fp.mul(0xc0020d8490, 0xc0020d83b0, 0xc0020d8370)
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/bandersnatch/fp/element_mul_amd64.s:46 +0x3d7
github.com/crate-crypto/go-ipa/bandersnatch/fp.(*Element).Mul(...)
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/bandersnatch/fp/element.go:288
github.com/crate-crypto/go-ipa/bandersnatch.(*PointProj).Double(0xc0020d86d0, 0xc0020d86d0)
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/bandersnatch/point.go:305 +0x3c9
github.com/crate-crypto/go-ipa/bandersnatch.PointAffine.IsInPrimeSubgroup({{0xa93a1b95e405524, 0xcbdcd03ccccd3bfe, 0xdd2f6874a5943d08, 0x25e667f67bbea689}, {0x9a5e23d862b9673f, 0x508db07c65c463cd, 0xc2d9b667ccc51dac, 0x6b98f119f248d762}})
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/bandersnatch/point.go:373 +0x17a
github.com/crate-crypto/go-ipa/ipa.GenerateRandomPoints(0x100)
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/ipa/config.go:204 +0x2de
github.com/crate-crypto/go-ipa/ipa.NewIPASettings()
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/ipa/config.go:34 +0x3f
github.com/ethereum/go-ethereum/trie/utils.GetTreeKey({0xc0020d8e8c, 0x14, 0x14}, 0xc00011b320, 0x1)
	/usr/home/gballet/src/go-ethereum/trie/utils/verkle.go:58 +0x2a5
github.com/ethereum/go-ethereum/trie/utils.GetTreeKeyBalance(...)
	/usr/home/gballet/src/go-ethereum/trie/utils/verkle.go:73
github.com/ethereum/go-ethereum/consensus/ethash.accumulateRewards(0xc0001b2180, 0xc000034700, 0xc001f7eb40, {0x131f208, 0x0, 0x0})
	/usr/home/gballet/src/go-ethereum/consensus/ethash/consensus.go:671 +0x671
github.com/ethereum/go-ethereum/consensus/ethash.(*Ethash).Finalize(0xc000234a80, {0x5239b8, 0xc000844000}, 0xc001f7eb40, 0xc000034700, {0x131f208, 0x0, 0x0}, {0x131f208, 0x0, ...})
	/usr/home/gballet/src/go-ethereum/consensus/ethash/consensus.go:597 +0x85
github.com/ethereum/go-ethereum/consensus/beacon.(*Beacon).Finalize(0xc000072e80, {0x5239b8, 0xc000844000}, 0xc001f7eb40, 0xc000034700, {0x131f208, 0x0, 0x0}, {0x131f208, 0x0, ...})
	/usr/home/gballet/src/go-ethereum/consensus/beacon/consensus.go:275 +0x13f
github.com/ethereum/go-ethereum/core.(*StateProcessor).Process(0xc000814c60, 0xc00078f170, 0xc000034700, {0x0, {0x0, 0x0}, 0x0, 0x0, 0x0, {0x0, ...}, ...})
	/usr/home/gballet/src/go-ethereum/core/state_processor.go:90 +0x102f
github.com/ethereum/go-ethereum/core.(*BlockChain).insertChain(0xc000844000, {0xc0007a8008, 0x3e7, 0x3ff}, 0x1, 0x1)
	/usr/home/gballet/src/go-ethereum/core/blockchain.go:1602 +0x21ac
github.com/ethereum/go-ethereum/core.(*BlockChain).InsertChain(0xc000844000, {0xc0007a8008, 0x3e7, 0x3ff})
	/usr/home/gballet/src/go-ethereum/core/blockchain.go:1368 +0xcf2
github.com/ethereum/go-ethereum/cmd/devp2p/internal/ethtest.setupGeth(0xc0003569c0)
	/usr/home/gballet/src/go-ethereum/cmd/devp2p/internal/ethtest/suite_test.go:105 +0x1a9
github.com/ethereum/go-ethereum/cmd/devp2p/internal/ethtest.runGeth()
	/usr/home/gballet/src/go-ethereum/cmd/devp2p/internal/ethtest/suite_test.go:72 +0x94
github.com/ethereum/go-ethereum/cmd/devp2p/internal/ethtest.TestEthSuite(0xc000502340)
	/usr/home/gballet/src/go-ethereum/cmd/devp2p/internal/ethtest/suite_test.go:38 +0x46
testing.tRunner(0xc000502340, 0x4297d8)
	/usr/local/go/src/testing/testing.go:1259 +0x102
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1306 +0x3ab

So basically, verkle is still enabled when it shouldn't.

@gballet
Copy link
Owner Author

gballet commented Jan 4, 2022

Another test fails for the same reasons:

goroutine 28 [runnable]:
github.com/crate-crypto/go-ipa/bandersnatch/fp.mul(0xc0001367b0, 0xc000136b30, 0xc000136b30)
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/bandersnatch/fp/element_mul_amd64.s:46 +0x3d7
github.com/crate-crypto/go-ipa/bandersnatch/fp.(*Element).Square(...)
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/bandersnatch/fp/element.go:295
github.com/crate-crypto/go-ipa/bandersnatch.(*PointProj).Double(0xc000136af0, 0xc000136af0)
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/bandersnatch/point.go:298 +0x28c
github.com/crate-crypto/go-ipa/bandersnatch.PointAffine.IsInPrimeSubgroup({{0xa93994b5e405d92, 0xb4796186bef8bb90, 0xaeca7c35d7df39f0, 0x633ee6ff4954bd29}, {0xae71b521ddf23201, 0x6500d1ae2c0eb2d8, 0x6e1942871f7c3c16, 0x4b29a1c92f261a25}})
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/bandersnatch/point.go:373 +0x17a
github.com/crate-crypto/go-ipa/ipa.GenerateRandomPoints(0x100)
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/ipa/config.go:204 +0x2de
github.com/crate-crypto/go-ipa/ipa.NewIPASettings()
	/home/gballet/go/pkg/mod/github.com/crate-crypto/go-ipa@v0.0.0-20211223165939-ab3f49447206/ipa/config.go:34 +0x3f
github.com/ethereum/go-ethereum/trie/utils.GetTreeKey({0xc000137318, 0x14, 0x14}, 0xc000137250, 0x8)
	/usr/home/gballet/src/go-ethereum/trie/utils/verkle.go:58 +0x2a5
github.com/ethereum/go-ethereum/trie/utils.GetTreeKeyStorageSlot({0xc000137318, 0x14, 0x14}, 0xc000260100)
	/usr/home/gballet/src/go-ethereum/trie/utils/verkle.go:116 +0x197
github.com/ethereum/go-ethereum/core/vm.opSload(0xc0001da4c8, 0xc00039c000, 0xc000582060)
	/usr/home/gballet/src/go-ethereum/core/vm/instructions.go:581 +0x25c
github.com/ethereum/go-ethereum/core/vm.(*EVMInterpreter).Run(0xc00039c000, 0xc000396e40, {0x0, 0x0, 0x0}, 0x0)
	/usr/home/gballet/src/go-ethereum/core/vm/interpreter.go:308 +0xdd9
github.com/ethereum/go-ethereum/core/vm.(*EVM).create(0xc0000ec000, {0x3a5760, 0xc000262360}, 0xc0001379a0, 0x9a91a, 0xc0001e0660, {0xcc, 0x2c, 0xd, 0x8, ...}, ...)
	/usr/home/gballet/src/go-ethereum/core/vm/evm.go:488 +0x928
github.com/ethereum/go-ethereum/core/vm.(*EVM).Create(0xc0000ec000, {0x3a5760, 0xc000262360}, {0xc000094000, 0x970, 0x1300}, 0x9a91a, 0xc0001e0660)
	/usr/home/gballet/src/go-ethereum/core/vm/evm.go:536 +0x212
github.com/ethereum/go-ethereum/core.(*StateTransition).TransitionDb(0xc0000e0500)
	/usr/home/gballet/src/go-ethereum/core/state_transition.go:344 +0x12c6
github.com/ethereum/go-ethereum/accounts/abi/bind/backends.(*SimulatedBackend).callContract(0xc0001dfa00, {0x3af570, 0xc000026098}, {{0xf, 0xfe, 0x93, 0x58, 0xde, 0xdf, 0x26, ...}, ...}, ...)
	/usr/home/gballet/src/go-ethereum/accounts/abi/bind/backends/simulated.go:638 +0x733
github.com/ethereum/go-ethereum/accounts/abi/bind/backends.(*SimulatedBackend).EstimateGas.func1(0xb0112)
	/usr/home/gballet/src/go-ethereum/accounts/abi/bind/backends/simulated.go:536 +0x1d1
github.com/ethereum/go-ethereum/accounts/abi/bind/backends.(*SimulatedBackend).EstimateGas(0xc0001dfa00, {0x3af570, 0xc000026098}, {{0xf, 0xfe, 0x93, 0x58, 0xde, 0xdf, 0x26, ...}, ...})
	/usr/home/gballet/src/go-ethereum/accounts/abi/bind/backends/simulated.go:550 +0x790
github.com/ethereum/go-ethereum/accounts/abi/bind.(*BoundContract).estimateGasLimit(0xc000116f00, 0xc0000cc150, 0x0, {0xc000094000, 0x970, 0x1300}, 0x0, 0xc0001e0680, 0xc0001e06a0, 0xc0001e0660)
	/usr/home/gballet/src/go-ethereum/accounts/abi/bind/base.go:347 +0x29d
github.com/ethereum/go-ethereum/accounts/abi/bind.(*BoundContract).createDynamicTx(0xc000116f00, 0xc0000cc150, 0x0, {0xc000094000, 0x970, 0x1300}, 0xc0000d3b00)
	/usr/home/gballet/src/go-ethereum/accounts/abi/bind/base.go:264 +0x285
github.com/ethereum/go-ethereum/accounts/abi/bind.(*BoundContract).transact(0xc000116f00, 0xc0000cc150, 0x0, {0xc000094000, 0x970, 0x1300})
	/usr/home/gballet/src/go-ethereum/accounts/abi/bind/base.go:376 +0x1c7
github.com/ethereum/go-ethereum/accounts/abi/bind.DeployContract(0xc0000cc150, {{{0x0, 0x0}, {0x0, 0x0}, 0x0, {0xc0001db2a0, 0xa}, 0x0, 0x0, ...}, ...}, ...)
	/usr/home/gballet/src/go-ethereum/accounts/abi/bind/base.go:137 +0x46b
github.com/ethereum/go-ethereum/contracts/checkpointoracle/contract.DeployCheckpointOracle(0xc0000cc150, {0x3b8408, 0xc0001dfa00}, {0xc00053adc0, 0x3, 0x3}, 0xc0001e0100, 0xc0001e0120, 0xc0001e0460)
	/usr/home/gballet/src/go-ethereum/contracts/checkpointoracle/contract/oracle.go:49 +0x310
github.com/ethereum/go-ethereum/contracts/checkpointoracle.TestCheckpointRegister(0xc000202b60)
	/usr/home/gballet/src/go-ethereum/contracts/checkpointoracle/oracle_test.go:190 +0x7bb
testing.tRunner(0xc000202b60, 0x310ce8)
	/usr/local/go/src/testing/testing.go:1259 +0x102
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1306 +0x3ab

@gballet
Copy link
Owner Author

gballet commented Jan 4, 2022

@jwasinger this PR is meant as an inspiration for an update of #47, in which the function signatures don't get changed.

I realize that some of these changes are already addressed by another one of your PRs, I just wanted to see the tests pass.

Comment on lines 673 to 678
if config.IsCancun(header.Number) {
coinbase := utils.GetTreeKeyBalance(header.Coinbase.Bytes())
if state.Witness() != nil {
state.Witness().TouchAddress(coinbase, state.GetBalance(header.Coinbase).Bytes())
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be:

	if state.Witnes() != nil {
		coinbase := utils.GetTreeKeyBalance(header.Coinbase.Bytes())
	        state.Witness().TouchAddress(coinbase, state.GetBalance(header.Coinbase).Bytes())
	}

@jwasinger
Copy link
Collaborator

lgtm

@gballet gballet changed the title potential fix: do not call NewAccessWitness in NewEVMTxContext fix for unit tests: do not call NewAccessWitness in NewEVMTxContext Jan 5, 2022
@gballet gballet merged this pull request into verkle/get-key-pedersen-hash Jan 6, 2022
gballet added a commit that referenced this pull request Jan 6, 2022
)

* potential fix: do not call NewAccessWitness in NewEVMTxContext

* more fixes: check for the existence of Accesses

* fix absence of witness in copy

* fix another witness issue

* workaround: ensure the prefetcher is off in verkle mode

* fix the remaining issues in tests

* review feedback

* fix witness allocation in stateless test
gballet added a commit that referenced this pull request Jan 6, 2022
)

* potential fix: do not call NewAccessWitness in NewEVMTxContext

* more fixes: check for the existence of Accesses

* fix absence of witness in copy

* fix another witness issue

* workaround: ensure the prefetcher is off in verkle mode

* fix the remaining issues in tests

* review feedback

* fix witness allocation in stateless test
gballet added a commit that referenced this pull request Jan 7, 2022
* replace sha256 with pedersen_hash

* fix: prevent an OOB

* workaround timeout in unit test

* update go-ipa and reduce the timeout

* fix for unit tests: do not call NewAccessWitness in NewEVMTxContext (#49)

* potential fix: do not call NewAccessWitness in NewEVMTxContext

* more fixes: check for the existence of Accesses

* fix absence of witness in copy

* fix another witness issue

* workaround: ensure the prefetcher is off in verkle mode

* fix the remaining issues in tests

* review feedback

* fix witness allocation in stateless test
gballet added a commit that referenced this pull request Jan 10, 2022
)

* potential fix: do not call NewAccessWitness in NewEVMTxContext

* more fixes: check for the existence of Accesses

* fix absence of witness in copy

* fix another witness issue

* workaround: ensure the prefetcher is off in verkle mode

* fix the remaining issues in tests

* review feedback

* fix witness allocation in stateless test
gballet added a commit that referenced this pull request Mar 18, 2022
verkle: Implement Trie, NodeIterator and Database ifs

Fix crash in TestDump

Fix TestDump

Fix TrieCopy

remove unnecessary traces

fix: Error() returned errIteratorEnd in verkle node iterator

rewrite the iterator and change the signature of OpenStorageTrie

add the adapter to reuse the account trie for storage

don't try to deserialize a storage leaf into an account

Fix statedb unit tests (#14)

* debug code

* Fix more unit tests

* remove traces

* Go back to the full range

One tree to rule them all

remove updateRoot, there is no root to update

store code inside the account leaf

fix build

save current state for Sina

Update go-verkle to latest

Charge WITNESS_*_COST gas on storage loads

Add witness costs for SSTORE as well

Charge witness gas in the case of code execution

corresponding code deletion

add a --verkle flag to separate verkle experiments from regular geth operations

use the snapshot to get data

stateless execution from block witness

AccessWitness functions

Add block generation test + genesis snapshot generation

test stateless block execution (#18)

* test stateless block execution

* Force tree resolution before generating the proof

increased coverage in stateless test execution (#19)

* test stateless block execution

* Force tree resolution before generating the proof

* increase coverage in stateless test execution

ensure geth compiles

fix issues in tests with verkle trees deactivated

Ensure stateless data is available when executing statelessly (#20)

* Ensure stateless data is available when executing statelessly

* Actual execution of a statless block

* bugfixes in stateless block execution

* code cleanup

 - Reduce PR footprint by reverting NewEVM to its original signature
 - Move the access witness to the block context
 - prepare for a change in AW semantics
   Need to store the initial values.
 - Use the touch helper function, DRY

* revert the signature of MustCommit to its original form (#21)

fix leaf proofs in stateless execution (#22)

* Fixes in witness pre-state

* Add the recipient's nonce to the witness

* reduce PR footprint and investigate issue in root state calculation

* quick build fix

cleanup: Remove extra parameter in ToBlock

revert ToBlock to its older signature

fix import cycle in vm tests

fix linter issue

fix appveyor build

fix nil pointers in tests

Add indices, yis and Cis to the block's Verkle proof

upgrade geth dependency to drop geth's common dep

fix cmd/devp2p tests

fix rebase issues

quell an appveyor warning

fix address touching in SLOAD and SSTORE

fix access witness for code size

touch target account data before calling

make sure the proper locations get touched in (ext)codecopy

touch all code pages in execution

add pushdata to witness

remove useless code in genesis snapshot generation

testnet: fix some of the rebase/drift issues

Fix verkle proof generation in block

fix an issue occuring when chunking past the code size

fix: ensure the code copy doesn't extend past the code size

Upgrade go-verkle to its IPA version (#24)

fixes for the IPA testnet

upgrade to latest go-verkle

update go-verkle to get more fixes

simplify code by removing all stateless references (#25)

fix verkle proof test by enforcing values alignment to 32 bytes

remove unneeded KZG tag

fix the stateless test

Move AccessWitness into StateDB (#27)

* move AccessWitness into StateDB

* set Accesses in TxContext constructor

* Ensures that a statedb is initialized with a witness

* copy AccessWitness in StateDB.Copy.  use copied state in miner worker.commit.

* remove redundant line

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Fix contract creation issue

enable verkle on cancun block: take 2 (#28)

* enable verkle on cancun block: take 2

* fix typo.  make unreachable line panic message more clear

fix rebase issues

fix linter issue

merge undefined instead of panicking (#30)

initialize the new access witness if not already present

fix boundary check in PUSH

fix bound check in code chunking

fix boundary condition check in PUSH32

add circleci support (#32)

* add circleci support

* disable linter, which is broken again

* actually run tests

remove unnecessary cancun block declaration in tests (#33)

upgrade go version (#34)

fix calculation in get_tree_key_for_storage_slot (#35)

use the witness in statedb, revert applyTx signature (#36)

* use the witness in statedb, revert applyTx signature

* fix miner tests

* fix catalyst build

Remove access witness from the signature of Process (#38)

consensus/ethash: ensure uncle accounts are included in block witness (#40)

consensus/ethash: move accumulation of coinbase witness before coinbase account is credited (#41)

remove outdated comment

miner: embed verkle proof in sealing block (#39)

* miner: embed verkle proof in sealing block

* add test to ensure that verkle proof is present in mined blocks

Refactor witness-accumulation in EVM (#42)

* make push dynamically-charged.  charge witness gas costs for push.  refactor evm witness gas charging to move logic for touching a range of bytecode into a helper method 'touchEachChunksAndChargeGas'

* add witness gas calculation for CodeCopy, ExtCodeCopy, SLoad back to gas_table.go

* witness gas charging for CALL

* remove explicit reference to evm.TxContext

* core/vm: make touchEachChunksAndCharge gas handle nil code value

* core/vm: call implementation, separate out witnesses into touch/set

* some fixes

* remove witness touching from opCall: this will go in evm.go

* remove witness touching for call from gas_table.go

* (hopefully) fix tests

* add SSTORE witness charging that was removed mistakenly

* charge witness gas for call

* clean up and comment touchEachChunksAndChargeGas

* make suggested changes

* address remaining points

* fix build issues

* remove double-charging for contract creation witness gas charging

call onleaf in verkle commit (#45)

replace sha256 with pedersen_hash in get_key (#46)

* replace sha256 with pedersen_hash

* fix: prevent an OOB

* workaround timeout in unit test

* update go-ipa and reduce the timeout

* fix for unit tests: do not call NewAccessWitness in NewEVMTxContext (#49)

* potential fix: do not call NewAccessWitness in NewEVMTxContext

* more fixes: check for the existence of Accesses

* fix absence of witness in copy

* fix another witness issue

* workaround: ensure the prefetcher is off in verkle mode

* fix the remaining issues in tests

* review feedback

* fix witness allocation in stateless test

reactivate working lines in test

fix: don't create settings each time a key is calculated (#53)

fix: don't use rlp in the tree for slot values (#51)

* fix: don't use rlp in the tree for slot values

* fix timeout after rebase

core/vm: don't include contract deployer bytecode in AccessWitness or charge witness access costs for it (#54)

Verkle EXTCODECOPY implementation (#55)

* core/vm: verkle extcodecopy naive way (do jumpdest analysis on target contract every EXTCODECOPY)

* no double-charge

* address edge-case in touchEachChunksAndChargeGas

* simplify line

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Use IsCancun where applicable (#56)

* replace Accesses != nil with IsCancun(...)

* fix

Charge witness gas when calling/creating a contract (#60)

* Charge witness gas when calling/creating a contract

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

* gofmt

* replace checks with evm.Access!=nil with IsCancun

* remove double-charging of witness access costs for contract creation initialization

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

verkle proof deserialization (#61)

* use proof serialization

* remove cruft

* save current state

* fix most issues up to this point

* fix remaining build issues

* update the go.mod to use the right branch

* remove custom-defined set type

* update go-verkle to get merged PRs

* extract key, value data from proof

* only activate precomp calculations if this is a verkle chain

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

fix: build more than one block in stateless test (#66)

* reproduce the bug

* fix the nil AccessWitness when Resetting

* fix nonce management in blocks

* fix: make sure the snapshot is reused during the chain generation

Fix GetTreeKey (#65)

* trie/utils: fix GetTreeKey

* disable the faulty test, tracking issue opened

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Implement latest stateless gas charging spec (#63)

* latest stateless gas-charging spec: add write event charging

* rename gas calculation function so that it doesn't seem it's charging gas

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Various fixes for rust-verkle proof format compatibility (#67)

* code to extract the block

* separate proof from keys in block

* display state root of block 0

* change file name to reflect the correct block number

* use RLP instead of flat binary for keyvals in block

* update go-verkle to fix build

* fix rebase issues

* make test pass

* fix issue in map copy

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

Fix contract creation aw (#68)

* fix contract creation AccessWitness

* remove tracing

* remove tracing

* remove block2.rlp

* move repeated error message into its own error type

fix a couple linter issues

Sload fix (#69)

* move SLOAD gas calculation to gasSLoadEIP2929

* re-add gasSLoad, used by SSTORE

* make requested changes

fix: witness gas costs being charged more than once (#70)

* fix: witness gas costs being charged more than one

* update test case

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

re-enable VerkleTrie::ToDot

core/state: implement GetCodeSize for VerkleDB (#75)

implement core/state: implement GetCodeSize for VerkleDB

copy the pre-state, use an untouched copy for the proof (#72)

test to compare the proof format with that of rust verkle (#73)

* test to compare the proof format with that of rust verkle

* comment block extraction code

check proof in verkle tree test

don't delete from the trie when we encounter empty state object if verkle is enabled (#74)

core/state: write code to rawdb under codehash key when verkle is enabled (#76)

force the 32-byte alignment of nonce and balance (#71)

* force the 32-byte alignment of nonce and balance

* review feedback: fix endianness in output

core/state - ensure storage value is set properly in snapshot when verkle is enabled (#78)

fixes for the return type change in go-verkle (#77)

* fixes for the return type change in go-verkle

* update go-verkle

* fix endianness issue + workaround for witness fix

* add changes to miner

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

calculate tree index correctly for SLOAD access event (#79)

core/vm: witness write event for SSTORE (#80)

* core/vm: witness write event for SSTORE

* remove TODO that is addressed by this branch

fix estimateGas (handle ErrInsufficientBalanceWitness).  Return proper error message when ErrInsufficientBalanceWitness is encountered (#81)

Verkle/fix/perdersen hash (#84)

* remove ineffective statement

* fix endianness in pedersen_hash

refactor: set witness account values in getStateObject() (#83)

* enforce 32-byte alignment

* save current state

* write account values to witness from getObject

* code cleanup + sanity checks

* fix RLP serialization of missing keys

* remove code redundancy for GetTreeKeyCodeChunk

* fix stem calculation issue for code and storage

* remove redundant SetLeafValue calls

* Add a contract creation tx to the test

* fix botched module version update

* detail gas calculation cost in verkle contract deployment

fix verkle trie iterator to not miss leaf values in the first index of a leaf node (#88)

creation-time CODECOPY adds no code to the witness (#86)

* creation-time CODECOPY adds no code to the witness

* Add a more complex test to ensure EXTCODECOPY is called

* Fix offset calculations in the slot function

* minor tweaks

* fix initial value capture for storage

* fix build

* fix: get the proper byte in get_tree_key_for_storage_slot

* fix length calculation in getDataAndAdjustedBounds

* fix verkle test

small simplification in order to facilitate rebase

upgrade go-ipa + go-verkle: faster precomp loading

fix: place slot value, not its rlp encoding in witness

fix: code chunification with incorrect header (#89)

* fix: code chunification with incorrect header

* add a test from rust-verkle and fix offset bug

* fix ci build

* complete coverage of rust-verkle chunking tests

* fix: make sure the witness value is set

* add another code chunking test from rust-verkle

spec: don't add the trailing pushdata for a final PUSHn (#90)
gballet added a commit that referenced this pull request May 2, 2022
verkle: Implement Trie, NodeIterator and Database ifs

Fix crash in TestDump

Fix TestDump

Fix TrieCopy

remove unnecessary traces

fix: Error() returned errIteratorEnd in verkle node iterator

rewrite the iterator and change the signature of OpenStorageTrie

add the adapter to reuse the account trie for storage

don't try to deserialize a storage leaf into an account

Fix statedb unit tests (#14)

* debug code

* Fix more unit tests

* remove traces

* Go back to the full range

One tree to rule them all

remove updateRoot, there is no root to update

store code inside the account leaf

fix build

save current state for Sina

Update go-verkle to latest

Charge WITNESS_*_COST gas on storage loads

Add witness costs for SSTORE as well

Charge witness gas in the case of code execution

corresponding code deletion

add a --verkle flag to separate verkle experiments from regular geth operations

use the snapshot to get data

stateless execution from block witness

AccessWitness functions

Add block generation test + genesis snapshot generation

test stateless block execution (#18)

* test stateless block execution

* Force tree resolution before generating the proof

increased coverage in stateless test execution (#19)

* test stateless block execution

* Force tree resolution before generating the proof

* increase coverage in stateless test execution

ensure geth compiles

fix issues in tests with verkle trees deactivated

Ensure stateless data is available when executing statelessly (#20)

* Ensure stateless data is available when executing statelessly

* Actual execution of a statless block

* bugfixes in stateless block execution

* code cleanup

 - Reduce PR footprint by reverting NewEVM to its original signature
 - Move the access witness to the block context
 - prepare for a change in AW semantics
   Need to store the initial values.
 - Use the touch helper function, DRY

* revert the signature of MustCommit to its original form (#21)

fix leaf proofs in stateless execution (#22)

* Fixes in witness pre-state

* Add the recipient's nonce to the witness

* reduce PR footprint and investigate issue in root state calculation

* quick build fix

cleanup: Remove extra parameter in ToBlock

revert ToBlock to its older signature

fix import cycle in vm tests

fix linter issue

fix appveyor build

fix nil pointers in tests

Add indices, yis and Cis to the block's Verkle proof

upgrade geth dependency to drop geth's common dep

fix cmd/devp2p tests

fix rebase issues

quell an appveyor warning

fix address touching in SLOAD and SSTORE

fix access witness for code size

touch target account data before calling

make sure the proper locations get touched in (ext)codecopy

touch all code pages in execution

add pushdata to witness

remove useless code in genesis snapshot generation

testnet: fix some of the rebase/drift issues

Fix verkle proof generation in block

fix an issue occuring when chunking past the code size

fix: ensure the code copy doesn't extend past the code size

Upgrade go-verkle to its IPA version (#24)

fixes for the IPA testnet

upgrade to latest go-verkle

update go-verkle to get more fixes

simplify code by removing all stateless references (#25)

fix verkle proof test by enforcing values alignment to 32 bytes

remove unneeded KZG tag

fix the stateless test

Move AccessWitness into StateDB (#27)

* move AccessWitness into StateDB

* set Accesses in TxContext constructor

* Ensures that a statedb is initialized with a witness

* copy AccessWitness in StateDB.Copy.  use copied state in miner worker.commit.

* remove redundant line

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Fix contract creation issue

enable verkle on cancun block: take 2 (#28)

* enable verkle on cancun block: take 2

* fix typo.  make unreachable line panic message more clear

fix rebase issues

fix linter issue

merge undefined instead of panicking (#30)

initialize the new access witness if not already present

fix boundary check in PUSH

fix bound check in code chunking

fix boundary condition check in PUSH32

add circleci support (#32)

* add circleci support

* disable linter, which is broken again

* actually run tests

remove unnecessary cancun block declaration in tests (#33)

upgrade go version (#34)

fix calculation in get_tree_key_for_storage_slot (#35)

use the witness in statedb, revert applyTx signature (#36)

* use the witness in statedb, revert applyTx signature

* fix miner tests

* fix catalyst build

Remove access witness from the signature of Process (#38)

consensus/ethash: ensure uncle accounts are included in block witness (#40)

consensus/ethash: move accumulation of coinbase witness before coinbase account is credited (#41)

remove outdated comment

miner: embed verkle proof in sealing block (#39)

* miner: embed verkle proof in sealing block

* add test to ensure that verkle proof is present in mined blocks

Refactor witness-accumulation in EVM (#42)

* make push dynamically-charged.  charge witness gas costs for push.  refactor evm witness gas charging to move logic for touching a range of bytecode into a helper method 'touchEachChunksAndChargeGas'

* add witness gas calculation for CodeCopy, ExtCodeCopy, SLoad back to gas_table.go

* witness gas charging for CALL

* remove explicit reference to evm.TxContext

* core/vm: make touchEachChunksAndCharge gas handle nil code value

* core/vm: call implementation, separate out witnesses into touch/set

* some fixes

* remove witness touching from opCall: this will go in evm.go

* remove witness touching for call from gas_table.go

* (hopefully) fix tests

* add SSTORE witness charging that was removed mistakenly

* charge witness gas for call

* clean up and comment touchEachChunksAndChargeGas

* make suggested changes

* address remaining points

* fix build issues

* remove double-charging for contract creation witness gas charging

call onleaf in verkle commit (#45)

replace sha256 with pedersen_hash in get_key (#46)

* replace sha256 with pedersen_hash

* fix: prevent an OOB

* workaround timeout in unit test

* update go-ipa and reduce the timeout

* fix for unit tests: do not call NewAccessWitness in NewEVMTxContext (#49)

* potential fix: do not call NewAccessWitness in NewEVMTxContext

* more fixes: check for the existence of Accesses

* fix absence of witness in copy

* fix another witness issue

* workaround: ensure the prefetcher is off in verkle mode

* fix the remaining issues in tests

* review feedback

* fix witness allocation in stateless test

reactivate working lines in test

fix: don't create settings each time a key is calculated (#53)

fix: don't use rlp in the tree for slot values (#51)

* fix: don't use rlp in the tree for slot values

* fix timeout after rebase

core/vm: don't include contract deployer bytecode in AccessWitness or charge witness access costs for it (#54)

Verkle EXTCODECOPY implementation (#55)

* core/vm: verkle extcodecopy naive way (do jumpdest analysis on target contract every EXTCODECOPY)

* no double-charge

* address edge-case in touchEachChunksAndChargeGas

* simplify line

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Use IsCancun where applicable (#56)

* replace Accesses != nil with IsCancun(...)

* fix

Charge witness gas when calling/creating a contract (#60)

* Charge witness gas when calling/creating a contract

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

* gofmt

* replace checks with evm.Access!=nil with IsCancun

* remove double-charging of witness access costs for contract creation initialization

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

verkle proof deserialization (#61)

* use proof serialization

* remove cruft

* save current state

* fix most issues up to this point

* fix remaining build issues

* update the go.mod to use the right branch

* remove custom-defined set type

* update go-verkle to get merged PRs

* extract key, value data from proof

* only activate precomp calculations if this is a verkle chain

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

fix: build more than one block in stateless test (#66)

* reproduce the bug

* fix the nil AccessWitness when Resetting

* fix nonce management in blocks

* fix: make sure the snapshot is reused during the chain generation

Fix GetTreeKey (#65)

* trie/utils: fix GetTreeKey

* disable the faulty test, tracking issue opened

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Implement latest stateless gas charging spec (#63)

* latest stateless gas-charging spec: add write event charging

* rename gas calculation function so that it doesn't seem it's charging gas

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Various fixes for rust-verkle proof format compatibility (#67)

* code to extract the block

* separate proof from keys in block

* display state root of block 0

* change file name to reflect the correct block number

* use RLP instead of flat binary for keyvals in block

* update go-verkle to fix build

* fix rebase issues

* make test pass

* fix issue in map copy

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

Fix contract creation aw (#68)

* fix contract creation AccessWitness

* remove tracing

* remove tracing

* remove block2.rlp

* move repeated error message into its own error type

fix a couple linter issues

Sload fix (#69)

* move SLOAD gas calculation to gasSLoadEIP2929

* re-add gasSLoad, used by SSTORE

* make requested changes

fix: witness gas costs being charged more than once (#70)

* fix: witness gas costs being charged more than one

* update test case

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

re-enable VerkleTrie::ToDot

core/state: implement GetCodeSize for VerkleDB (#75)

implement core/state: implement GetCodeSize for VerkleDB

copy the pre-state, use an untouched copy for the proof (#72)

test to compare the proof format with that of rust verkle (#73)

* test to compare the proof format with that of rust verkle

* comment block extraction code

check proof in verkle tree test

don't delete from the trie when we encounter empty state object if verkle is enabled (#74)

core/state: write code to rawdb under codehash key when verkle is enabled (#76)

force the 32-byte alignment of nonce and balance (#71)

* force the 32-byte alignment of nonce and balance

* review feedback: fix endianness in output

core/state - ensure storage value is set properly in snapshot when verkle is enabled (#78)

fixes for the return type change in go-verkle (#77)

* fixes for the return type change in go-verkle

* update go-verkle

* fix endianness issue + workaround for witness fix

* add changes to miner

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

calculate tree index correctly for SLOAD access event (#79)

core/vm: witness write event for SSTORE (#80)

* core/vm: witness write event for SSTORE

* remove TODO that is addressed by this branch

fix estimateGas (handle ErrInsufficientBalanceWitness).  Return proper error message when ErrInsufficientBalanceWitness is encountered (#81)

Verkle/fix/perdersen hash (#84)

* remove ineffective statement

* fix endianness in pedersen_hash

refactor: set witness account values in getStateObject() (#83)

* enforce 32-byte alignment

* save current state

* write account values to witness from getObject

* code cleanup + sanity checks

* fix RLP serialization of missing keys

* remove code redundancy for GetTreeKeyCodeChunk

* fix stem calculation issue for code and storage

* remove redundant SetLeafValue calls

* Add a contract creation tx to the test

* fix botched module version update

* detail gas calculation cost in verkle contract deployment

fix verkle trie iterator to not miss leaf values in the first index of a leaf node (#88)

creation-time CODECOPY adds no code to the witness (#86)

* creation-time CODECOPY adds no code to the witness

* Add a more complex test to ensure EXTCODECOPY is called

* Fix offset calculations in the slot function

* minor tweaks

* fix initial value capture for storage

* fix build

* fix: get the proper byte in get_tree_key_for_storage_slot

* fix length calculation in getDataAndAdjustedBounds

* fix verkle test

small simplification in order to facilitate rebase

upgrade go-ipa + go-verkle: faster precomp loading

fix: place slot value, not its rlp encoding in witness

fix: code chunification with incorrect header (#89)

* fix: code chunification with incorrect header

* add a test from rust-verkle and fix offset bug

* fix ci build

* complete coverage of rust-verkle chunking tests

* fix: make sure the witness value is set

* add another code chunking test from rust-verkle

spec: don't add the trailing pushdata for a final PUSHn (#90)

cleanup: don't evaluate 0 coeffs in get_tree_key (#87)

ensure no overflow occurs in the chunk touch function

load the full coinbase account in reward

fix: previous commit broke stateless test

rework the way the gas is charged per code chunk

update go-verkle to grab the fix for #200

fix: don't add PUSH1-data during contract deployment (#91)

* fix: don't add PUSH1-data during contract deployment

* remove import cycle in tests (#92)

fix: redundant proof-of-absence stems (#93)

test: missing children of the same internal node produce a single extension status (#94)

ParseNode with commitment parameter (#95)

review feedback for rebase over kiln (#97)

safe gas add (#98)

* safe gas add

* fix build

ensure 10-byte addresses are right-aligned in pedersen_hash (#99)

Co-authored-by: Tanishq Jasoria <jasoriatanishq@gmail.com>
gballet added a commit that referenced this pull request Jul 1, 2022
verkle: Implement Trie, NodeIterator and Database ifs

Fix crash in TestDump

Fix TestDump

Fix TrieCopy

remove unnecessary traces

fix: Error() returned errIteratorEnd in verkle node iterator

rewrite the iterator and change the signature of OpenStorageTrie

add the adapter to reuse the account trie for storage

don't try to deserialize a storage leaf into an account

Fix statedb unit tests (#14)

* debug code

* Fix more unit tests

* remove traces

* Go back to the full range

One tree to rule them all

remove updateRoot, there is no root to update

store code inside the account leaf

fix build

save current state for Sina

Update go-verkle to latest

Charge WITNESS_*_COST gas on storage loads

Add witness costs for SSTORE as well

Charge witness gas in the case of code execution

corresponding code deletion

add a --verkle flag to separate verkle experiments from regular geth operations

use the snapshot to get data

stateless execution from block witness

AccessWitness functions

Add block generation test + genesis snapshot generation

test stateless block execution (#18)

* test stateless block execution

* Force tree resolution before generating the proof

increased coverage in stateless test execution (#19)

* test stateless block execution

* Force tree resolution before generating the proof

* increase coverage in stateless test execution

ensure geth compiles

fix issues in tests with verkle trees deactivated

Ensure stateless data is available when executing statelessly (#20)

* Ensure stateless data is available when executing statelessly

* Actual execution of a statless block

* bugfixes in stateless block execution

* code cleanup

 - Reduce PR footprint by reverting NewEVM to its original signature
 - Move the access witness to the block context
 - prepare for a change in AW semantics
   Need to store the initial values.
 - Use the touch helper function, DRY

* revert the signature of MustCommit to its original form (#21)

fix leaf proofs in stateless execution (#22)

* Fixes in witness pre-state

* Add the recipient's nonce to the witness

* reduce PR footprint and investigate issue in root state calculation

* quick build fix

cleanup: Remove extra parameter in ToBlock

revert ToBlock to its older signature

fix import cycle in vm tests

fix linter issue

fix appveyor build

fix nil pointers in tests

Add indices, yis and Cis to the block's Verkle proof

upgrade geth dependency to drop geth's common dep

fix cmd/devp2p tests

fix rebase issues

quell an appveyor warning

fix address touching in SLOAD and SSTORE

fix access witness for code size

touch target account data before calling

make sure the proper locations get touched in (ext)codecopy

touch all code pages in execution

add pushdata to witness

remove useless code in genesis snapshot generation

testnet: fix some of the rebase/drift issues

Fix verkle proof generation in block

fix an issue occuring when chunking past the code size

fix: ensure the code copy doesn't extend past the code size

Upgrade go-verkle to its IPA version (#24)

fixes for the IPA testnet

upgrade to latest go-verkle

update go-verkle to get more fixes

simplify code by removing all stateless references (#25)

fix verkle proof test by enforcing values alignment to 32 bytes

remove unneeded KZG tag

fix the stateless test

Move AccessWitness into StateDB (#27)

* move AccessWitness into StateDB

* set Accesses in TxContext constructor

* Ensures that a statedb is initialized with a witness

* copy AccessWitness in StateDB.Copy.  use copied state in miner worker.commit.

* remove redundant line

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Fix contract creation issue

enable verkle on cancun block: take 2 (#28)

* enable verkle on cancun block: take 2

* fix typo.  make unreachable line panic message more clear

fix rebase issues

fix linter issue

merge undefined instead of panicking (#30)

initialize the new access witness if not already present

fix boundary check in PUSH

fix bound check in code chunking

fix boundary condition check in PUSH32

add circleci support (#32)

* add circleci support

* disable linter, which is broken again

* actually run tests

remove unnecessary cancun block declaration in tests (#33)

upgrade go version (#34)

fix calculation in get_tree_key_for_storage_slot (#35)

use the witness in statedb, revert applyTx signature (#36)

* use the witness in statedb, revert applyTx signature

* fix miner tests

* fix catalyst build

Remove access witness from the signature of Process (#38)

consensus/ethash: ensure uncle accounts are included in block witness (#40)

consensus/ethash: move accumulation of coinbase witness before coinbase account is credited (#41)

remove outdated comment

miner: embed verkle proof in sealing block (#39)

* miner: embed verkle proof in sealing block

* add test to ensure that verkle proof is present in mined blocks

Refactor witness-accumulation in EVM (#42)

* make push dynamically-charged.  charge witness gas costs for push.  refactor evm witness gas charging to move logic for touching a range of bytecode into a helper method 'touchEachChunksAndChargeGas'

* add witness gas calculation for CodeCopy, ExtCodeCopy, SLoad back to gas_table.go

* witness gas charging for CALL

* remove explicit reference to evm.TxContext

* core/vm: make touchEachChunksAndCharge gas handle nil code value

* core/vm: call implementation, separate out witnesses into touch/set

* some fixes

* remove witness touching from opCall: this will go in evm.go

* remove witness touching for call from gas_table.go

* (hopefully) fix tests

* add SSTORE witness charging that was removed mistakenly

* charge witness gas for call

* clean up and comment touchEachChunksAndChargeGas

* make suggested changes

* address remaining points

* fix build issues

* remove double-charging for contract creation witness gas charging

call onleaf in verkle commit (#45)

replace sha256 with pedersen_hash in get_key (#46)

* replace sha256 with pedersen_hash

* fix: prevent an OOB

* workaround timeout in unit test

* update go-ipa and reduce the timeout

* fix for unit tests: do not call NewAccessWitness in NewEVMTxContext (#49)

* potential fix: do not call NewAccessWitness in NewEVMTxContext

* more fixes: check for the existence of Accesses

* fix absence of witness in copy

* fix another witness issue

* workaround: ensure the prefetcher is off in verkle mode

* fix the remaining issues in tests

* review feedback

* fix witness allocation in stateless test

reactivate working lines in test

fix: don't create settings each time a key is calculated (#53)

fix: don't use rlp in the tree for slot values (#51)

* fix: don't use rlp in the tree for slot values

* fix timeout after rebase

core/vm: don't include contract deployer bytecode in AccessWitness or charge witness access costs for it (#54)

Verkle EXTCODECOPY implementation (#55)

* core/vm: verkle extcodecopy naive way (do jumpdest analysis on target contract every EXTCODECOPY)

* no double-charge

* address edge-case in touchEachChunksAndChargeGas

* simplify line

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Use IsCancun where applicable (#56)

* replace Accesses != nil with IsCancun(...)

* fix

Charge witness gas when calling/creating a contract (#60)

* Charge witness gas when calling/creating a contract

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

* gofmt

* replace checks with evm.Access!=nil with IsCancun

* remove double-charging of witness access costs for contract creation initialization

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

verkle proof deserialization (#61)

* use proof serialization

* remove cruft

* save current state

* fix most issues up to this point

* fix remaining build issues

* update the go.mod to use the right branch

* remove custom-defined set type

* update go-verkle to get merged PRs

* extract key, value data from proof

* only activate precomp calculations if this is a verkle chain

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

fix: build more than one block in stateless test (#66)

* reproduce the bug

* fix the nil AccessWitness when Resetting

* fix nonce management in blocks

* fix: make sure the snapshot is reused during the chain generation

Fix GetTreeKey (#65)

* trie/utils: fix GetTreeKey

* disable the faulty test, tracking issue opened

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Implement latest stateless gas charging spec (#63)

* latest stateless gas-charging spec: add write event charging

* rename gas calculation function so that it doesn't seem it's charging gas

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Various fixes for rust-verkle proof format compatibility (#67)

* code to extract the block

* separate proof from keys in block

* display state root of block 0

* change file name to reflect the correct block number

* use RLP instead of flat binary for keyvals in block

* update go-verkle to fix build

* fix rebase issues

* make test pass

* fix issue in map copy

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

Fix contract creation aw (#68)

* fix contract creation AccessWitness

* remove tracing

* remove tracing

* remove block2.rlp

* move repeated error message into its own error type

fix a couple linter issues

Sload fix (#69)

* move SLOAD gas calculation to gasSLoadEIP2929

* re-add gasSLoad, used by SSTORE

* make requested changes

fix: witness gas costs being charged more than once (#70)

* fix: witness gas costs being charged more than one

* update test case

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

re-enable VerkleTrie::ToDot

core/state: implement GetCodeSize for VerkleDB (#75)

implement core/state: implement GetCodeSize for VerkleDB

copy the pre-state, use an untouched copy for the proof (#72)

test to compare the proof format with that of rust verkle (#73)

* test to compare the proof format with that of rust verkle

* comment block extraction code

check proof in verkle tree test

don't delete from the trie when we encounter empty state object if verkle is enabled (#74)

core/state: write code to rawdb under codehash key when verkle is enabled (#76)

force the 32-byte alignment of nonce and balance (#71)

* force the 32-byte alignment of nonce and balance

* review feedback: fix endianness in output

core/state - ensure storage value is set properly in snapshot when verkle is enabled (#78)

fixes for the return type change in go-verkle (#77)

* fixes for the return type change in go-verkle

* update go-verkle

* fix endianness issue + workaround for witness fix

* add changes to miner

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

calculate tree index correctly for SLOAD access event (#79)

core/vm: witness write event for SSTORE (#80)

* core/vm: witness write event for SSTORE

* remove TODO that is addressed by this branch

fix estimateGas (handle ErrInsufficientBalanceWitness).  Return proper error message when ErrInsufficientBalanceWitness is encountered (#81)

Verkle/fix/perdersen hash (#84)

* remove ineffective statement

* fix endianness in pedersen_hash

refactor: set witness account values in getStateObject() (#83)

* enforce 32-byte alignment

* save current state

* write account values to witness from getObject

* code cleanup + sanity checks

* fix RLP serialization of missing keys

* remove code redundancy for GetTreeKeyCodeChunk

* fix stem calculation issue for code and storage

* remove redundant SetLeafValue calls

* Add a contract creation tx to the test

* fix botched module version update

* detail gas calculation cost in verkle contract deployment

fix verkle trie iterator to not miss leaf values in the first index of a leaf node (#88)

creation-time CODECOPY adds no code to the witness (#86)

* creation-time CODECOPY adds no code to the witness

* Add a more complex test to ensure EXTCODECOPY is called

* Fix offset calculations in the slot function

* minor tweaks

* fix initial value capture for storage

* fix build

* fix: get the proper byte in get_tree_key_for_storage_slot

* fix length calculation in getDataAndAdjustedBounds

* fix verkle test

small simplification in order to facilitate rebase

upgrade go-ipa + go-verkle: faster precomp loading

fix: place slot value, not its rlp encoding in witness

fix: code chunification with incorrect header (#89)

* fix: code chunification with incorrect header

* add a test from rust-verkle and fix offset bug

* fix ci build

* complete coverage of rust-verkle chunking tests

* fix: make sure the witness value is set

* add another code chunking test from rust-verkle

spec: don't add the trailing pushdata for a final PUSHn (#90)

cleanup: don't evaluate 0 coeffs in get_tree_key (#87)

ensure no overflow occurs in the chunk touch function

load the full coinbase account in reward

fix: previous commit broke stateless test

rework the way the gas is charged per code chunk

update go-verkle to grab the fix for #200

fix: don't add PUSH1-data during contract deployment (#91)

* fix: don't add PUSH1-data during contract deployment

* remove import cycle in tests (#92)

fix: redundant proof-of-absence stems (#93)

test: missing children of the same internal node produce a single extension status (#94)

ParseNode with commitment parameter (#95)

review feedback for rebase over kiln (#97)

safe gas add (#98)

* safe gas add

* fix build

ensure 10-byte addresses are right-aligned in pedersen_hash (#99)

Co-authored-by: Tanishq Jasoria <jasoriatanishq@gmail.com>

Remove the code analysis interface to reduce PR footprint (#100)

* Remove the code analysis interface to reduce PR footprint

* fix unit test

fix: set IsDeployment=false outside of contract creation (#101)

remove trailing line

verkle test: count the code chunks in the witness (#102)

temp fix: little endian pedersen_hash (#103)

factor the amount of polynomial evaluations in access witness (#106)

reuse polynomial evaluation in account update (#108)

fix: remove redundant balance touching in access witness (#107)

add github workflow

reuse address point evaluation in code chunking (#109)
gballet added a commit that referenced this pull request Jul 8, 2022
verkle: Implement Trie, NodeIterator and Database ifs

Fix crash in TestDump

Fix TestDump

Fix TrieCopy

remove unnecessary traces

fix: Error() returned errIteratorEnd in verkle node iterator

rewrite the iterator and change the signature of OpenStorageTrie

add the adapter to reuse the account trie for storage

don't try to deserialize a storage leaf into an account

Fix statedb unit tests (#14)

* debug code

* Fix more unit tests

* remove traces

* Go back to the full range

One tree to rule them all

remove updateRoot, there is no root to update

store code inside the account leaf

fix build

save current state for Sina

Update go-verkle to latest

Charge WITNESS_*_COST gas on storage loads

Add witness costs for SSTORE as well

Charge witness gas in the case of code execution

corresponding code deletion

add a --verkle flag to separate verkle experiments from regular geth operations

use the snapshot to get data

stateless execution from block witness

AccessWitness functions

Add block generation test + genesis snapshot generation

test stateless block execution (#18)

* test stateless block execution

* Force tree resolution before generating the proof

increased coverage in stateless test execution (#19)

* test stateless block execution

* Force tree resolution before generating the proof

* increase coverage in stateless test execution

ensure geth compiles

fix issues in tests with verkle trees deactivated

Ensure stateless data is available when executing statelessly (#20)

* Ensure stateless data is available when executing statelessly

* Actual execution of a statless block

* bugfixes in stateless block execution

* code cleanup

 - Reduce PR footprint by reverting NewEVM to its original signature
 - Move the access witness to the block context
 - prepare for a change in AW semantics
   Need to store the initial values.
 - Use the touch helper function, DRY

* revert the signature of MustCommit to its original form (#21)

fix leaf proofs in stateless execution (#22)

* Fixes in witness pre-state

* Add the recipient's nonce to the witness

* reduce PR footprint and investigate issue in root state calculation

* quick build fix

cleanup: Remove extra parameter in ToBlock

revert ToBlock to its older signature

fix import cycle in vm tests

fix linter issue

fix appveyor build

fix nil pointers in tests

Add indices, yis and Cis to the block's Verkle proof

upgrade geth dependency to drop geth's common dep

fix cmd/devp2p tests

fix rebase issues

quell an appveyor warning

fix address touching in SLOAD and SSTORE

fix access witness for code size

touch target account data before calling

make sure the proper locations get touched in (ext)codecopy

touch all code pages in execution

add pushdata to witness

remove useless code in genesis snapshot generation

testnet: fix some of the rebase/drift issues

Fix verkle proof generation in block

fix an issue occuring when chunking past the code size

fix: ensure the code copy doesn't extend past the code size

Upgrade go-verkle to its IPA version (#24)

fixes for the IPA testnet

upgrade to latest go-verkle

update go-verkle to get more fixes

simplify code by removing all stateless references (#25)

fix verkle proof test by enforcing values alignment to 32 bytes

remove unneeded KZG tag

fix the stateless test

Move AccessWitness into StateDB (#27)

* move AccessWitness into StateDB

* set Accesses in TxContext constructor

* Ensures that a statedb is initialized with a witness

* copy AccessWitness in StateDB.Copy.  use copied state in miner worker.commit.

* remove redundant line

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Fix contract creation issue

enable verkle on cancun block: take 2 (#28)

* enable verkle on cancun block: take 2

* fix typo.  make unreachable line panic message more clear

fix rebase issues

fix linter issue

merge undefined instead of panicking (#30)

initialize the new access witness if not already present

fix boundary check in PUSH

fix bound check in code chunking

fix boundary condition check in PUSH32

add circleci support (#32)

* add circleci support

* disable linter, which is broken again

* actually run tests

remove unnecessary cancun block declaration in tests (#33)

upgrade go version (#34)

fix calculation in get_tree_key_for_storage_slot (#35)

use the witness in statedb, revert applyTx signature (#36)

* use the witness in statedb, revert applyTx signature

* fix miner tests

* fix catalyst build

Remove access witness from the signature of Process (#38)

consensus/ethash: ensure uncle accounts are included in block witness (#40)

consensus/ethash: move accumulation of coinbase witness before coinbase account is credited (#41)

remove outdated comment

miner: embed verkle proof in sealing block (#39)

* miner: embed verkle proof in sealing block

* add test to ensure that verkle proof is present in mined blocks

Refactor witness-accumulation in EVM (#42)

* make push dynamically-charged.  charge witness gas costs for push.  refactor evm witness gas charging to move logic for touching a range of bytecode into a helper method 'touchEachChunksAndChargeGas'

* add witness gas calculation for CodeCopy, ExtCodeCopy, SLoad back to gas_table.go

* witness gas charging for CALL

* remove explicit reference to evm.TxContext

* core/vm: make touchEachChunksAndCharge gas handle nil code value

* core/vm: call implementation, separate out witnesses into touch/set

* some fixes

* remove witness touching from opCall: this will go in evm.go

* remove witness touching for call from gas_table.go

* (hopefully) fix tests

* add SSTORE witness charging that was removed mistakenly

* charge witness gas for call

* clean up and comment touchEachChunksAndChargeGas

* make suggested changes

* address remaining points

* fix build issues

* remove double-charging for contract creation witness gas charging

call onleaf in verkle commit (#45)

replace sha256 with pedersen_hash in get_key (#46)

* replace sha256 with pedersen_hash

* fix: prevent an OOB

* workaround timeout in unit test

* update go-ipa and reduce the timeout

* fix for unit tests: do not call NewAccessWitness in NewEVMTxContext (#49)

* potential fix: do not call NewAccessWitness in NewEVMTxContext

* more fixes: check for the existence of Accesses

* fix absence of witness in copy

* fix another witness issue

* workaround: ensure the prefetcher is off in verkle mode

* fix the remaining issues in tests

* review feedback

* fix witness allocation in stateless test

reactivate working lines in test

fix: don't create settings each time a key is calculated (#53)

fix: don't use rlp in the tree for slot values (#51)

* fix: don't use rlp in the tree for slot values

* fix timeout after rebase

core/vm: don't include contract deployer bytecode in AccessWitness or charge witness access costs for it (#54)

Verkle EXTCODECOPY implementation (#55)

* core/vm: verkle extcodecopy naive way (do jumpdest analysis on target contract every EXTCODECOPY)

* no double-charge

* address edge-case in touchEachChunksAndChargeGas

* simplify line

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Use IsCancun where applicable (#56)

* replace Accesses != nil with IsCancun(...)

* fix

Charge witness gas when calling/creating a contract (#60)

* Charge witness gas when calling/creating a contract

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

* gofmt

* replace checks with evm.Access!=nil with IsCancun

* remove double-charging of witness access costs for contract creation initialization

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

verkle proof deserialization (#61)

* use proof serialization

* remove cruft

* save current state

* fix most issues up to this point

* fix remaining build issues

* update the go.mod to use the right branch

* remove custom-defined set type

* update go-verkle to get merged PRs

* extract key, value data from proof

* only activate precomp calculations if this is a verkle chain

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

fix: build more than one block in stateless test (#66)

* reproduce the bug

* fix the nil AccessWitness when Resetting

* fix nonce management in blocks

* fix: make sure the snapshot is reused during the chain generation

Fix GetTreeKey (#65)

* trie/utils: fix GetTreeKey

* disable the faulty test, tracking issue opened

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Implement latest stateless gas charging spec (#63)

* latest stateless gas-charging spec: add write event charging

* rename gas calculation function so that it doesn't seem it's charging gas

Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>

Various fixes for rust-verkle proof format compatibility (#67)

* code to extract the block

* separate proof from keys in block

* display state root of block 0

* change file name to reflect the correct block number

* use RLP instead of flat binary for keyvals in block

* update go-verkle to fix build

* fix rebase issues

* make test pass

* fix issue in map copy

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

Fix contract creation aw (#68)

* fix contract creation AccessWitness

* remove tracing

* remove tracing

* remove block2.rlp

* move repeated error message into its own error type

fix a couple linter issues

Sload fix (#69)

* move SLOAD gas calculation to gasSLoadEIP2929

* re-add gasSLoad, used by SSTORE

* make requested changes

fix: witness gas costs being charged more than once (#70)

* fix: witness gas costs being charged more than one

* update test case

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

re-enable VerkleTrie::ToDot

core/state: implement GetCodeSize for VerkleDB (#75)

implement core/state: implement GetCodeSize for VerkleDB

copy the pre-state, use an untouched copy for the proof (#72)

test to compare the proof format with that of rust verkle (#73)

* test to compare the proof format with that of rust verkle

* comment block extraction code

check proof in verkle tree test

don't delete from the trie when we encounter empty state object if verkle is enabled (#74)

core/state: write code to rawdb under codehash key when verkle is enabled (#76)

force the 32-byte alignment of nonce and balance (#71)

* force the 32-byte alignment of nonce and balance

* review feedback: fix endianness in output

core/state - ensure storage value is set properly in snapshot when verkle is enabled (#78)

fixes for the return type change in go-verkle (#77)

* fixes for the return type change in go-verkle

* update go-verkle

* fix endianness issue + workaround for witness fix

* add changes to miner

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>

calculate tree index correctly for SLOAD access event (#79)

core/vm: witness write event for SSTORE (#80)

* core/vm: witness write event for SSTORE

* remove TODO that is addressed by this branch

fix estimateGas (handle ErrInsufficientBalanceWitness).  Return proper error message when ErrInsufficientBalanceWitness is encountered (#81)

Verkle/fix/perdersen hash (#84)

* remove ineffective statement

* fix endianness in pedersen_hash

refactor: set witness account values in getStateObject() (#83)

* enforce 32-byte alignment

* save current state

* write account values to witness from getObject

* code cleanup + sanity checks

* fix RLP serialization of missing keys

* remove code redundancy for GetTreeKeyCodeChunk

* fix stem calculation issue for code and storage

* remove redundant SetLeafValue calls

* Add a contract creation tx to the test

* fix botched module version update

* detail gas calculation cost in verkle contract deployment

fix verkle trie iterator to not miss leaf values in the first index of a leaf node (#88)

creation-time CODECOPY adds no code to the witness (#86)

* creation-time CODECOPY adds no code to the witness

* Add a more complex test to ensure EXTCODECOPY is called

* Fix offset calculations in the slot function

* minor tweaks

* fix initial value capture for storage

* fix build

* fix: get the proper byte in get_tree_key_for_storage_slot

* fix length calculation in getDataAndAdjustedBounds

* fix verkle test

small simplification in order to facilitate rebase

upgrade go-ipa + go-verkle: faster precomp loading

fix: place slot value, not its rlp encoding in witness

fix: code chunification with incorrect header (#89)

* fix: code chunification with incorrect header

* add a test from rust-verkle and fix offset bug

* fix ci build

* complete coverage of rust-verkle chunking tests

* fix: make sure the witness value is set

* add another code chunking test from rust-verkle

spec: don't add the trailing pushdata for a final PUSHn (#90)

cleanup: don't evaluate 0 coeffs in get_tree_key (#87)

ensure no overflow occurs in the chunk touch function

load the full coinbase account in reward

fix: previous commit broke stateless test

rework the way the gas is charged per code chunk

update go-verkle to grab the fix for #200

fix: don't add PUSH1-data during contract deployment (#91)

* fix: don't add PUSH1-data during contract deployment

* remove import cycle in tests (#92)

fix: redundant proof-of-absence stems (#93)

test: missing children of the same internal node produce a single extension status (#94)

ParseNode with commitment parameter (#95)

review feedback for rebase over kiln (#97)

safe gas add (#98)

* safe gas add

* fix build

ensure 10-byte addresses are right-aligned in pedersen_hash (#99)

Co-authored-by: Tanishq Jasoria <jasoriatanishq@gmail.com>

Remove the code analysis interface to reduce PR footprint (#100)

* Remove the code analysis interface to reduce PR footprint

* fix unit test

fix: set IsDeployment=false outside of contract creation (#101)

remove trailing line

verkle test: count the code chunks in the witness (#102)

temp fix: little endian pedersen_hash (#103)

factor the amount of polynomial evaluations in access witness (#106)

reuse polynomial evaluation in account update (#108)

fix: remove redundant balance touching in access witness (#107)

add github workflow

reuse address point evaluation in code chunking (#109)
gballet pushed a commit that referenced this pull request Feb 14, 2023
* ethdb: use pebble

Co-authored-by: Gary Rong <garyrong0905@gmail.com>

foo

update

* apply suggested changes

* flags: go format

node: fix ddir lookup mistake

accounts/abi/bind: fix go.mod replacement for generated binding

deps: update pebble + with fix 32-bit build

* ethdb/pebble: respect max memtable size

* core/rawdb, ethdb: enable pebble on non-32bit platforms only

* core/rawdb: fix build tags, fix some review concerns

* core/rawdb: refactor methods for database opening

* core/rawdb: remove erroneous build tag

* cmd/geth: fix the flag default handling + testcase

* cmd/geth: improve testing regarding custom backends

* ethdb/pebble, deps: update pebble dependency

* core/rawdb: replace method with Open

* ethdb/pebble: several updates for pebble (#49)

* ethdb/pebble: fix size count in batch

* ethdb/pebble: disable seek compaction

* ethdb/pebble: more fixes

* ethdb, core, cmd: polish and fixes (#50)

* cmd/utils, core/rawdb, ethdb/pebble: address some review concerns

* Update flags.go

* ethdb/pebble: minor refactors

* ethdb/pebble: avoid copy on batch replay

* ethdb: fix compilation flaw

* cmd: fix test fail due to mismatching error message

* cmd/geth, node: rename backingdb to db.engine

---------

Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants