This repository has been archived by the owner on May 21, 2024. It is now read-only.
Rebase statediffing service with 1.9.9 #2
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Accounts in alloc field must already exist Note that accounts in alloc field must already exist, as pointed out by Simeon Vanov in https://gettoshare.com/2017/10/30/how-to-use-genesis-json-alloc-property/ * Change wording per PR review comment * README: minor fixups
Most of these changes are related to the Go 1.13 changes to test binary flag handling. * cmd/geth: make attach tests more reliable This makes the test wait for the endpoint to come up by polling it instead of waiting for two seconds. * tests: fix test binary flags for Go 1.13 Calling flag.Parse during package initialization is prohibited as of Go 1.13 and causes test failures. Call it in TestMain instead. * crypto/ecies: remove useless -dump flag in tests * p2p/simulations: fix test binary flags for Go 1.13 Calling flag.Parse during package initialization is prohibited as of Go 1.13 and causes test failures. Call it in TestMain instead. * build: remove workaround for ./... vendor matching This workaround was necessary for Go 1.8. The Go 1.9 release changed the expansion rules to exclude vendored packages. * Makefile: use relative path for GOBIN This makes the "Run ./build/bin/..." line look nicer. * les: fix test binary flags for Go 1.13 Calling flag.Parse during package initialization is prohibited as of Go 1.13 and causes test failures. Call it in TestMain instead.
* rlp: improve nil pointer handling
In both encoder and decoder, the rules for encoding nil pointers were a
bit hard to understand, and didn't leave much choice. Since RLP allows
two empty values (empty list, empty string), any protocol built on RLP
must choose either of these values to represent the null value in a
certain context.
This change adds choice in the form of two new struct tags, "nilString"
and "nilList". These can be used to specify how a nil pointer value is
encoded. The "nil" tag still exists, but its implementation is now
explicit and defines exactly how nil pointers are handled in a single
place.
Another important change in this commit is how nil pointers and the
Encoder interface interact. The EncodeRLP method was previously called
even on nil values, which was supposed to give users a choice of how
their value would be handled when nil. It turns out this is a stupid
idea. If you create a network protocol containing an object defined in
another package, it's better to be able to say that the object should be
a list or string when nil in the definition of the protocol message
rather than defining the encoding of nil on the object itself.
As of this commit, the encoding rules for pointers now take precedence
over the Encoder interface rule. I think the "nil" tag will work fine
for most cases. For special kinds of objects which are a struct in Go
but strings in RLP, code using the object can specify the desired
encoding of nil using the "nilString" and "nilList" tags.
* rlp: propagate struct field type errors
If a struct contained fields of undecodable type, the encoder and
decoder would panic instead of returning an error. Fix this by
propagating type errors in makeStruct{Writer,Decoder} and add a test.
This change adds tests for the virtual clock and aligns the interface with the time package by renaming Cancel to Stop. It also removes the binary search from Stop because it complicates the code unnecessarily.
* txpool: smaller lock portion * core/tx_pool: fix data race
* les: detailed relative cost metrics * les: filter txpool relative request statistic * les: initialize price factors * les: increased connected bias to lower churn rate * les: fixed clientPool.setLimits * core: do not use mutex in GetAncestor * les: bump factor db version again * les: add metrics * les, light: minor fixes
deepCopy didn't copy pending storage updates, leading to the creation of blocks with invalid state root.
The original idea behind this change was to remove a use of the deprecated CancelRequest method. Simply removing it would've been an option, but I couldn't resist and did a bit of a refactoring instead. All remote sealing code was contained in a single giant function. Remote sealing is now extracted into its own object, remoteSealer.
This change: - removes the PostChainEvents method on core.BlockChain. - sorts 'removed log' events by block number. - fire the NewChainHead event if we inject a canonical block into the chain even if the entire insertion is not successful. - guarantees correct event ordering in all cases.
) * consensus/ethash, params: implement eip-2384: bump difficulty bomb * params: EIP 2384 compat checks * consensus, params: add Muir Glacier block number (mainnet,ropsten) + official name * core/forkid: forkid tests for muir glacier * params/config: address review concerns * params, core/forkid: review nitpicks * cmd/geth,eth,les: add override option for muir glacier * params: nit fix
elizabethengelman
force-pushed
the
rebase-statediffing-service-with-1.9.9
branch
2 times, most recently
from
86e833b to
e766f03
Compare
- Remove vendor dir - Removing unused geth cmd config - Apply govet updates
elizabethengelman
force-pushed
the
rebase-statediffing-service-with-1.9.9
branch
from
e766f03 to
e3bcdd6
Compare
Merged
rmulhol
approved these changes
Dec 23, 2019
There was a problem hiding this comment.
Not really making a serious attempt to go through the diff, but I'm good with getting everything onto one branch and eventually merging to master (so that hopefully the diff bw ethereum/go-ethereum:master and makerdao/go-ethereum:master is as clear/minimal as possible). Lmk if it'd be helpful to diver deeper into a specific part of the diff 👍
|
I'm good with not getting too deep of a review of the diffing stuffing at the moment, since that has been previously reviewed on the old fork. But once we get some PRs up for simplifying the changes, getting a thorough review would be great. |
elizabethengelman
added a commit
that referenced
this pull request
Dec 30, 2019
* port statediff from https://github.com/jpmorganchase/quorum/blob/9b7fd9af8082795eeeb6863d9746f12b82dd5078/statediff/statediff.go; minor fixes * integrating state diff extracting, building, and persisting into geth processes * work towards persisting created statediffs in ipfs; based off github.com/vulcanize/eth-block-extractor * Add a state diff service * Remove diff extractor from blockchain * Update imports * Move statediff on/off check to geth cmd config * Update starting state diff service * Add debugging logs for creating diff * Add statediff extractor and builder tests and small refactoring * Start to write statediff to a CSV * Restructure statediff directory * Pull CSV publishing methods into their own file * Reformatting due to go fmt * Add gomega to vendor dir * Remove testing focuses * Update statediff tests to use golang test pkg instead of ginkgo - builder_test - extractor_test - publisher_test * Use hexutil.Encode instead of deprecated common.ToHex * Remove OldValue from DiffBigInt and DiffUint64 fields * Update builder test * Remove old storage value from updated accounts * Remove old values from created/deleted accounts * Update publisher to account for only storing current account values * Update service loop and fetching previous block * Update testing - remove statediff ginkgo test suite file - move mocks to their own dir * Updates per go fmt * Updates to tests * Pass statediff mode and path in through cli * Return filename from publisher * Remove some duplication in builder * Remove code field from state diff output this is the contract byte code, and it can still be obtained by querying the db by the codeHash * Consolidate acct diff structs for updated & updated/deleted accts * Include block number in csv filename * Clean up error logging * Cleanup formatting, spelling, etc * Address PR comments * Add contract address and storage value to csv * Refactor accumulating account row in csv publisher * Add DiffStorage struct * Add storage key to csv * Address PR comments * Fix publisher to include rows for accounts that don't have store updates * Update builder test after merging in release/1.8 * Update test contract to include storage on contract intialization - so that we're able to test that storage diffing works for created and deleted accounts (not just updated accounts). * Factor out a common trie iterator method in builder
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I still need to figure out the OS failing build - the other two failing environments are failing on the more recent release as well, so I probably won't worry about them as much.
Tested that this works with our most recent execute release.