eth: add debug_accountRange #17438

jwasinger · 2018-08-18T02:14:18Z

WIP Adds a debug API method AccountRangeAt which returns all accounts in the state for a given block and transaction index.

jwasinger · 2018-08-18T04:53:21Z

eth/api.go

+}
+
+//block hash or number, tx index, start address hash, max results
+func (api *PrivateDebugAPI) AccountRangeAt(ctx context.Context, blockNr uint64, txIndex int, startAddr *common.Hash, maxResults int) (AccountRangeAtResult, error) {


Is there a way to overload this so that I can have the rpc method accept a string for block number (e.g. "latest")?

fjl · 2018-08-18T17:09:21Z

I have not looked at the code, but we should think about the naming of the method. Existing *At methods access properties of an account at a specific address. Maybe it should just be debug_accountRange.

jwasinger · 2018-08-18T17:17:50Z

CC @cdetrio @gballet @winsvega

axic · 2018-08-18T17:19:58Z

Existing *At methods access properties of an account at a specific address.

Hm, since it has a starting address, would something like accountRangeBetween or accountRangeFrom be better?

jwasinger · 2018-08-18T17:21:53Z

For cpp-eth the method is called debug_accountRangeAt. I think we can look at changing the naming there in order to get this method included in Geth. We are using this method for ewasm testnet integration with https://github.com/gobitfly/etherchain-light .

jwasinger · 2018-08-18T17:29:21Z

core/state/statedb.go

@@ -131,6 +131,10 @@ func (self *StateDB) Reset(root common.Hash) error {
 	return nil
 }

+func (self *StateDB) GetTrie() Trie {


I added this because I couldn't see any existing way to get access to the trie from the statedb

You can do statedb.Database().OpenTrie()

jwasinger · 2018-08-18T17:32:59Z

This also needs unit tests.

axic · 2018-08-18T17:33:58Z

eth/api.go

+}
+
+//block hash or number, tx index, start address hash, max results
+func (api *PrivateDebugAPI) AccountRangeAt(ctx context.Context, blockNr uint64, txIndex int, startAddr *common.Hash, maxResults int) (AccountRangeAtResult, error) {


Are you sure startAddr should not be of common.Address type?

I think you're right.

winsvega · 2018-08-18T17:48:14Z

If to remove At.
Then what about accountStorageAt?

jwasinger · 2018-08-18T18:33:48Z

Existing *At methods access properties of an account at a specific address.

Looks like it wouldn't have be renamed.

jwasinger · 2018-08-18T19:43:54Z

Okay, have made suggested changes from @axic and @fjl . Another thing that @cdetrio pointed out was that I needed to be looking up the account key in preimage database to get the account address instead of keccak256(account address).

jwasinger · 2018-08-18T19:46:14Z

eth/api.go

+	}
+
+	blockHash := api.eth.blockchain.GetBlockByNumber(blockNr).Hash()
+	_, _, statedb, err := api.computeTxEnv(blockHash, txIndex, 0)


I'm getting an error from this line:

> curl -X POST -H 'Content-Type: application/json' --data '{"jsonrpc":"2.0","method":"debug_accountRange","params":[0, 0, "0x0000000000000000000000000000000000000000", 20],"id":1}' localhost:8545 {"jsonrpc":"2.0","id":1,"error":{"code":-32000,"message":"parent 0000000000000000000000000000000000000000000000000000000000000000 not found"}}

Not sure what I'm doing wrong here.

I had a very quick look, and it seems that the message comes from here:

block := api.eth.blockchain.GetBlockByHash(blockHash) if block == nil { return nil, vm.Context{}, nil, fmt.Errorf("block %x not found", blockHash) } parent := api.eth.blockchain.GetBlock(block.ParentHash(), block.NumberU64()-1) if parent == nil { return nil, vm.Context{}, nil, fmt.Errorf("parent %x not found", block.ParentHash()) }

it seems normal that block 0's parent will not be found. The GetBlock function uses both the hash and the block number to find the parent, and the block number will be invalid.

holiman · 2018-08-19T06:00:44Z

On mobile here, so apologies for brevity... We usually use a blocknumOrHash type for specifying blocks, which also accept e.g the string 'latest'. Also, any use of this on mainchain will go OOM pretty quickly, unless a subscription is used instead of a full json dump. Perhaps a default limit should be used; so one doesn't accidentally OOM a node just by checking out "oh I wonder how this method works..."

axic · 2018-08-19T17:04:13Z

Perhaps a default limit should be used; so one doesn't accidentally OOM a node just by checking out "oh I wonder how this method works..."

Sounds good. Also the debug namespace must be turned on explicitly, correct? So this will not pose a main risk to a regular client.

cdetrio · 2018-08-19T19:07:01Z

I have not looked at the code, but we should think about the naming of the method. Existing *At methods access properties of an account at a specific address. Maybe it should just be debug_accountRange

I thought the At in debug_storageRangeAt referred to the starting key in the range. But I'm fine with just debug_accountRange.

jwasinger · 2018-08-20T16:57:18Z

blocknumOrHash

@holiman I looked through the codebase but couldn't find this type. Tried variations of capitalization as well.

holiman · 2018-08-20T17:34:10Z

See

go-ethereum/rpc/types.go

Line 133 in 5d30be4

func (bn *BlockNumber) UnmarshalJSON(data []byte) error {

Usage:

go-ethereum/eth/api_backend.go

Line 60 in 21c059b

    
           func (b *EthAPIBackend) HeaderByNumber(ctx context.Context, blockNr rpc.BlockNumber) (*types.Header, error) {

blockNr rpc.BlockNumber

eth/api.go

GitCop · 2018-08-20T19:23:45Z

Thank you for your contribution! Your commits seem to not adhere to the repository coding standards

Commit: 4d090fc84040a5a3b15dffcfa1d63fa318524d35
Commits must be prefixed with the package(s) they modify
Commit: a6ca2cb1e1664a0c7a43fb6578870f6bc2092063
Commits must be prefixed with the package(s) they modify
Commit: 6bfaed3abacaa3591e053ba2c82f31711d8af0de
Commits must be prefixed with the package(s) they modify
Commit: 46cf98d32d5b196a8dfc3a77f37ff06c0ee81eae
Commits must be prefixed with the package(s) they modify

Please check the contribution guidelines for more details.

This message was auto-generated by https://gitcop.com

GitCop · 2018-08-20T19:32:53Z

Thank you for your contribution! Your commits seem to not adhere to the repository coding standards

Commit: 4d090fc84040a5a3b15dffcfa1d63fa318524d35
Commits must be prefixed with the package(s) they modify
Commit: a6ca2cb1e1664a0c7a43fb6578870f6bc2092063
Commits must be prefixed with the package(s) they modify
Commit: 6bfaed3abacaa3591e053ba2c82f31711d8af0de
Commits must be prefixed with the package(s) they modify
Commit: 46cf98d32d5b196a8dfc3a77f37ff06c0ee81eae
Commits must be prefixed with the package(s) they modify

Please check the contribution guidelines for more details.

This message was auto-generated by https://gitcop.com

axic · 2018-08-21T03:19:12Z

Updated this to be compatible with aleth/testeth.

gballet

I have made the requested changes, added some unit tests and made sure that the call could be invoked with a nil pointer made sure that it worked with empty results.

fjl · 2019-07-13T13:46:14Z

@gballet not sure why you renamed this to "rpc: ...". The main change is in package "eth", the commit message prefix "eth" is appropriate.

gballet · 2019-07-14T07:24:47Z

the feature was an RPC call, seems logical enough to me.

jwasinger requested review from holiman and karalabe as code owners August 18, 2018 02:14

jwasinger commented Aug 18, 2018

View reviewed changes

axic reviewed Aug 18, 2018

View reviewed changes

jwasinger force-pushed the accountrangeat branch from 39c72f4 to 4d090fc Compare August 18, 2018 19:37

jwasinger commented Aug 18, 2018

View reviewed changes

jwasinger force-pushed the accountrangeat branch from acd69b8 to 6bfaed3 Compare August 20, 2018 17:30

holiman reviewed Aug 20, 2018

View reviewed changes

eth/api.go Outdated Show resolved Hide resolved

jwasinger commented Aug 20, 2018

View reviewed changes

eth/api.go Outdated Show resolved Hide resolved

jwasinger force-pushed the accountrangeat branch from 2f76eb2 to d23b14e Compare August 20, 2018 23:20

axic force-pushed the accountrangeat branch 2 times, most recently from e4446a1 to b7040ae Compare August 21, 2018 03:18

axic force-pushed the accountrangeat branch from b7040ae to cbdf3b0 Compare August 21, 2018 04:51

jwasinger and others added 17 commits July 10, 2019 12:20

eth: address some concerns (squash this)

a373953

eth: squash this

23f47b7

more

8dbe2bb

more

d680d14

more

33c0e7e

eth: format (squash this)

c74bd0e

eth: clean up accountRange, add unit tests

045772e

eth: try generating random addresses for accountRange test vectors

49f6bc3

eth: fix accountrange at, unit test passing

adcecf3

eth: fix accountRangeAt after rebase

407ff0f

eth: simplify accountRangeAt

a32c95b

Add unit tests covering empty results

70de8fb

rpc: Rename AccountRangeAt to AccountRange

1783c3b

eth: Include Felix' feedback

795cfed

rpc: return a Next hash even without a preimage

99e7ebe

rpc: fix linter warning

8efbf21

rpc: allow nil as a start hash param

0e5dcef

gballet force-pushed the accountrangeat branch from e6b4473 to 0e5dcef Compare July 10, 2019 11:52

gballet approved these changes Jul 10, 2019

View reviewed changes

gballet changed the title ~~eth: Add AccountRangeAt to debug API~~ rpc: Add AccountRange to debug API Jul 10, 2019

rpc: add a unit test for pagination overlap

795798d

gballet force-pushed the accountrangeat branch from 9ccd9d9 to 795798d Compare July 10, 2019 13:07

fjl changed the title ~~rpc: Add AccountRange to debug API~~ et Jul 13, 2019

fjl changed the title et eth: add debug_accountRange Jul 13, 2019

fjl approved these changes Jul 13, 2019

View reviewed changes

fjl merged commit 6bd896a into ethereum:master Jul 13, 2019

jwasinger deleted the accountrangeat branch July 14, 2019 21:08

tjayrush mentioned this pull request Oct 23, 2020

rpcdaemon debug_accountRange - method handler crashed - slice bounds out of range [:20] with capacity 8 erigontech/erigon#1275

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

eth: add debug_accountRange #17438

eth: add debug_accountRange #17438

jwasinger commented Aug 18, 2018 •

edited

Loading

jwasinger Aug 18, 2018 •

edited

Loading

fjl commented Aug 18, 2018

jwasinger commented Aug 18, 2018 •

edited

Loading

axic commented Aug 18, 2018 •

edited

Loading

jwasinger commented Aug 18, 2018 •

edited

Loading

jwasinger Aug 18, 2018

karalabe Aug 24, 2018

jwasinger commented Aug 18, 2018

axic Aug 18, 2018

jwasinger Aug 18, 2018

winsvega commented Aug 18, 2018

jwasinger commented Aug 18, 2018

jwasinger commented Aug 18, 2018

jwasinger Aug 18, 2018 •

edited

Loading

gballet Aug 20, 2018

holiman commented Aug 19, 2018

axic commented Aug 19, 2018

cdetrio commented Aug 19, 2018

jwasinger commented Aug 20, 2018

holiman commented Aug 20, 2018

GitCop commented Aug 20, 2018

GitCop commented Aug 20, 2018

axic commented Aug 21, 2018

gballet left a comment

fjl commented Jul 13, 2019

gballet commented Jul 14, 2019

eth: add debug_accountRange #17438

eth: add debug_accountRange #17438

Conversation

jwasinger commented Aug 18, 2018 • edited Loading

jwasinger Aug 18, 2018 • edited Loading

Choose a reason for hiding this comment

fjl commented Aug 18, 2018

jwasinger commented Aug 18, 2018 • edited Loading

axic commented Aug 18, 2018 • edited Loading

jwasinger commented Aug 18, 2018 • edited Loading

jwasinger Aug 18, 2018

Choose a reason for hiding this comment

karalabe Aug 24, 2018

Choose a reason for hiding this comment

jwasinger commented Aug 18, 2018

axic Aug 18, 2018

Choose a reason for hiding this comment

jwasinger Aug 18, 2018

Choose a reason for hiding this comment

winsvega commented Aug 18, 2018

jwasinger commented Aug 18, 2018

jwasinger commented Aug 18, 2018

jwasinger Aug 18, 2018 • edited Loading

Choose a reason for hiding this comment

gballet Aug 20, 2018

Choose a reason for hiding this comment

holiman commented Aug 19, 2018

axic commented Aug 19, 2018

cdetrio commented Aug 19, 2018

jwasinger commented Aug 20, 2018

holiman commented Aug 20, 2018

GitCop commented Aug 20, 2018

GitCop commented Aug 20, 2018

axic commented Aug 21, 2018

gballet left a comment

Choose a reason for hiding this comment

fjl commented Jul 13, 2019

gballet commented Jul 14, 2019

jwasinger commented Aug 18, 2018 •

edited

Loading

jwasinger Aug 18, 2018 •

edited

Loading

jwasinger commented Aug 18, 2018 •

edited

Loading

axic commented Aug 18, 2018 •

edited

Loading

jwasinger commented Aug 18, 2018 •

edited

Loading

jwasinger Aug 18, 2018 •

edited

Loading