Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eth/tracers: implement trace live filter #30255

Draft
wants to merge 50 commits into
base: master
Choose a base branch
from

Conversation

jsvisa
Copy link
Contributor

@jsvisa jsvisa commented Aug 1, 2024

Hi @s1na,

This is a poc version of the trace_xxx RPC. I primarily use the freezer database to store the block traces and register the RPC in stack with a callback (which may be refactored later). In this branch, I attempt to implement a simple version of the trace_filter, trace_block, and trace_transaction RPC methods. I then run these changes locally to verify the correctness of the data produced.

Here are some areas that need further refactoring:

  1. Dependency Injection: We need to pass an eth.Backend or a similar object to the filter_tracer to fetch the chain’s inner services. For example, trace_transaction requires the transaction-to-block number mapping.
  2. Tracer Compatibility: We need to adjust some aspects of the prestate and call tracers to be compatible with both live and debug traces. For instance, in live tracing, the block context is passed via the OnBlockStart hook, whereas in debug tracing, it is provided during the initialization of tracing.
  3. RPC Design: I'm intending to maintain compatibility with Parity’s RPC. For example:
{
  "jsonrpc": "2.0",
  "method": "trace_block",
  "id": 67,
  "params": [
    "0x5990",
    {"tracer": "callTracer"}  // Added in our implementation
  ]
}

Should the default tracer be parity? In Parity’s case, the response should also be the same as Parity’s(need more redesign). For other tracers (e.g., callTracer, prestateTracer), the response format should be consistent with the debug_traceXXX methods.

Starting geth with filter live tracer:

geth
... # other arguments
--http.api=engine,web3,eth,debug,trace # append trace namespace
--vmtrace=filter # enable the filter live tracer
--vmtrace.jsonconfig="{\"path\": \"/data/trace-filter\", \"config\": {\"callTracer\": {}, \"flatCallTracer\": {\"includePrecompiles\": true, \"convertParityErrors\": true}}}"

Looking forward to your feedback and suggestions.

eth/tracers/live/filter.go Outdated Show resolved Hide resolved
…stead

Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
This reverts commit c378326.

Signed-off-by: jsvisa <delweng@gmail.com>
This reverts commit 3d5a8b3.

Signed-off-by: jsvisa <delweng@gmail.com>
This reverts commit 0bdbcba.

Signed-off-by: jsvisa <delweng@gmail.com>
This reverts commit 54f4a12.

Signed-off-by: jsvisa <delweng@gmail.com>
This reverts commit d33b24c.

Signed-off-by: jsvisa <delweng@gmail.com>
This reverts commit 682cc5a.

Signed-off-by: jsvisa <delweng@gmail.com>
This reverts commit 9e1d9ec.

Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>

fix(eth/tracers): clear callstack in OnTxStart

Signed-off-by: jsvisa <delweng@gmail.com>

eth/trace: add filter storage

Signed-off-by: jsvisa <delweng@gmail.com>

eth/tracers: register trace API

Signed-off-by: jsvisa <delweng@gmail.com>

eth/tracers: add filter_api

Signed-off-by: jsvisa <delweng@gmail.com>

eth/tracers: add trace_block rpc

Signed-off-by: jsvisa <delweng@gmail.com>

eth/tracers/native: MuxTracer as public

Signed-off-by: jsvisa <delweng@gmail.com>

eth/tracers: put/get trace by tracer name

Signed-off-by: jsvisa <delweng@gmail.com>

fix: the opposit blknum cmp f.latest

Signed-off-by: jsvisa <delweng@gmail.com>

feat: no need to handle reorg if blknum is the same

Signed-off-by: jsvisa <delweng@gmail.com>

fix: restore f.latest if offset

Signed-off-by: jsvisa <delweng@gmail.com>

feat: always update f.latest

Signed-off-by: jsvisa <delweng@gmail.com>

fix blknum=latest

Signed-off-by: jsvisa <delweng@gmail.com>

fix: latest use atomic.Uint64

Signed-off-by: jsvisa <delweng@gmail.com>

fix: f.latest use atomic

Signed-off-by: jsvisa <delweng@gmail.com>

fix: ignore if block < latest

Signed-off-by: jsvisa <delweng@gmail.com>

ignore all later ops is ignored

Signed-off-by: jsvisa <delweng@gmail.com>

ignored -> skipped

Signed-off-by: jsvisa <delweng@gmail.com>

db -> frdb

Signed-off-by: jsvisa <delweng@gmail.com>

add kvdb

Signed-off-by: jsvisa <delweng@gmail.com>

no skipper

Signed-off-by: jsvisa <delweng@gmail.com>

wip: save into kvdb

Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
jsvisa and others added 16 commits September 27, 2024 00:51
This reverts commit f646b87.

Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
return nil, nil
}

found, tx, blockHash, blockNumber, index, err := n.backend.GetTransaction(ctx, common.BytesToHash(txHash))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Myeah I thought we would just return the txhash? Is any other client returning full tx objects?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm following the RPC in https://github.com/manifoldfinance/rpc-eip-drafts/blob/master/eth_getTransactionBySenderAndNonce/README.md, so in reth, I return the full tx, ref paradigmxyz/reth#10540.

I think I might need to reconsider whether to return the txhash or full tx, I'll take a look at the behavior of other clients or providers again. Indeed returning the txhash is more simple and then sdk's side can fetching the tx body or receipt by calling the eth_getTransaction or eth_getTransactionReceipt.

frpath = path.Join(config.Path, "frdb")
)

kvdb, err := rawdb.NewPebbleDBDatabase(kvpath, 128, 1024, "trace", false, false)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw now that we are passing in the *node.Node handle we can use that to access databases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but I don't want to put the temporary data into the chaindata, as the data we used here is temporary, and acts in the order: write-once, read-once, deleted(level merged), so the user can set a separate, and cheaper SSD for such temporary data needs.

Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
@jsvisa
Copy link
Contributor Author

jsvisa commented Oct 17, 2024

@s1na maybe we can support this tracer after #30611

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants