core/tracing: add system call callback when performing ProcessBeaconBlockRoot

[maoueh]

Added a start/end system where tracer can be notified that processing of some Ethereum system calls is starting processing and also notifies it when the processing has completed.

Doing a start/end for system call will enable tracers to "route" incoming next tracing events to go to a separate bucket than other EVM calls. Those not interested by this fact can simply avoid registering the hooks.

The EVM call is going to be traced normally afterward between the signals provided by those 2 new hooks but outside of a transaction context OnTxStart/End. That something implementors of live tracers will need to be aware of (since only "trx tracers" are not concerned by ProcessBeaconRoot).

I've looked into providing some kind of unit tests for that but it prove harder than anticipated. Since this has a minimal implementation surface, it think it's fine but I can take a second look if deemed required.

[s1na]

LGTM thanks!

[holiman]

What would a tracer do with this information?

EDIT: why not just use the fact that the tx sender is a magic system sender, and use that to distinguish?

[s1na]

EDIT: why not just use the fact that the tx sender is a magic system sender, and use that to distinguish?

The reason we added this is the tracer will be surprised to see a call frame enter outside of a tx. It's true that OnEnter we can check system address and figure out that we are in a system call, hence no OnTxStart was emitted. @maoueh what do you think?

[lightclient]

I'm a little skeptical of the need for this. @maoueh could you elaborate on your intended use case? And if you're using EVM tracing to build an external DB of data, how do you deal with withdrawals which are directly applied to the state?

I could imagine that other downstream chains using go-ethereum might appreciate these modifications the tracer API though if they have developed more complex system calls.

[maoueh]

@lightclient That is totally correct. Live tracer has an interest to whatever make state changes to the chain which is the case here with beacon root.

The EVM call that is being made is going to trace a StorageChange out when executing the contract because AFAIK, that's that the contract does.

We have a real use case today for this like you said which is a virtual archive node fed by state changes (storage,balance,nonce,etc.) generated by our Firehose tracer. We are able to then implement eth_call (and a few others) on top of that using distributed kvdb. You see how missing those thing can be problematic, I wouldn't be able to serve eth_call anymore that touchs this state because I would never have seen it.

Completely omitting those tracing is contrary to the live tracer spirit IMO.

It's true that I could infer from the block header and generate a StorageChange "manually" but having proper marker enable me to right now to support any future "system call" doing any kind of state changes.

About withdrawals, they generate a BalanceChange today with the now merged new tracing hooks, which also happens outside any transaction and "could" be seen as system level but here too, I wouldn't want to miss a balance change just because it's "system" based.

[maoueh]

Now, how this can be done, there is different possibilities of course. I think the marker OnSystemCallStart/End is the easiest. You can them simply prepare yourself to treat upcoming events as part of a system call and you get the full model.

Quite open to check other possibilities, a specific OnProcessBeaconRootStart/End would be quite fine too and more specific. With the way hooks are made, shouldn't be a problem to add more stuff later.

Just a OnProcessBeaconRoot would prove more brittle for me as when receiving the signal, I would "trap" the next OnEnter/OnExit sequence. Less ideal but I've did worst :)

[maoueh]

@lightclient Just saw your other PR, going to take a look.

[maoueh]

For extra clarity, actually right now I get all that data I need from the live tracer even without this PR because the EVM and StateDB are still correctly traced so storage changes are properly sent. I'm just unable to group them into a "special" bucket "system_call" which is more contextual.

[lightclient]

Thanks @maoueh, I think I understand now. This PR seems reasonable.

[maoueh]

why not just use the fact that the tx sender is a magic system sender, and use that to distinguish?

I would be fine with this too, but it seems more and more "system call" are considered from that I saw in @lightclient PR. I think it would make sense to have something that works well with all those. Of course not alway easy to design future proof API from the start.

It would be fine waiting a bit too to ensure we correctly cover other currently planned system calls.

[maoueh]

Closed by mistake

[holiman]

why not just use the fact that the tx sender is a magic system sender, and use that to distinguish?

I would be fine with this too, but it seems more and more "system call" are considered from that I saw in @lightclient PR. I think it would make sense to have something that works well with all those. Of course not alway easy to design future proof API from the start.

It would work for all of them -- they all use the same magic system sender, don't they?

[holiman]

Yeah ok

[holiman]

core/tracing/hooks.go:184: File is not `goimports`-ed (goimports)
  OnClose          CloseHook
util.go:47: exit status 1

[maoueh]

core/tracing/hooks.go:184: File is not goimports-ed (goimports)

Hmm, I'll check that

[maoueh]

@holiman Can you make the CI re-run? Locally I cannot see any lints error using go run build/ci.go lint, I've rebased the work now to check if it's gonna help.

[fjl]

The CI is ok now. Please add the new hooks to the changelog in core/tracing/changelog.md

[s1na]

Updated the changelog here s1na@827f13c but can't push it to the PR directly as the organization permissions don't allow it.

[maoueh]

@fjl @s1na Commit added