Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Haoyu/scroll-v5.7.11_broadcast #1074

Closed
wants to merge 485 commits into from
Closed

Haoyu/scroll-v5.7.11_broadcast #1074

wants to merge 485 commits into from

Conversation

0xmountaintop
Copy link
Member

1. Purpose or design rationale of this PR

...

2. PR title

Your PR title must follow conventional commits (as we are doing squash merge for each PR), so it must start with one of the following types:

  • build: Changes that affect the build system or external dependencies (example scopes: yarn, eslint, typescript)
  • ci: Changes to our CI configuration files and scripts (example scopes: vercel, github, cypress)
  • docs: Documentation-only changes
  • feat: A new feature
  • fix: A bug fix
  • perf: A code change that improves performance
  • refactor: A code change that doesn't fix a bug, or add a feature, or improves performance
  • style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
  • test: Adding missing tests or correcting existing tests

3. Deployment tag versioning

Has the version in params/version.go been updated?

  • This PR doesn't involve a new deployment, git tag, docker image tag, and it doesn't affect traces
  • Yes

4. Breaking change label

Does this PR have the breaking-change label?

  • This PR is not a breaking change
  • Yes

iczc and others added 30 commits August 24, 2023 10:30
@iczc @0xmountaintop
refactor: simplify ccc revert to snapshot (#480)
d02a410 
* simplify ccc revert to snapshot

* Update version.go

---------

Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
@Thegaram
feat: use --gcmode=archive and --cache.noprefetch=true by default (#482)
e93930d 
* feat: use --gcmode=archive and --cache.noprefetch=true by default

* refuse to start with invalid config

* lint

* lint
@mask-pp @icemelon
feat(sdk): support compressed response (#469)
eb79758 
* enable use compression algorithm

* fix ci

* Just enable decode compressed content at ethclient

* fix comments

---------

Co-authored-by: Haichen Shen <shenhaichen@gmail.com>
@Thegaram @colinlyguo
fix: disable pruning and prefetch if not flags are provided (#483)
b0657a7 
Co-authored-by: colin <102356659+colinlyguo@users.noreply.github.com>
@silathdiir
fix: update libzkp to use scroll-prover v0.7.5 (#484)
637cb37 
* Update libzkp to use scroll-prover `v0.7.5`.

* Update version.
@Thegaram @mask-pp
@silathdiir @0xmountaintop @iczc @icemelon @colinlyguo
feat: store skipped transactions in local db (#467)
7de261b 
* feat: store skipped txs in local db

* bump version

* bump version

* fix test

* include L2 txs in skipped index

* goimports

* rename more

* bump version

* fix missed renames

* fix the bug when calculating l2TxCount. (#479)

* fix bug when calculate l2 tx count

* Update version

* fix: exclude L1 message from block payload size validation (#476)

* fix: exclude L1 message from block payload size validation

* fix the bug when calculating l2TxCount. (#479)

* fix bug when calculate l2 tx count

* Update version

* bump version

---------

Co-authored-by: maskpp <maskpp266@gmail.com>

* fix: update row estimation with scroll-prover `v0.7.2` (#475)

* Fix row estimation.

* Update libzkp.

* more

* prepare

* finish

* upgrade

* bump version

* fix tests

* Update to scroll-prover `v0.7.2`.

* fix tests

* Update miner/worker.go

Co-authored-by: Péter Garamvölgyi <peter@scroll.io>

* Update miner/worker.go

Co-authored-by: Péter Garamvölgyi <peter@scroll.io>

* Reset ccc when skips first tx.

* do not unnecessarily skip L1 message

* fix ccc reset and improve code readability

* seal block on circuitcapacitychecker.ErrUnknown

---------

Co-authored-by: HAOYUatHZ <haoyu@protonmail.com>
Co-authored-by: Péter Garamvölgyi <peter@scroll.io>

* refactor: simplify ccc revert to snapshot (#480)

* simplify ccc revert to snapshot

* Update version.go

---------

Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>

* feat: use --gcmode=archive and --cache.noprefetch=true by default (#482)

* feat: use --gcmode=archive and --cache.noprefetch=true by default

* refuse to start with invalid config

* lint

* lint

* feat(sdk): support compressed response (#469)

* enable use compression algorithm

* fix ci

* Just enable decode compressed content at ethclient

* fix comments

---------

Co-authored-by: Haichen Shen <shenhaichen@gmail.com>

* fix: disable pruning and prefetch if not flags are provided (#483)

Co-authored-by: colin <102356659+colinlyguo@users.noreply.github.com>

* fix: update libzkp to use scroll-prover `v0.7.5` (#484)

* Update libzkp to use scroll-prover `v0.7.5`.

* Update version.

* address comments

* bump version

* nit

---------

Co-authored-by: maskpp <maskpp266@gmail.com>
Co-authored-by: Steven <asongala@163.com>
Co-authored-by: HAOYUatHZ <haoyu@protonmail.com>
Co-authored-by: iczc <iczcalan@gmail.com>
Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
Co-authored-by: Haichen Shen <shenhaichen@gmail.com>
Co-authored-by: colin <102356659+colinlyguo@users.noreply.github.com>
@0xmountaintop @colinlyguo @Thegaram
feat(miner): consider l1msg in txCount (#486)
d650299 
* feat(miner): consider l1msg in txCount

* refactor

* tweak

* fix comments

* fix unit test

* set chainConfig.Scroll.MaxTxPerBlock in l1 message test

* add a skipped tx in TestAcceptableTxlimit

* add l1 skipped msg in test

* fix unit test

* fix unit test

* Update miner/worker.go

Co-authored-by: Péter Garamvölgyi <peter@scroll.io>

* revert some test changes

* tweak

* bump version

---------

Co-authored-by: colinlyguo <colinlyguo@scroll.io>
Co-authored-by: colin <102356659+colinlyguo@users.noreply.github.com>
Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
@0xmountaintop
docs(ccc): add more logs for skipping msg (#490)
f883dcd 
* docs(ccc): add more logs for skipping msg

* bump version

* format codes

* improve
@0xmountaintop @Thegaram
fix(miner): fix repeatedly processing dropped txs in a new block (#491)
ea3a3c9 
* squash

* improve comments

* Remove tx (#493)

* force drop

* minor

* store skipped L2 tx for `circuitcapacitychecker.ErrUnknown`

* Update miner/worker.go

Co-authored-by: Péter Garamvölgyi <peter@scroll.io>

* add lock

* use mu

---------

Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
@lispc @0xmountaintop
chore(ccc): v0.7.6: fix: if rws reaches limit, stop early (#495)
63cbb77 
* chore(ccc): v0.7.6: fix: if rws reaches limit, stop early

* Update version.go

---------

Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
@lispc @0xmountaintop
fix(modexp): disallow len of 0x80000000000000000000000000000000000000…
ef328db 
…00000… (#496)

* modexp: disallow len of 0x8000000000000000000000000000000000000000000000000000000000000000

* minor

---------

Co-authored-by: HAOYUatHZ <haoyu@protonmail.com>
@Lawliet-Chan
feat(metrics): add skipped txs metrics for miner (#494)
24d8567
@lispc @0xmountaintop @Thegaram
fix(trace): more consistent invalid opcode trace (#487)
d45613f 
* fix: more consistent invalid opcode trace

* Update version.go

---------

Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
@lispc @holiman @Thegaram
fix(trace): fix memory trace size (#497)
c60fa3a 
* core/vm: for tracing, do not report post-op memory

* fmt

* bump version

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
@lispc @Thegaram @0xmountaintop
fix(trace): record account proof for ErrContractAddressCollision (#488)
2d9d48d 
* fix(trace): record account proof for ErrContractAddressCollision

* bump version

* fix memory reading

* bump version

---------

Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
Co-authored-by: HAOYUatHZ <37070449+HAOYUatHZ@users.noreply.github.com>
Co-authored-by: HAOYUatHZ <haoyu@protonmail.com>
@0xmountaintop
feat(api/rawdb): allow storing traces for skipped txs (#500)
0b6c3be
@Lawliet-Chan
refactor(ccc): throw error instead of panic for recognized errors (#499)
293deff
@silathdiir
fix: libzkp v0.8.1 (#502)
fde2c3b
@silathdiir
fix(libzkp): upgrade to v0.8.2 (#504)
c75ebdb
@0xmountaintop
fix(rawdb, api): fix rawdb.writeSkippedTransaction & `api.GetSkippe…
dade96d 
…dTransaction` (#503)

* fix(rawsdb): fix `writeSkippedTransaction`

* Update version.go

* minor

* clean up
@silathdiir @lispc
@Thegaram @0xmountaintop
fix (libzkp): check tx num when CCC ErrUnknown error occurred (#505)
7d79f27 
* Check tx num for CCC ErrUnknown error.

* Update worker.go

* Small fix.

* Update

* Fix log.

* Update version.

* Update miner/worker.go

Co-authored-by: Péter Garamvölgyi <peter@scroll.io>

* Update according to review.

* Fix to return 0 for uint64.

* Add `tcount` to mock CCC, and function `SetTxNum`.

* refactor

* fix

* add comments

---------

Co-authored-by: Zhang Zhuo <mycinbrin@gmail.com>
Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
Co-authored-by: HAOYUatHZ <haoyu@protonmail.com>
@Thegaram
fix(ccc): fix CCC build errors (#506)
1d824f4
@Thegaram
fix(trace): use correct roots in trace of block with one transaction (#…
9c1433b 
…508)

fix: use correct roots in trace of block with one transaction
@iczc @fjl
p2p: limit ping requests from a single peer (#510)
734021f 
* p2p: move ping handling into pingLoop goroutine (ethereum#27887)

Moving the response sending there allows tracking all peer goroutines
in the peer WaitGroup.

* bump version

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
@mask-pp
fix(sdk): accept blocks where coinbase is null (e.g. pending block) (#…
417dd02 
…509)

* fix error(when get pengding block) coinbase address is null

* fix comments
@Thegaram @mask-pp
fix(worker): check gas limit before running ccc (#511)
530830a 
Co-authored-by: maskpp <maskpp266@gmail.com>
@silathdiir
fix(libzkp): upgrade to use prover v0.9.0 (#513)
2b22bf9
@silathdiir
fix(libzkp): upgrade to use prover v0.9.1 (#517)
5b787be 
Upgrade libzkp to use prover `v0.9.1`.
@Thegaram
refactor(worker): order L1 messages by queue index (#512)
93a9497 
* refactor(worker): order L1 messages by queue index

* bump version

* bump version
@silathdiir
fix(libzkp): upgrade libzkp to v0.9.2 (fix ccc panic CodeNotFound) (#…
fa0be69 
…520)

Upgrade libzkp to `v0.9.2` (fix ccc bug).
0xmountaintop and others added 29 commits August 27, 2024 23:03
@0xmountaintop @Thegaram
feat: add more metrics to investigate tx broadcast issue (#1007)
118a80b 
* update eth/fetcher/tx_fetcher.go

* update eth/handler.go

* update eth/handler.go

* update eth/protocols/eth/peer.go

* update eth/protocols/eth/broadcast.go

* update eth/protocols/eth/handlers.go

* chore: auto version bump [bot]

* fix

---------

Co-authored-by: HAOYUatHZ <HAOYUatHZ@users.noreply.github.com>
Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
@0xmountaintop
fix(metrics): fix tx broadcast investigation metrics (#1010)
89e9a21 
* update eth/protocols/eth/broadcast.go

* update eth/protocols/eth/handlers.go

* chore: auto version bump [bot]

---------

Co-authored-by: HAOYUatHZ <HAOYUatHZ@users.noreply.github.com>
@0xmountaintop
fix(metrics): fix tx broadcast investigation metrics again (#1012)
2865e13 
* update eth/protocols/eth/broadcast.go

* chore: auto version bump [bot]

* update eth/protocols/eth/handlers.go

---------

Co-authored-by: HAOYUatHZ <HAOYUatHZ@users.noreply.github.com>
@omerfirmak @colinlyguo
@lispc @zimpha
feat: implement and integrate counter-based CCC (#982)
e9259b7 
Co-authored-by: colin <102356659+colinlyguo@users.noreply.github.com>
Co-authored-by: Zhang Zhuo <mycinbrin@gmail.com>
Co-authored-by: Xi Lin <zimpha@gmail.com>
@0xmountaintop
feat(txpool): add more logs to runReorg (#1019)
8f8ef30 
* feat(txpool): add more logs to `runReorg`

* chore: auto version bump [bot]

* improve

* chore: auto version bump [bot]

---------

Co-authored-by: HAOYUatHZ <HAOYUatHZ@users.noreply.github.com>
@omerfirmak
fix: disable headCCCCheck for follower nodes (#1021)
0573f6e
@omerfirmak
feat: double check ancestor RowConsumption before committing (#1022)
e0203bf
@omerfirmak
feat: add miner idle metric (#1023)
bdb1ffb
@omerfirmak
fix: nil-check curRc (#1024)
bf5cca7 
* fix: nil-check curRc

* chore: auto version bump [bot]

---------

Co-authored-by: omerfirmak <omerfirmak@users.noreply.github.com>
@omerfirmak
fix: skip ccc.ErrUnknown txns immediately (#1027)
a4b738f
@lispc @omerfirmak
@Thegaram @colinlyguo
feat(rpc): add getTxByTxTrace api, used for ccc testing & debugging (#…
665f6c3 
…1026)

* ++

* upgrade go lint version

* lint

* remove a useless "`"

* Update l2geth_ci.yml

---------

Co-authored-by: Ömer Faruk Irmak <omerfirmak@gmail.com>
Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
Co-authored-by: colin <102356659+colinlyguo@users.noreply.github.com>
@omerfirmak
fix: increase keccak usage safety buffer (#1028)
590e104
@omerfirmak
fix: ignore RC check if force committing (#1029)
e426d4b 
* fix: ignore RC check if force committing

* rename force to reorging

* use prev goimports version
@omerfirmak @Thegaram
fix: properly handle wrapped retryable errors (#1030)
51082b2 
* fix: properly handle wrapped retryable errors

* chore: auto version bump [bot]

---------

Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
Co-authored-by: Thegaram <Thegaram@users.noreply.github.com>
@omerfirmak
fix: reuse timestamp for blocks failing CCC (#1031)
6cdcff6 
we assume that same height wont trigger multiple reorgs to be able
to put an upper bound on the reorg depth. We rely on the fact that
AsyncChecker executes transactions one-by-one and tells worker the
safe set of transactions to include in the replacement block that
wont trigger another error on the same block. If worker changes the
timestamp and that causes significant changes to the execution flow
of included transactions; we might have a height where multiple
reorgs happen.
@omerfirmak
refactor: eliminate double re-execution in AsyncChecker (#1036)
7f79c7f
@omerfirmak
feat: allow removing txns from pool via CLI/RPC (#1041)
521a074
@colinlyguo
feat(scroll-worker): add nil consumption block metrics (#1033)
1d02d04 
* feat(scroll-worker): add nil consumption block metrics

* increase the version
@omerfirmak @Thegaram
feat: allow triggering one-off CCC on a block (#1043)
89639dd 
Co-authored-by: Péter Garamvölgyi <peter@scroll.io>
@colinlyguo
feat(tx-pool): fast reject transactions that cannot fit into a block (#…
86965f8 
…1042)

* feat(tx pool): fast reject transactions that cannot fit into a block

* add a TestValidateTxBlockSize unit test

* chore: auto version bump [bot]

---------

Co-authored-by: colinlyguo <colinlyguo@users.noreply.github.com>
@colinlyguo
fix: remove a duplicated cmd in Dockerfile.mockccc (#1048)
f044582 
* fix: remove a duplicated cmd in Dockerfile.mockccc

* increase version
@omerfirmak
fix: make reorg mode explicit (#1049)
d599666
@omerfirmak
fix: avoid committing empty blocks after the deadline (#1051)
c0094ee
@omerfirmak
fix: initialize pending block with an empty block (#1052)
7ff93ca
@colinlyguo
feat: allow changing L1 synced height via admin RPC/CLI (#1044)
cf1ca84 
* feat: allow changing L1 synced height via RPC/CLI

* chore: auto version bump [bot]

* fix typos

* add height validity check

* change implementation

* moved hotfix apis from ScrollAPI to PrivateAdminAPI

* change namespace from scroll to admin

* fix bugs

* add locks

* add a lock to protect latestProcessedBlock and db state

* revert some changes

* remove lock and use atomic in latestProcessedBlock

* fix CI

* tweak

* use cas in updating latestProcessedBlock

* renaming

* also update sync height in error paths

* fix a bug

* use mutex lock

---------

Co-authored-by: colinlyguo <colinlyguo@users.noreply.github.com>
@Thegaram
feat: update L2 base fee formula (#1053)
2ed7456 
* feat: update L2 base fee formula

* update 1559 unit test

* use current block's base fee in txpool

* fix unit test

* Revert "use current block's base fee in txpool"

This reverts commit 8a7d7ff.
@omerfirmak
fix: account for tx subcircuit confidence factor (#1065)
f00dc06
@omerfirmak
fix: TestValidateTxBlockSize (#1067)
aa3af46
@sbaizet-ledger
add debugging tool to l2geth images (#1066)
c88cc48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.