Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable txpool when not in sync #6302

Merged
merged 19 commits into from
Jan 10, 2024

Conversation

fab-10
Copy link
Contributor

@fab-10 fab-10 commented Dec 14, 2023

PR description

With this PR the transaction pool factory is now aware if Besu is in sync or not, using the events triggered by the SyncState, and will disable the txpool as soon as the node is out of sync, so avoiding validating txs while not necessary.
This PR also implements a better way to process the onBlockAdded event in the txpool, avoiding the possibility of a tight loop that could result in high cpu usage.

Fixed Issue(s)

fixes #6001

…ze full sync

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Copy link

github-actions bot commented Dec 14, 2023

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.
  • I thought about the changelog and included a changelog update if required.
  • If my PR includes database changes (e.g. KeyValueSegmentIdentifier) I have thought about compatibility and performed forwards and backwards compatibility tests

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
@fab-10 fab-10 force-pushed the disable-txpool-during-fullsync branch 5 times, most recently from 1ec9dec to f63e65d Compare December 19, 2023 14:59
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
@fab-10 fab-10 force-pushed the disable-txpool-during-fullsync branch from f63e65d to 2509e05 Compare December 19, 2023 17:05
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
@fab-10 fab-10 changed the title Disable txpool during fullsync Disable txpool when not in sync Dec 21, 2023
# Conflicts:
#	ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
@fab-10 fab-10 marked this pull request as ready for review December 21, 2023 13:51
@ahamlat
Copy link
Contributor

ahamlat commented Jan 2, 2024

@fab-10 I tested on a mainnet forest archive sync'ing node, and it worked. I don't see anymore the validation of remote transactions. I see the deactivation in the logs :

{"@timestamp":"2024-01-02T08:52:02,252","level":"INFO","thread":"nioEventLoopGroup-3-6","class":"TransactionPoolFactory","message":"Node out of sync, disabling transaction handling","throwable":""}

I also confirmed by checking the metrics :
image
image

However, I still see that we are announcing new transactions hashes :

image

@fab-10
Copy link
Contributor Author

fab-10 commented Jan 3, 2024

@ahamlat about

I still see that we are announcing new transactions hashes

what you see are the incoming messages that we still receive, but are just ignored and not processed, you should check the outbound panel to verify that we are not sending them.

@ahamlat
Copy link
Contributor

ahamlat commented Jan 4, 2024

Right, I was looking to the inbound messages. When checking outbound messages on the transaction pool, we can see that we have no traffic, which validates this PR.

image

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
@fab-10 fab-10 enabled auto-merge (squash) January 10, 2024 13:24
@fab-10 fab-10 merged commit 0b4b815 into hyperledger:main Jan 10, 2024
18 checks passed
@fab-10 fab-10 deleted the disable-txpool-during-fullsync branch January 10, 2024 14:01
fab-10 added a commit to Consensys/linea-besu that referenced this pull request Jan 12, 2024
* mark deleted slot during clear storage step (hyperledger#6305)

Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Co-authored-by: garyschulte <garyschulte@gmail.com>

* made directory structure of tests match source; fixed one typo (hyperledger#6337)

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* migrate controller tests to junit 5 (hyperledger#6338)

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* add new forkids for testnets, update forkid test to Junit5, no longer need named network specific trusted setups (hyperledger#6322)

Signed-off-by: jflo <justin+github@florentine.us>

* Fix trielog shipping issue during self destruct (hyperledger#6340)

* fix trielog shipping issue

Signed-off-by: Karim Taam <karim.t2am@gmail.com>

* bump gradle properties version and adjust changelog to match release (hyperledger#6347)

Signed-off-by: garyschulte <garyschulte@gmail.com>

* finalized cancun spec (hyperledger#6351)

* finalized cancun spec

Signed-off-by: jflo <justin+github@florentine.us>

* finalized cancun spec

Signed-off-by: jflo <justin+github@florentine.us>

---------

Signed-off-by: jflo <justin+github@florentine.us>

* Optimize RocksDB WAL file (hyperledger#6328)


Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Make RPC reason settable, pass execution failure reason in RPC error message (hyperledger#6343)

* Make RPC reason settable, pass execution failure reason in RPC error message

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Update unit tests

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Update tests

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Update change log

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Update integration tests

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

---------

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* TestWatcher junit5 (hyperledger#6339)

* TestWatcher junit5
* add test class and method name to context
* moved the testwatcher junit5 function to a new junit5 superclass
* one qbft test to junit5 superclass

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>

* Migrate BFT tests to junit 5 (hyperledger#6350)

* bft tests to junit 5
* base class for pki extend AcceptanceTestBaseJunit5
* try/catch in case of empty optionals
* fixed parameterization method

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* fixing on selfdestruct (hyperledger#6359)

Signed-off-by: Karim Taam <karim.t2am@gmail.com>

* migrate clique tests fully to junit5 (hyperledger#6362)

* migrate clique tests fully to junit5

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* fixed link to logging docs (hyperledger#6366)

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* Move logging to RunnerBuilder (hyperledger#6367)

Signed-off-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>

* Use synchronized call to access the chain head block in `eth_estimateGas` (hyperledger#6345)

* Use synchronized call to access the chain head block in estimateGas()

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Add error log entries when throwing internal error from estimateGas()

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Update unit tests

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Update changelog

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

---------

Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>

* Add --X-trie-log subcommand (hyperledger#6303)

* Add x-trie-log subcommand for one-off trie log backlog prune

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>

---------

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>

* fix typos (hyperledger#6368)

Signed-off-by: vuittont60 <81072379+vuittont60@users.noreply.github.com>

* Added alias --sync-min-peers for --fast-sync-min-peers (hyperledger#6372)

* sync-min-peers as an alias
* added unit tests

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

---------

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* Fix: Fallback to getName when canonicalName is null in BlockHeaderValidator DEBUG log (hyperledger#6332)

* fallback to simple name when canonical name is null
* use getName instead of getSimpleName to include the package name

Signed-off-by: Manoj P R <manojpramesh@gmail.com>

---------

Signed-off-by: Manoj P R <manojpramesh@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>

* fix: use UID 1000 for besu user (hyperledger#6358) (hyperledger#6360)

The openjdk-latest Docker image is using UID 1001 for besu, because its
base image ubuntu:23.10 now contains a default "ubuntu" user with UID
1000. (This UID change causes the besu user with UID 1001 to not have
access to files created for past versions with UID 1000.)

We now remove the default ubuntu user and explicitly use UID 1000 when
creating the besu user.

Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk>

* Ignore generated files when running the spdx license check (hyperledger#6379)

Signed-off-by: Meredith Baxter <mbaxter.dev@gmail.com>

* full sync - don't fail startup if sync-min-peers specified (hyperledger#6373)

Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>

* Copy also computed fields, when doing a Transaction detached copy (hyperledger#6329)

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Disable txpool when not in sync (hyperledger#6302)


Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump to nex release snapshot 24.1.1 (hyperledger#6383)

* release next snapshot 24.1.1

Signed-off-by: garyschulte <garyschulte@gmail.com>

* move recent changelog items to 24.1.1-SNAPSHOT

Signed-off-by: garyschulte <garyschulte@gmail.com>

---------

Signed-off-by: garyschulte <garyschulte@gmail.com>

* Correct Tangerine Whistle definition in Fluent EVM APIs. (hyperledger#6382)

The fluent API incorrectly added the code size limit in Tangerine
Whistle instead of first adding it in Spurious Dragon.

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>

* [MINOR] Fix pki tests condition check on mac (hyperledger#6387)

Signed-off-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>

* Upgrade dependencies (hyperledger#6377)

* Bump com.github.oshi:oshi-core to 6.4.10

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump com.github.tomakehurst to org.wiremock 3.3.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump com.google.auto.service:auto-service to 1.1.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump com.google.dagger group to 2.50

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump com.graphql-java:graphql-java to 21.3

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump com.splunk.logging:splunk-library-javalogging to 1.11.8

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump com.squareup.okhttp3:okhttp to 4.12.0
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump commons-io:commons-io to 2.15.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump dnsjava:dnsjava to 3.5.3

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump info.picocli group to 4.7.5

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump io.grpc group to 1.60.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump io.kubernetes:client-java to 18.0.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump io.netty group to 4.1.104.Final

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump net.java.dev.jna:jna to 5.14.0

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.apache.commons:commons-compress to 1.25.0

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.apache.commons:commons-lang3 to 3.14.0

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.apache.commons:commons-text to 1.11.0

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.apache.logging.log4j group to 2.22.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Redorder io.tmio group

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.assertj:assertj-core to 3.25.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.bouncycastle group to 1.77

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.fusesource.jansi:jansi to 2.4.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.immutables group 2.10.0

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.java-websocket:Java-WebSocket to 1.5.5

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.jetbrains.kotlin:kotlin-stdlib to 1.9.22

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.junit.jupiter group to 5.10.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.jupnp group to 2.7.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.rocksdb:rocksdbjni to 8.9.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.slf4j group to 2.0.10

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.springframework.security:spring-security-crypto to 6.2.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.testcontainers:testcontainers to 1.19.3

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.web3j group to 4.10.3

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Bump org.xerial.snappy:snappy-java to 1.1.10.5

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Regenerate gradle verification metadata

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update commons-codec:commons-codec to 1.16.0

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update org.junit.vintage:junit-vintage-engine to 5.10.1

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

---------

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* add a fallback for docker detection on Mac (hyperledger#6356)

Signed-off-by: garyschulte <garyschulte@gmail.com>

* Fix test flackyness of acceptanceTestsPermissioning  (hyperledger#6384)


Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>

* Upgrade `com.fasterxml.jackson` dependencies (hyperledger#6378)


Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

* Use mining beneficiary from protocol spec in TraceServiceImpl (hyperledger#6390)

* use mining beneficiary from protocol spec

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

---------

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>

* Update verification metadata and allowed licenses for Linea-Besu

---------

Signed-off-by: Karim Taam <karim.t2am@gmail.com>
Signed-off-by: Sally MacFarlane <macfarla.github@gmail.com>
Signed-off-by: jflo <justin+github@florentine.us>
Signed-off-by: garyschulte <garyschulte@gmail.com>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Matthew Whitehead <matthew1001@gmail.com>
Signed-off-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Signed-off-by: vuittont60 <81072379+vuittont60@users.noreply.github.com>
Signed-off-by: Manoj P R <manojpramesh@gmail.com>
Signed-off-by: Hal Blackburn <hwtb2@cam.ac.uk>
Signed-off-by: Meredith Baxter <mbaxter.dev@gmail.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Co-authored-by: Karim TAAM <karim.t2am@gmail.com>
Co-authored-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: Matt Whitehead <matthew1001@gmail.com>
Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: Gabriel-Trintinalia <gabriel.trintinalia@consensys.net>
Co-authored-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: vuittont60 <81072379+vuittont60@users.noreply.github.com>
Co-authored-by: Manoj P R <manojpramesh@gmail.com>
Co-authored-by: Hal Blackburn <hal.blackburn@gmail.com>
Co-authored-by: mbaxter <mbaxter.dev@gmail.com>
Co-authored-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Co-authored-by: daniellehrner <daniel.lehrner@consensys.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Disable TransactionPool observer on new blocks during Full sync
3 participants