-
Notifications
You must be signed in to change notification settings - Fork 882
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
make mark/sweep prepare async with main thread #947
make mark/sweep prepare async with main thread #947
Conversation
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
@@ -84,12 +84,14 @@ public Pruner( | |||
} | |||
|
|||
public void start() { | |||
|
|||
if (state.compareAndSet(State.IDLE, State.RUNNING)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the whole block be in the execute runnable? i.e. add the compare and set check into the async part? Are we sure start will only ever be called once?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we could call this as many times as we like and the compareAndSet
would make sure that the if body is only executed once, right? That said, I don't mind pushing the whole thing into the execute. Works for me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The executor may back up, so we could queue up a bunch of the lambdas for execution before the first one starts (which is where I presume the stat gets set to running or prepare). While the test is idempotent the execution does not appear on its surface to be, especially since we are adding observers inside the async function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The state
variable is an AtomicReference
- the code inside the block is guaranteed to execute at most once as it's currently written.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nm - I think I see the issue: the startup logic could actually run after the shutdown logic because you don't know when it will actually execute 👍
public void start() { | ||
|
||
if (state.compareAndSet(State.IDLE, State.RUNNING)) { | ||
LOG.info("Starting Pruner."); | ||
executorService = executorServiceSupplier.get(); | ||
pruningStrategy.prepare(); | ||
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock); | ||
execute( | ||
() -> { | ||
pruningStrategy.prepare(); | ||
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock); | ||
}); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
public void start() { | |
if (state.compareAndSet(State.IDLE, State.RUNNING)) { | |
LOG.info("Starting Pruner."); | |
executorService = executorServiceSupplier.get(); | |
pruningStrategy.prepare(); | |
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock); | |
execute( | |
() -> { | |
pruningStrategy.prepare(); | |
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock); | |
}); | |
} | |
} | |
public void start() { | |
execute(this::startAsync); | |
} | |
private void startAsync() { | |
LOG.info("Starting Pruner."); | |
executorService = executorServiceSupplier.get(); | |
pruningStrategy.prepare(); | |
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock); | |
pruningStrategy.prepare(); | |
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock); | |
} | |
} |
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
@mbaxter I think you were the one who put the |
Yeah - I think the point of |
ethereum/core/src/main/java/org/hyperledger/besu/ethereum/worldstate/Pruner.java
Show resolved
Hide resolved
ethereum/core/src/main/java/org/hyperledger/besu/ethereum/worldstate/Pruner.java
Show resolved
Hide resolved
Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
…)" Suspected of making PrunerTest intermittent.
…)" Suspected of making PrunerTest intermittent. Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
…)" (hyperledger#1001) Suspected of making PrunerTest intermittent. Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
…)" (hyperledger#1001) Suspected of making PrunerTest intermittent. Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net> (cherry picked from commit f781dce) Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
We cleared the mark storage upon starting the pruner in order to prevent the first sweep from being smaller than it needs to be. // Optimization for the case where the previous cycle was interrupted (like the node was shut // down). If the previous cycle was interrupted, there will be marks in the mark storage from // last time, causing the first sweep to be smaller than it needs to be. clearMarks(); Unfortunately, it's currently called in prepare, which is synchronous with the main thread. This means that we can delay startup by on the order of 10 minutes if the storage was especially full. This commit makes the pruner `start` asynchronous. Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com> Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
…)" (hyperledger#1001) Suspected of making PrunerTest intermittent. Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net> Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Fixed Issue(s)
fixes #917