Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make mark/sweep prepare async with main thread #947

Merged
merged 9 commits into from
May 28, 2020

Conversation

RatanRSur
Copy link
Contributor

Fixed Issue(s)

fixes #917

Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
@@ -84,12 +84,14 @@ public Pruner(
}

public void start() {

if (state.compareAndSet(State.IDLE, State.RUNNING)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the whole block be in the execute runnable? i.e. add the compare and set check into the async part? Are we sure start will only ever be called once?

Copy link
Contributor Author

@RatanRSur RatanRSur May 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could call this as many times as we like and the compareAndSet would make sure that the if body is only executed once, right? That said, I don't mind pushing the whole thing into the execute. Works for me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The executor may back up, so we could queue up a bunch of the lambdas for execution before the first one starts (which is where I presume the stat gets set to running or prepare). While the test is idempotent the execution does not appear on its surface to be, especially since we are adding observers inside the async function.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The state variable is an AtomicReference - the code inside the block is guaranteed to execute at most once as it's currently written.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nm - I think I see the issue: the startup logic could actually run after the shutdown logic because you don't know when it will actually execute 👍

Comment on lines 86 to 96
public void start() {

if (state.compareAndSet(State.IDLE, State.RUNNING)) {
LOG.info("Starting Pruner.");
executorService = executorServiceSupplier.get();
pruningStrategy.prepare();
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock);
execute(
() -> {
pruningStrategy.prepare();
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock);
});
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public void start() {
if (state.compareAndSet(State.IDLE, State.RUNNING)) {
LOG.info("Starting Pruner.");
executorService = executorServiceSupplier.get();
pruningStrategy.prepare();
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock);
execute(
() -> {
pruningStrategy.prepare();
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock);
});
}
}
public void start() {
execute(this::startAsync);
}
private void startAsync() {
LOG.info("Starting Pruner.");
executorService = executorServiceSupplier.get();
pruningStrategy.prepare();
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock);
pruningStrategy.prepare();
blockAddedObserverId = blockchain.observeBlockAdded(this::handleNewBlock);
}
}

@RatanRSur
Copy link
Contributor Author

@mbaxter I think you were the one who put the ExecutorService behind a supplier. Do you remember your rationale for that? I inlined it in this change so that we could run the start code on the executor itself.

@mbaxter
Copy link
Contributor

mbaxter commented May 28, 2020

@mbaxter I think you were the one who put the ExecutorService behind a supplier. Do you remember your rationale for that?

Yeah - I think the point of Supplier was to prevent the Pruner from spinning up resources before start is called. Otherwise, if you setup a single-threaded executor in the constructor, but never actually start the Pruner, the executor will never get shutdown because the stop method won't run.

@RatanRSur RatanRSur merged commit 303c3a1 into hyperledger:master May 28, 2020
@RatanRSur RatanRSur deleted the pruner-start-asynchrony branch May 28, 2020 18:15
ajsutton added a commit to ajsutton/besu that referenced this pull request May 29, 2020
…)"

Suspected of making PrunerTest intermittent.
ajsutton added a commit to ajsutton/besu that referenced this pull request May 29, 2020
…)"

Suspected of making PrunerTest intermittent.

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
ajsutton added a commit that referenced this pull request May 29, 2020
Suspected of making PrunerTest intermittent.

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
RatanRSur pushed a commit to RatanRSur/besu that referenced this pull request Jun 2, 2020
…)" (hyperledger#1001)

Suspected of making PrunerTest intermittent.

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
shemnon pushed a commit to shemnon/besu that referenced this pull request Jun 2, 2020
…)" (hyperledger#1001)

Suspected of making PrunerTest intermittent.

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
(cherry picked from commit f781dce)
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>
macfarla pushed a commit to macfarla/besu that referenced this pull request Jun 3, 2020
We cleared the mark storage upon starting the pruner in order to prevent the first sweep from being smaller than it needs to be.

 // Optimization for the case where the previous cycle was interrupted (like the node was shut 
 // down). If the previous cycle was interrupted, there will be marks in the mark storage from 
 // last time, causing the first sweep to be smaller than it needs to be. 
 clearMarks(); 

Unfortunately, it's currently called in prepare, which is synchronous with the main thread. This means that we can delay startup by on the order of 10 minutes if the storage was especially full.

This commit makes the pruner `start` asynchronous.

Signed-off-by: Ratan Rai Sur <ratan.r.sur@gmail.com>
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
macfarla pushed a commit to macfarla/besu that referenced this pull request Jun 3, 2020
…)" (hyperledger#1001)

Suspected of making PrunerTest intermittent.

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move initial pruning storage clear out of synchronous path
3 participants