Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use batched executor for shutdown node actions #86018

Merged
merged 4 commits into from
Apr 20, 2022

Conversation

rjernst
Copy link
Member

@rjernst rjernst commented Apr 19, 2022

The transport actions for put and delete of the node shutdown api
currently use an unbatched executor. This commit converts the two
classes to each use a batched executor.

closes #84847

The transport actions for put and delete of the node shutdown api
currently use an unbatched executor. This commit converts the two
classes to each use a batched executor.

closes elastic#84847
@rjernst rjernst added >refactoring :Core/Infra/Node Lifecycle Node startup, bootstrapping, and shutdown v8.3.0 labels Apr 19, 2022
@elasticmachine elasticmachine added the Team:Core/Infra Meta label for core/infra team label Apr 19, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@rjernst
Copy link
Member Author

rjernst commented Apr 19, 2022

Note: this can likely be simplified further, sharing a single executor across both put and delete, but I think that is probably beyond the scope of this PR.

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I left one question (which applies to both actions)

taskContext.onFailure(e);
continue;
}
var reroute = taskContext.getTask().rerouteService();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why pass in the reroute service through the task? It is available from ClusterService#getRerouteService at construction time, and I was expecting it to be captured as a field on the executor.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then the executor would not be static. I don't feel strongly one way or another, but I was trying to make the executor stateless (then eventually it could work across both delete and put actions).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I'd prefer to capture the reroute service on the executor. There's no real need to re-use the same executor instance across both kinds of action - in practice we'll eventually be able to safely batch tasks across executors if they're written in this style.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed e0e12a2

continue;
}
var reroute = taskContext.getTask().rerouteService();
taskContext.success(taskContext.getTask().listener().delegateFailure((l, s) -> ackAndReroute(request, l, reroute)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Observation: this is definitely emerging boilerplate:

taskContext.success(taskContext.getTask().listener().delegateFailure((l, s) -> ...

At the moment not every task has an underlying listener() so we can't yet do this in a generic way, but many tasks do and the remainder should be fixable. I hope to be able to address that to the point where this noise goes away.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole thing feels like boilerplate to me. 😄 I think with a nice base task class everything could be extracted out so this pattern is solidified and a common executor implementation can be used (at least in most cases).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted ;)

Copy link
Contributor

@pgomulka pgomulka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM2

@rjernst rjernst merged commit 7fae147 into elastic:master Apr 20, 2022
@rjernst rjernst deleted the shutdown/batch branch April 20, 2022 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/Node Lifecycle Node startup, bootstrapping, and shutdown >refactoring Team:Core/Infra Meta label for core/infra team v8.3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Node shutdown cluster state task improvements
4 participants