LifecycleGroup concurrent start and start timeout

[fralalonde]

Context

Our app communicates with multiple (physical) machines, each being driven by a separate bean. When the app starts, each driver bean resets its machine to a known state. This reset can take a few minutes before which the rest of the app (UI, etc) should not be available. The driver beans implement SmartLifecycle, hardware reset is performed on start().

All machines can be reset at the same time, so to minimize startup time I'm currently using a custom LifecycleProcessor forked off the DefaultLifecycleProcessor , for which I've implemented concurrent start for beans within the same phase (in LifecycleGroup::start()). This makes the LifecycleGroup start take as long as the slowest machine's reset, rather than sequentially resetting each one which would be unnecessarily slow.

Moreover, I've duplicated the existing per-phase (stop) timeout parameter to a start timeout, to make sure the app fails if starting takes more than some expected limit rather than remain in start limbo forever.

Proposal

My main assumption is that situations where Lifecycle beans depend on external resources happen frequently, not just in our case. Spring applications could gain both in performance and resilience by making it easier to use concurrent start / stop operations.

To do so would involve either:

• Integrate concurrent operation + timeout in the existing DefaultLifecycleProcessor, using an opt-in switch per phase (default behavior would remain sequential start with no timeout) (e.g. .setConcurrentStart(phase, true))

OR

• Make it possible to override the LifecycleGroup class with a custom class to allow implementing aforementioned concurrent start mechanism without having to duplicate the entire DefaultLifecycleProcessor.

Thoughts

• Beans within the same phase should be safe to start concurrently, since by design they should not have inter-dependencies. While I know this is true for my app, this is not a part of the current Lifecycle contract and thus the default should remain sequential, non-timeout start.

• The underlying support for concurrency could vary a lot from case to case (virtual threads? limit number of concurrent processes?), so maybe the extendable DefaultLifecycleProcessor option is more interesting.

[jhoeller]

On a related note: As of 6.2, the framework comes with a first-class option for background bootstrapping: #19487. However, that's at the bean creation level, not at the lifecycle start/stop level which comes as a final step in application startup. That background bootstrap mechanism is rather about using multiple CPU cores for expensive configuration and bean construction efforts on startup.

It's interesting to see that there is also a benefit in concurrent invocations of Lifecycle.start. Our assumption was that start() would usually be implemented in a quick enough fashion, doing most of the work asynchronously. Whereas in your scenario, the start() calls actually seem to block, so I assume that the component-level start() implementations do not hand off anything to a thread pool themselves? In your custom LifecycleProcessor variant, are you using a processor-level thread pool to manage concurrent calls to component-level start() implementations and eventually joining for their collective completion (per lifecycle group)?

[fralalonde]

I've considered it, but wouldn't asynchronous initialization defeat the purpose of having Lifecycle phases? I assume that if component B initializes in a later phase than component A, it can rely on it having completed it's initialization, which wouldn't be guaranteed without a callback from A saying it's done starting() (...or failed) or the LifecycleProcessor polling beans of a phase until they're all running() before moving on to the next phase.

My goal in binding bean initialization to the context's lifecycle is to guarantee the integrity of the app. If a bean fails to start() (or takes too long), the context fails to refresh(), making the error obvious rather than having to deal with dirty intermediate states. The blocking behavior of start() is a benefit because it prevents accessing the app before the system is fully up.

I was thinking that "ready-flag" bean could be declared as the sole bean of the last lifecycle phase and serve as an indicator to a RequestFilter that the system is now in a stable state. This way the webserver would be accessible early, giving out polite error messages until the context init either completes, fails or times out.

[jhoeller]

Sounds sensible. I don't think there is anything wrong with this approach, and we can certainly open up DefaultLifecycleProcessor for better extensibility and consider concurrent starting as a first-class option.

Can you elaborate a bit on the behavior that you are currently implementing in a custom LifecycleProcessor variant?

[fralalonde]

Nothing fancy, I copied the existing DefaultLifecycleProcessor as a basis, then stripped out the CRaC stuff because I don't have the required dependencies.

The core of my changes are in ConcurrentLifecycleProcessor.LifecycleGroup::start()

        public void start() {
            if (this.members.isEmpty()) {
                return;
            }
            if (logger.isDebugEnabled()) {
                logger.debug("Starting beans in phase " + this.phase);
            }

            List<CompletableFuture<Void>> startFutures = this.members.stream()
                    .map(member -> CompletableFuture.runAsync(() -> doStart(this.lifecycleBeans, member.name, this.autoStartupOnly)))
                    .toList();

            try {
                CompletableFuture.allOf(startFutures.toArray(new CompletableFuture[0]))
                        .get(startTimeout, TimeUnit.MILLISECONDS);
            } catch (TimeoutException e) {
                throw new IllegalStateException("Timeout exceeded starting beans in phase " + this.phase, e);
            } catch (InterruptedException | ExecutionException e) {
                throw new IllegalStateException("Error starting beans in phase " + this.phase, e);
            }
        }

The only changes to the containing ConcurrentLifecycleProcessor class are about the now-split start/stop timeout config fields:

    private final Map<Integer, Long> timeoutsForStartPhases = new ConcurrentHashMap<>();

    private volatile long timeoutPerStartPhase = 10000;

    private final Map<Integer, Long> timeoutsForShutdownPhases = new ConcurrentHashMap<>();

    private volatile long timeoutPerShutdownPhase = 10000;

 [...]

    private long determineStartTimeout(int phase) {
        Long timeout = this.timeoutsForStartPhases.get(phase);
        return (timeout != null ? timeout : this.timeoutPerStartPhase);
    }

    private long determineStopTimeout(int phase) {
        Long timeout = this.timeoutsForShutdownPhases.get(phase);
        return (timeout != null ? timeout : this.timeoutPerShutdownPhase);
    }

Keep in mind this is code from my proof on concept and so may not cover all the bases. It ran ok when I tested it, both beans started() at the same time and total init time was equivalent to the longest delay of the two beans. I'm comfortable enough with it to ship it as part of the app but I'm conscious it may not be up to the standards of a reusable library.

Notably missing here are the boolean flags to enable parallel operation per phase, which I do not require.

Cursory analysis makes me think that making DefaultLifecycleProcessor itself extensible might not be possible because of its LifecycleGroup private "associated" class, which is where the core changes go. Opening up the mechanism may require a parent AbstractLifecycleProcessor of some sort as it is done elsewhere.

I'm happy to keep discussing this but I'm sure you have preferences about how this should be done so I'll let you go first.