Repository navigation
fix: app stopped before ready is restarted - #6170
Open
aletorrado wants to merge 1 commit into
Open
aletorrado wants to merge 1 commit into
aletorrado wants to merge 1 commit into
Conversation
Since the exit-before-online change, an app that exits before being ready concludes executeApp through readyCb(), which sets its status to online. When the exit comes from pm2 stop, this overwrites the 'stopping' status, so handleExit() takes it as a crash and restarts the app. Conclude the caller without marking the app online, so handleExit() sees the real status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi! I work in production with workers in cluster_mode + wait_ready + kill_timeout + listen_timeout. I've been noticing that sometimes, after just after a cold start (while booting a server), some workers randomly were marked as online but with memory = 0.
I've debugged this issue with the help of Claude, and came to the conclusion that there was a bug in PM2 which got fixed by commit c87dd75. Nevertheless, Claude found another bug introduced in that same commit in which callbacks are clashing. I'm leaving the full description co-authored with Claude, including reproduction code below for your reference.
Problem
Since c87dd75 ("process flow refactor"), when a process exits before it is ready, both exit handlers in
God.executeApp(cluster and fork) callreadyCb()to answer the caller.readyCb()also setsstatus = online(and logs/notifiesonline) for a process that has just exited.When the exit was caused by
pm2 stop, this overwrites thestoppingstatus.handleExit()then no longer sees a stop, treats the exit as a crash, and restarts the app.Reproduction (7.0.4 and
development):Expected:
stopped. Actual: the app was restarted (↺ 1) and isonline. The log showsStopping app,exited with code [0],online,starting in -cluster mode-.Fix
Add
exitedCb(), which concludes the caller (keeping the no-hang guarantee from c87dd75) without marking the process online. Both exit handlers use it, sohandleExit()sees the real status.The
cb_calledguard still makes the pending ready timer /process:msglistener a no-op once the process has exited.Context
On 6.x the same path showed up differently: the ready timer was not cleared on exit, so
listen_timeoutlater marked an exited processonlinewith pid 0 / memory 0 (e.g. afterpm2 stopduring boot, or during arestart_delaywait). c87dd75 fixed that part; this PR fixes the restart regression it introduced.Tests
test/programmatic/stop_before_ready.mocha.js(cluster and fork mode), registered intest/unit.sh. It fails without the fix (expected 'online' to equal 'stopped') and passes with it.cluster_exit_before_online,issue_6142_stale_exit_event,issue_6129_reload_collision,graceful,cluster,god,auto_restart,exp_backoff_restart_delay.