fix(docker): start rabbitmq directly and wait for readiness - #332
Open
chrip wants to merge 2 commits into
Open
Conversation
PR #249 switched the standalone entrypoint from a direct detached start to `service rabbitmq-server start` for consistency with postgres and redis. The init script goes through `start-stop-daemon --background`, which wedges under a very large RLIMIT_NOFILE. Docker daemons that pass one through to containers (26.1.5+dfsg1 defaulting to 1073741816 was reported twice) leave the entrypoint stuck before nginx and supervisord ever start: the container looks hung, /healthcheck resets the connection, and nothing in the log says why. Setting `ulimits.nofile` in compose is a workaround, but users have no way to know that is what is wrong. Go back to `rabbitmq-server -detached`, which is unaffected, and follow it with a bounded `rabbitmqctl await_startup` poll (AMQP_START_TIMEOUT, default 60s). `-detached` returns before the node registers with epmd, so a single await_startup call fails outright and has to be polled. A timeout now logs rabbitmqctl's own diagnostics and continues to supervisord, like the schema bootstrap above it, so the failure is visible instead of silent. Apply the same at image build time, where the identical hang would break builds on affected hosts, and drop the mnesia database the build leaves behind: the node name follows the container hostname, so the baked `rabbit@buildkitsandbox` directory is never read at runtime. Fixes #326 Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Christoph Schaefer <christoph.schaefer@nextcloud.com>
Member
|
Not sure, but the failed builds appear to be related to the changes. |
The build-time wait added alongside the entrypoint one called `rabbitmqctl await_startup` a single time, and lost the race on three of the four CI builders: exit 69 after seven seconds with "epmd reports: node 'rabbit' not running at all", long before its own 60s timeout could matter. The fourth passed only because the node happened to register first, so a green run there proved nothing. start_rabbitmq() in entrypoint.sh already documents why: `-detached` returns before the node registers with epmd, and until it does await_startup fails outright instead of waiting. The Dockerfile needed the same poll and did not get it. Bounded with `timeout` so a node that never starts still fails the build, and the wait is then repeated unsuppressed so rabbitmqctl's diagnostics reach the build log rather than just an exit code. Assisted-by: ClaudeCode:claude-opus-5 Signed-off-by: Christoph Schaefer <christoph.schaefer@nextcloud.com>
Contributor
Author
|
The wait during the image build asked RabbitMQ "are you ready?" ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR #249 switched the standalone entrypoint from a direct detached start to
service rabbitmq-server startfor consistency with postgres and redis. The init script goes throughstart-stop-daemon --background, which wedges under a very large RLIMIT_NOFILE. Docker daemons that pass one through to containers (26.1.5+dfsg1 defaulting to 1073741816 was reported twice) leave the entrypoint stuck before nginx and supervisord ever start: the container looks hung, /healthcheck resets the connection, and nothing in the log says why. Settingulimits.nofilein compose is a workaround, but users have no way to know that is what is wrong.Go back to
rabbitmq-server -detached, which is unaffected, and follow it with a boundedrabbitmqctl await_startuppoll (AMQP_START_TIMEOUT, default 60s).-detachedreturns before the node registers with epmd, so a single await_startup call fails outright and has to be polled. A timeout now logs rabbitmqctl's own diagnostics and continues to supervisord, like the schema bootstrap above it, so the failure is visible instead of silent.Apply the same at image build time, where the identical hang would break builds on affected hosts, and drop the mnesia database the build leaves behind: the node name follows the container hostname, so the baked
rabbit@buildkitsandboxdirectory is never read at runtime.Fixes #326
Assisted-by: ClaudeCode:claude-opus-5