Feature: docker-compose up -d with waiting for healthy statuses

[mik-laj]

Is your feature request related to a problem? Please describe.
I am working on a project that heavy use of depends_on filed with conditions when creating containers. This allows us to run databases, migrations, and applications reliably.

My situation can be compared to the below docker-compose.yaml file

services:

  db:
    image: debian:buster-slim
    command:
      - bash
      - -c
      - |
        echo "Start"
        for x in $$(seq 1 5); do sleep 1; echo "sleep $${x}/5"; done;
        touch /migartion-done;
        echo "OK";
        for x in $$(seq 1 60); do sleep 10; echo "sleep $$((x * 10))/600";done;
    healthcheck:
      test: ["CMD-SHELL", "test -f /migartion-done"]
      interval: "1s"
      retries: 30

  app-init:
    image: debian:buster-slim
    command:
      - bash
      - -c
      - |
        echo "OK";
    depends_on:
      db:
        condition: service_healthy

  app:
    image: debian:buster-slim
    command:
      - bash
      - -c
      - |
        echo "Start"
        for x in $$(seq 1 5); do sleep 1; echo "sleep $${x}/5"; done;
        touch /webserver-started"
        for x in $$(seq 1 60); do sleep 10; echo "sleep $$((x * 10))/600";done;
    healthcheck:
      test: [ "CMD-SHELL", "test -f /webserver-started" ]
      interval: "1s"
      retries: 30
    depends_on:
      app-init:
        condition: service_completed_successfully
      db:
        condition: service_healthy

Where

• db is a service that runs continuously but takes a while to start. Other containers should not start until this service is started and is healthy. condition: service_healthy
• app-init is a one-run container. Other containers should not start until this service is started and finish. condition: service_completed_successfully
• app is a service that runs continuously but takes a while to start.

Currently, after executing the docker-compose up command, all containers are started/completed, but docker-compose only ensures that the db and app-init containers are healthy/completed. The app container is still in an unhealthy condition.

Describe the solution you'd like
It would be fantastic if docker-compose also took into account the state of the last container (app) and waited for it to report the correct healthy status. This may be protected by an additional flag, e.g. --wait-for-healthy-status.

Describe alternatives you've considered

For now, I have implemented the health check logic in my scripts, but this is code duplication.

Alternatively, we can also add the ability to list containers by health status to docker ps --services . The docker-compose command has filters, but they are very limited. It supports only 4 states: running, stopped, paused, restarting.

compose/compose/cli/main.py

Lines 1634 to 1667 in 73f8cf9

	def has_container_with_state(containers, state):
	states = {
	'running': lambda c: c.is_running,
	'stopped': lambda c: not c.is_running,
	'paused': lambda c: c.is_paused,
	'restarting': lambda c: c.is_restarting,
	}
	for container in containers:
	if state not in states:
	raise UserError("Invalid state: %s" % state)
	if states[state](container):
	return True
	def filter_services(filt, services, project):
	def should_include(service):
	for f in filt:
	if f == 'status':
	state = filt[f]
	containers = project.containers([service.name], stopped=True)
	if not has_container_with_state(containers, state):
	return False
	elif f == 'source':
	source = filt[f]
	if source == 'image' or source == 'build':
	if source not in service.options:
	return False
	else:
	raise UserError("Invalid value for source filter: %s" % source)
	else:
	raise UserError("Invalid filter: %s" % f)
	return True
	return filter(should_include, services)

When we add support for the health status filter, it will be much easier for me to write a script waiting for all containers.

while [[ "$(docker-compose ps --services --filter state=unhealthy)" == "" ]]; do sleep 1; echo sleep 1; done;

@trajano proposed to add a separate command that will allow us to check the condition of the containers. #6408
Additional context
Add any other context or screenshots about the feature request here.

CC: @shin- , @ulyssessouza

[mik-laj]

For now, I use the following the code to wait for containers:
wait-for-container.sh

#!/usr/bin/env bash

set -euo pipefail

function usage() {
CMDNAME="$(basename -- "$0")"

  cat << EOF
Usage: ${CMDNAME} <contatiner_id>

Waits for the container to be in a healthy condition.

EXAMPLE:
$ docker-compose ps -q | xargs -n 1 -P 8 ${CMDNAME}
EOF

}

if [[ "$#" -ne 1 ]]; then
    echo "You must provide a one argument."
    echo
    usage
    exit 1
fi

CONTAINER_ID="$1"

function wait_for_container {
    container_id="$1"
    container_name="$(docker inspect "${container_id}" --format '{{ .Name }}')"
    echo "Waiting for container: ${container_name} [${container_id}]"
    waiting_done="false"
    while [[ "${waiting_done}" != "true" ]]; do
        container_state="$(docker inspect "${container_id}" --format '{{ .State.Status }}')"
        if [[ "${container_state}" == "running" ]]; then
            health_status="$(docker inspect "${container_id}" --format '{{ .State.Health.Status }}')"
            echo "${container_name}: container_state=${container_state}, health_status=${health_status}"
            if [[ ${health_status} == "healthy" ]]; then
                waiting_done="true"
            fi
        else
            echo "${container_name}: container_state=${container_state}"
            waiting_done="true"
        fi
        sleep 1;
    done;
}
wait_for_container "$1"

COMPOSE_FILE="${compose_file}" docker-compose ps -q | xargs -n 1 -P 8 ./wait-for-container.sh

Example output

Waiting for container: /airflow-docker-compose-examples_airflow-init_1 [a1cd0b5b93d25d33d1e4c640cb225cf19d81f4da462dee85ea7edd82697e0c47]
Waiting for container: /airflow-docker-compose-examples_redis_1 [ee5d03e5a988b6c5f775f26a74cdbf48666fb9c12015a3b5a382d0f0fc331495]
Waiting for container: /airflow-docker-compose-examples_postgres_1 [dcfd2bb529a7acd73e116506f4ed3076aeeb290881ddac15391b6a5dac5a768a]
Waiting for container: /airflow-docker-compose-examples_flower_1 [1d318511d40ebf016719667006430d919b712d3ebf81a3c7ea8af6a2e425c901]
/airflow-docker-compose-examples_airflow-init_1: container_state=exited
/airflow-docker-compose-examples_airflow-worker_1: container_state=running, health_status=starting
/airflow-docker-compose-examples_airflow-webserver_1: container_state=running, health_status=starting
/airflow-docker-compose-examples_airflow-scheduler_1: container_state=running, health_status=starting
/airflow-docker-compose-examples_postgres_1: container_state=running, health_status=healthy
/airflow-docker-compose-examples_redis_1: container_state=running, health_status=healthy
/airflow-docker-compose-examples_flower_1: container_state=running, health_status=starting
/airflow-docker-compose-examples_airflow-scheduler_1: container_state=running, health_status=healthy
/airflow-docker-compose-examples_flower_1: container_state=running, health_status=healthy
/airflow-docker-compose-examples_airflow-worker_1: container_state=running, health_status=starting
/airflow-docker-compose-examples_airflow-webserver_1: container_state=running, health_status=starting
/airflow-docker-compose-examples_airflow-worker_1: container_state=running, health_status=healthy
/airflow-docker-compose-examples_airflow-webserver_1: container_state=running, health_status=starting
/airflow-docker-compose-examples_airflow-webserver_1: container_state=running, health_status=healthy

[ndeloof]

I have the same need:
we use postgreSQL and, for development, run it as docker-compose up -d, then have a wait script, and eventually apply database migrations. Would be nice we can just run docker-compose up -d --wait SERVICE:condition with the same syntax used by depends_on.

I'll prepare a PR to implement this feature

[mik-laj]

@ndeloof Have you thought about adding a flag that will allow you to wait until ALL services are healthy? I think. about something like docker-compose up -d --wait-for-all condition or simply docker-compose up -d --wait-for-all-healthy. I don't need to wait for a specific one, but I just want to know when the docker environment has been fully launched.

On your proposal, I will still have to use scripts to achieve my goal. It will be something similar to the following.

 docker-compose ps --services | xargs -I {} echo "--wait {}:healthy" | xargs docker-compose up -d

[ndeloof]

yes, compose up --wait healthy would make sense indeed. Could support both SERVICE:condition and just plain condition to target all services.

[max-tet]

This can be closed, right?
👉 #8777

[Luc45]

The newly added --wait flag in Docker Compose v2 is the way to go, but if you are stuck with docker-compose v1, this bogus service will achieve a similar result:

wait:
  image: hello-world:latest
  depends_on:
      some-container:
          condition: service_healthy

[ericslandry]

@mik-laj The multiple calls to docker inspect were too slow for me. I used jq to help

while : ; do
  # shellcheck disable=SC2046
  problem=$(docker inspect $(docker compose ps -aq) | \
    jq -r '.[] | "\(.State.Status) \(.State.ExitCode) \(.State.Health.Status)"' | \
    sed "/exited 0.*/d"|sed "/running 0 healthy/d"|sed "/running 0 null/d")
  [ -n "$problem" ] || break
  sleep 1
done

[ndeloof]

Closing as fixed by #8777