Suggestion: a "docker pull --dry-run" option

[jmlord]

Description

It would be useful in some automation environment to detect if there are new images available without pulling them right away. In my case, the launcher script detects if there is an update, and offers it to the user.

In an ideal world, I would launch docker pull --dry-run and either parse the output or check the return code.

Migrated from this issue, as suggested in the thread to make a feature request in the Docker CLI :
distribution/distribution#2412

Workaround:

It is possible to achieve this using image inspection. The following script works in my case but has major flaws : is is slow, will break if the manifest structure changes, and works only because the variant I use happens to be the first in the remote repo's manifest. I feel that there is nothing better that a docker pull to know what a docker pull should be doing, and any scripting attempt like this one is an approximation.

    images=$(docker compose config | grep 'image:' | awk '{print $2}')
    for image in $images; do
        # Get the local digest in the format sha256:<hash>
        localDigest=$(docker image inspect --format='{{index .Id}}' $image 2>/dev/null)
        if [[ $? -ne 0 ]]; then
            echo "At least one image not found locally: $image"
            updatesFound=true
            break
        fi
        echo $localDigest

        # Get the remote digest in format sha256:<hash>
        # This works ONLY if the desired config is from the first variant
        # Would have been cleaner with jq but it is not available in a git bash on Windows...
        remoteDigest=$(docker manifest inspect -v $image \
            | awk '/"config":/ {found=1} found&& /"digest"/ {print $2; exit}' \
            | tr -d '",')
        echo $remoteDigest

        # Perform comparison
        if [[ "$localDigest" == "$remoteDigest" ]]; then
            echo "Up to date: $image"
        else
            echo "At least one image outdated: $image"
            updatesFound=true
            break
        fi
    done

[thaJeztah]

This is something that has been discussed a few times (and possibly could be considered; it would require changes on the daemon-side though, which is where the actual pull is handled). That said; do you have more context on your use-case; effectively, a docker pull would not pull anything if the image is up-to-date;

docker pull alpine
Using default tag: latest
latest: Pulling from library/alpine
Digest: sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c
Status: Image is up to date for alpine:latest
docker.io/library/alpine:latest

docker run --pull always --rm alpine echo foo
latest: Pulling from library/alpine
Digest: sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c
Status: Image is up to date for alpine:latest
foo

I should note that with the containerd image-store integration enabled;

• https://docs.docker.com/desktop/features/containerd/
• https://docs.docker.com/engine/storage/containerd/

The digest of the image will now reflect the digests of the "distributed" image, so should match the top-most digest from docker manifest inspect (or `docker buildx imagetools inspect)

docker image ls --no-trunc alpine
REPOSITORY   TAG       IMAGE ID                                                                  CREATED        SIZE
alpine       latest    sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c   2 months ago   12.8MB

docker buildx imagetools inspect alpine | head -n 3
Name:      docker.io/library/alpine:latest
MediaType: application/vnd.oci.image.index.v1+json
Digest:    sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c

[jmlord]

Hi Sebastiaan,

Thanks for your reply,

do you have more context on your use-case; effectively, a docker pull would not pull anything if the image is up-to-date;

In our context, we have a few services running, that can be started by the users. Most users will know nothing about docker except that it is installed on their machines, the rest is handled by the launcher script.

Why do we want to check for updates, instead of automatically pulling them?

1. The total download is over 3GB, which can be slow for some of our users with limited bandwidth.
2. Our containers contain dependencies (R, conda) that are installed at runtime. Installing a single R dependency can take from a few seconds to nearly an hour. While the dependencies were previously stored on volumes to avoid losing them, this caused compatibility problems when the containers updated. Hence, they are now discarded with the container. It is therefore natural to ask them permission before discarding the containers, knowing it can seriously delay their work.

I therefore added a the above digest checking procedure to our launcher script prod-server.sh, currently in a branch here: https://github.com/GEO-BON/bon-in-a-box-pipeline-engine/tree/mamba_update_staging
I am wary though that this can break at any moment with updates from the docker CLI tools.