Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker pull library/nginx:latest is pulling a 3 month old release instead of most current #863

Open
cah-michael-mcghee opened this issue Feb 1, 2024 · 3 comments

Comments

@cah-michael-mcghee
Copy link

Describe the bug

I have attempted to perform a docker pull nginx:latest and have even tried nginx: on several different types of systems (Ubuntu (WSL), RaspberryPi, Windows/Docker Desktop). All of them pull an older version of nginx then the latest. Performing docker images show created 3 months ago and docker inspect show a different digest. Have tried nginx:stable and that one says created 9 months ago.

other docker pull of different repo appear to pull the correct release.

docker releases are 24.x and higher

@yosifkit
Copy link
Contributor

yosifkit commented Feb 2, 2024

The images are actually very new (built yesterday because of the Debian base image update: docker-library/official-images#16155).

To work toward more reproducible layers/images, the Docker Official Images builds are moving to be built with SOURCE_DATE_EPOCH set to the commit timestamp of the respective Dockerfile+context. This is where the old dates are coming from. You can see it has been built recently in a few ways.

  • By looking at the org.opencontainers.image.created annotation in the manifest: https://explore.ggcr.dev/?image=nginx%3Alatest. This timestamp is saved just before the docker build happens.

  • by inspecting the layers of it and its parent image (debian:bookworm-slim) and compare their timestamps. Condensed output showing their created date and the new debian layer diff_id in the layer diff_ids of the "old" nginx image:

$ docker buildx imagetools inspect debian:bookworm-slim --format '{{ json (index .Image "linux/amd64")}}'
{
  "created": "2024-01-31T22:35:18.796241083Z",
...
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:fb1bd2fc52827db4ce719cc1aafd4a035d68bc71183b3bc39014f23e9e5fa256"
    ]
...
$ docker buildx imagetools inspect nginx:latest --format '{{ json (index .Image "linux/amd64")}}'
{
  "created": "2023-10-24T22:44:45Z",
...
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:fb1bd2fc52827db4ce719cc1aafd4a035d68bc71183b3bc39014f23e9e5fa256",
      "sha256:83bdf27d9eaa4f0e8f11b95dc952e9ab92619792dbd3d72f775d54c67eae1248",
      "sha256:84e0c9ef07d70aff145d02f297c94d7d5cc0ff7d87449c20eac02f5408a48fd0",
      "sha256:06536efc503aa0d192e8a5232eb57cad57fce55a8a92dce48d45eaf284f0cc26",
      "sha256:9f21a390e3f6d9e5af2d02fdb6a622eded91c58c9fe82af76204ad4cd854da39",
      "sha256:2b28485849eadbf9f3066df698b1579f169a7a306ce754a9809349099a58d11d",
      "sha256:f205d290cd763a4d8dddb06bee8fbd813cea79b179515fde9dec1b9fd9525856"
    ]
...
full `docker buildx imagetools` output:
$ docker buildx imagetools inspect debian:bookworm-slim --format '{{ json (index .Image "linux/amd64")}}'
{
  "created": "2024-01-31T22:35:18.796241083Z",
  "architecture": "amd64",
  "os": "linux",
  "config": {
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Cmd": [
      "bash"
    ]
  },
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:fb1bd2fc52827db4ce719cc1aafd4a035d68bc71183b3bc39014f23e9e5fa256"
    ]
  },
  "history": [
    {
      "created": "2024-01-31T22:35:18.499703981Z",
      "created_by": "/bin/sh -c #(nop) ADD file:af0f4e41d68b67ca88a1ce6297326159e18e27670d7bfc0bf5804a4e2b268cc8 in / "
    },
    {
      "created": "2024-01-31T22:35:18.796241083Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"bash\"]",
      "empty_layer": true
    }
  ]
}
$ docker buildx imagetools inspect nginx:latest --format '{{ json (index .Image "linux/amd64")}}'
{
  "created": "2023-10-24T22:44:45Z",
  "architecture": "amd64",
  "os": "linux",
  "config": {
    "ExposedPorts": {
      "80/tcp": {}
    },
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "NGINX_VERSION=1.25.3",
      "NJS_VERSION=0.8.2",
      "PKG_RELEASE=1~bookworm"
    ],
    "Entrypoint": [
      "/docker-entrypoint.sh"
    ],
    "Cmd": [
      "nginx",
      "-g",
      "daemon off;"
    ],
    "Labels": {
      "maintainer": "NGINX Docker Maintainers \u003cdocker-maint@nginx.com\u003e"
    },
    "StopSignal": "SIGQUIT",
    "ArgsEscaped": true
  },
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:fb1bd2fc52827db4ce719cc1aafd4a035d68bc71183b3bc39014f23e9e5fa256",
      "sha256:83bdf27d9eaa4f0e8f11b95dc952e9ab92619792dbd3d72f775d54c67eae1248",
      "sha256:84e0c9ef07d70aff145d02f297c94d7d5cc0ff7d87449c20eac02f5408a48fd0",
      "sha256:06536efc503aa0d192e8a5232eb57cad57fce55a8a92dce48d45eaf284f0cc26",
      "sha256:9f21a390e3f6d9e5af2d02fdb6a622eded91c58c9fe82af76204ad4cd854da39",
      "sha256:2b28485849eadbf9f3066df698b1579f169a7a306ce754a9809349099a58d11d",
      "sha256:f205d290cd763a4d8dddb06bee8fbd813cea79b179515fde9dec1b9fd9525856"
    ]
  },
  "history": [
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "/bin/sh -c #(nop) ADD file:af0f4e41d68b67ca88a1ce6297326159e18e27670d7bfc0bf5804a4e2b268cc8 in / "
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"bash\"]",
      "empty_layer": true
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "LABEL maintainer=NGINX Docker Maintainers \u003cdocker-maint@nginx.com\u003e",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "ENV NGINX_VERSION=1.25.3",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "ENV NJS_VERSION=0.8.2",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "ENV PKG_RELEASE=1~bookworm",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "RUN /bin/sh -c set -x     \u0026\u0026 groupadd --system --gid 101 nginx     \u0026\u0026 useradd --system --gid nginx --no-create-home --home /nonexistent --comment \"nginx user\" --shell /bin/false --uid 101 nginx     \u0026\u0026 apt-get update     \u0026\u0026 apt-get install --no-install-recommends --no-install-suggests -y gnupg1 ca-certificates     \u0026\u0026     NGINX_GPGKEY=573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62;     NGINX_GPGKEY_PATH=/usr/share/keyrings/nginx-archive-keyring.gpg;     export GNUPGHOME=\"$(mktemp -d)\";     found='';     for server in         hkp://keyserver.ubuntu.com:80         pgp.mit.edu     ; do         echo \"Fetching GPG key $NGINX_GPGKEY from $server\";         gpg1 --keyserver \"$server\" --keyserver-options timeout=10 --recv-keys \"$NGINX_GPGKEY\" \u0026\u0026 found=yes \u0026\u0026 break;     done;     test -z \"$found\" \u0026\u0026 echo \u003e\u00262 \"error: failed to fetch GPG key $NGINX_GPGKEY\" \u0026\u0026 exit 1;     gpg1 --export \"$NGINX_GPGKEY\" \u003e \"$NGINX_GPGKEY_PATH\" ;     rm -rf \"$GNUPGHOME\";     apt-get remove --purge --auto-remove -y gnupg1 \u0026\u0026 rm -rf /var/lib/apt/lists/*     \u0026\u0026 dpkgArch=\"$(dpkg --print-architecture)\"     \u0026\u0026 nginxPackages=\"         nginx=${NGINX_VERSION}-${PKG_RELEASE}         nginx-module-xslt=${NGINX_VERSION}-${PKG_RELEASE}         nginx-module-geoip=${NGINX_VERSION}-${PKG_RELEASE}         nginx-module-image-filter=${NGINX_VERSION}-${PKG_RELEASE}         nginx-module-njs=${NGINX_VERSION}+${NJS_VERSION}-${PKG_RELEASE}     \"     \u0026\u0026 case \"$dpkgArch\" in         amd64|arm64)             echo \"deb [signed-by=$NGINX_GPGKEY_PATH] https://nginx.org/packages/mainline/debian/ bookworm nginx\" \u003e\u003e /etc/apt/sources.list.d/nginx.list             \u0026\u0026 apt-get update             ;;         *)             echo \"deb-src [signed-by=$NGINX_GPGKEY_PATH] https://nginx.org/packages/mainline/debian/ bookworm nginx\" \u003e\u003e /etc/apt/sources.list.d/nginx.list                         \u0026\u0026 tempDir=\"$(mktemp -d)\"             \u0026\u0026 chmod 777 \"$tempDir\"   \u0026\u0026 savedAptMark=\"$(apt-mark showmanual)\"                         \u0026\u0026 apt-get update             \u0026\u0026 apt-get build-dep -y $nginxPackages             \u0026\u0026 (                 cd \"$tempDir\"                 \u0026\u0026 DEB_BUILD_OPTIONS=\"nocheck parallel=$(nproc)\"                     apt-get source --compile $nginxPackages             )                         \u0026\u0026 apt-mark showmanual | xargs apt-mark auto \u003e /dev/null             \u0026\u0026 { [ -z \"$savedAptMark\" ] || apt-mark manual $savedAptMark; }                         \u0026\u0026 ls -lAFh \"$tempDir\"             \u0026\u0026 ( cd \"$tempDir\" \u0026\u0026 dpkg-scanpackages . \u003e Packages )             \u0026\u0026 grep '^Package: ' \"$tempDir/Packages\"             \u0026\u0026 echo \"deb [ trusted=yes ] file://$tempDir ./\" \u003e /etc/apt/sources.list.d/temp.list             \u0026\u0026 apt-get -o Acquire::GzipIndexes=false update             ;;     esac         \u0026\u0026 apt-get install --no-install-recommends --no-install-suggests -y                         $nginxPackages                         gettext-base                         curl     \u0026\u0026 apt-get remove --purge --auto-remove -y \u0026\u0026 rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/nginx.list         \u0026\u0026 if [ -n \"$tempDir\" ]; then         apt-get purge -y --auto-remove         \u0026\u0026 rm -rf \"$tempDir\" /etc/apt/sources.list.d/temp.list;     fi     \u0026\u0026 ln -sf /dev/stdout /var/log/nginx/access.log     \u0026\u0026 ln -sf /dev/stderr /var/log/nginx/error.log     \u0026\u0026 mkdir /docker-entrypoint.d # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "COPY docker-entrypoint.sh / # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "COPY 10-listen-on-ipv6-by-default.sh /docker-entrypoint.d # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "COPY 15-local-resolvers.envsh /docker-entrypoint.d # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "COPY 20-envsubst-on-templates.sh /docker-entrypoint.d # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "COPY 30-tune-worker-processes.sh /docker-entrypoint.d # buildkit",
      "comment": "buildkit.dockerfile.v0"
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "ENTRYPOINT [\"/docker-entrypoint.sh\"]",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "EXPOSE map[80/tcp:{}]",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "STOPSIGNAL SIGQUIT",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    },
    {
      "created": "2023-10-24T22:44:45Z",
      "created_by": "CMD [\"nginx\" \"-g\" \"daemon off;\"]",
      "comment": "buildkit.dockerfile.v0",
      "empty_layer": true
    }
  ]
}

@cah-michael-mcghee
Copy link
Author

I some what understand what you are say about the base being updated....in the case of nginx:stable date created shows 9 months ago, have validated the debian:bullseye-slim is the latest digest, but if I look at NJS version it shows 0.7.12 but the latest stable/debian/Dockerfile source show 0.8.0.

This make things very confusing when trying to make sure one has the latest version.

@thresheek
Copy link
Collaborator

Yeah, there's a discrepancy between what can be in the repository and what is there in the image. This happens because we pin specific commit hashes to build the images. See https://github.com/nginxinc/docker-nginx?tab=readme-ov-file#see-a-change-merged-here-that-doesnt-show-up-on-docker-hub-yet for more details of the workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants