Can't create annotations with `docker buildx imagetools create`

[stevehipwell]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

When running the docker buildx imagetools create command the index is created correctly but the annotations are ignored.

It's also worth noting that the documented pattern for annotations (--annotation "index:org.opencontainers.image.authors=dvdksn") errors as the " need to only be around the value side of the = (so should be --annotation index:org.opencontainers.image.authors="dvdksn").

Expected behaviour

I expect that I can create annotations on the index when using docker buildx imagetools create.

Actual behaviour

The command succeeds but I have no annotations.

Buildx version

github.com/docker/buildx v0.21.1 7c2359c

Docker info

/usr/bin/docker version
  Client: Docker Engine - Community
   Version:           26.1.3
   API version:       1.45
   Go version:        go1.21.10
   Git commit:        b72abbb
   Built:             Thu May 16 08:33:35 2024
   OS/Arch:           linux/amd64
   Context:           default
  
  Server: Docker Engine - Community
   Engine:
    Version:          26.1.3
    API version:      1.45 (minimum version 1.24)
    Go version:       go1.21.10
    Git commit:       8e96db1
    Built:            Thu May 16 08:33:35 2024
    OS/Arch:          linux/amd64
    Experimental:     false
   containerd:
    Version:          1.7.25
    GitCommit:        bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
   runc:
    Version:          1.2.4
    GitCommit:        v1.2.4-0-g6c52b3f
   docker-init:
    Version:          0.19.0
    GitCommit:        de40ad0
  /usr/bin/docker info
  Client: Docker Engine - Community
   Version:    26.1.3
   Context:    default
   Debug Mode: false
   Plugins:
    buildx: Docker Buildx (Docker Inc.)
      Version:  v0.21.1
      Path:     /usr/libexec/docker/cli-plugins/docker-buildx
    compose: Docker Compose (Docker Inc.)
      Version:  v2.27.1
      Path:     /usr/libexec/docker/cli-plugins/docker-compose
  
  Server:
   Containers: 0
    Running: 0
    Paused: 0
    Stopped: 0
   Images: 0
   Server Version: 26.1.3
   Storage Driver: overlay2
    Backing Filesystem: extfs
    Supports d_type: true
    Using metacopy: false
    Native Overlay Diff: false
    userxattr: false
   Logging Driver: json-file
   Cgroup Driver: systemd
   Cgroup Version: 2
   Plugins:
    Volume: local
    Network: bridge host ipvlan macvlan null overlay
    Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
   Swarm: inactive
   Runtimes: io.containerd.runc.v2 runc
   Default Runtime: runc
   Init Binary: docker-init
   containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
   runc version: v1.2.4-0-g6c52b3f
   init version: de40ad0
   Security Options:
    apparmor
    seccomp
     Profile: builtin
    cgroupns
   Kernel Version: 6.8.0-1021-azure
   Operating System: Ubuntu 24.04.2 LTS
   OSType: linux
   Architecture: x86_64
   CPUs: 4
   Total Memory: 15.62GiB
   Name: fv-az1921-127
   ID: bbc378f6-28b4-4eff-b0f8-05e209eb801e
   Docker Root Dir: /var/lib/docker
   Debug Mode: false
   Username: githubactions
   Experimental: false
   Insecure Registries:
    127.0.0.0/8
   Live Restore Enabled: false

Builders list

{
    "nodes": [
      {
        "name": "builder-0e383fc4-e7ac-467b-a714-305b5b22c5990",
        "endpoint": "unix:///var/run/docker.sock",
        "status": "running",
        "buildkitd-flags": "--allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host",
        "buildkit": "v0.20.1",
        "platforms": "linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/386",
        "features": {
          "Automatically load images to the Docker Engine image store": true,
          "Cache export": true,
          "Docker exporter": true,
          "Multi-platform build": true,
          "OCI exporter": true
        },
        "labels": {
          "org.mobyproject.buildkit.worker.executor": "oci",
          "org.mobyproject.buildkit.worker.hostname": "eaec96985380",
          "org.mobyproject.buildkit.worker.network": "host",
          "org.mobyproject.buildkit.worker.oci.process-mode": "sandbox",
          "org.mobyproject.buildkit.worker.selinux.enabled": "false",
          "org.mobyproject.buildkit.worker.snapshotter": "overlayfs"
        },
        "gcPolicy": [
          {
            "all": false,
            "filter": [
              "type==source.local",
              "type==exec.cachemount",
              "type==source.git.checkout"
            ],
            "keepDuration": "48h0m0s"
          },
          {
            "all": false,
            "keepDuration": "1440h0m0s"
          },
          {
            "all": false
          },
          {
            "all": true
          }
        ]
      }
    ],
    "name": "builder-0e383fc4-e7ac-467b-a714-305b5b22c599",
    "driver": "docker-container",
    "lastActivity": "2025-03-12T11:08:22.000Z"
  }

Configuration

docker buildx imagetools create \
  --tag ghcr.io/stevehipwell/fluentd-aggregator:sha-9a9f5df \
  --annotation index:org.opencontainers.image.created="2025-03-12T10:31:27.347Z" \
  --annotation index:org.opencontainers.image.description="Fluentd aggregator OCI image based on the default Fluentd OCI image." \
  --annotation index:org.opencontainers.image.licenses="Apache-2.0" \
  --annotation index:org.opencontainers.image.revision="9a9f5df4395e0ed164155282919649bfae8d80c7" \
  --annotation index:org.opencontainers.image.source="https://github.com/stevehipwell/fluentd-aggregator" \
  --annotation index:org.opencontainers.image.title="fluentd-aggregator" \
  --annotation index:org.opencontainers.image.url="https://github.com/stevehipwell/fluentd-aggregator" \
  --annotation index:org.opencontainers.image.version="pr-410" \
  ghcr.io/stevehipwell/fluentd-aggregator@sha256:3df303a1d990c0a6ac847bc4f72b1fe78eb373014142db1d63e3a3e264ff66be \
  ghcr.io/stevehipwell/fluentd-aggregator@sha256:8eb64eb4ba96fc4de8ebd78e83436919e61ef62fdedcee3af12397467f683836

Build logs

#1 [internal] pushing ghcr.io/stevehipwell/fluentd-aggregator:sha-c3419d7
#1 0.000 pushing sha256:71c7683543a84402c4529e61b6b5cc71deef923e8b418a104d981e6d8ffac7e2 to ghcr.io/stevehipwell/fluentd-aggregator:sha-c3419d7
#1 DONE 0.6s

Additional info

No response

[crazy-max]

It's also worth noting that the documented pattern for annotations (--annotation "index:org.opencontainers.image.authors=dvdksn") errors as the " need to only be around the value side of the = (so should be --annotation index:org.opencontainers.image.authors="dvdksn").

No this is documented correctly, it should be treated as a single unit. This aligns with the conventional format of key-value flags. But your format works with cobra because it's lenient. It collects all arguments following the --annotation flag into a list, meaning both forms result in the same internal representation. But that leniency doesn't mean it's the cleaner option. So both work, but the first form is cleaner and more correct because it reduces ambiguity and aligns with broader conventions.

And btw what error do you have?

When running the docker buildx imagetools create command the index is created correctly but the annotations are ignored.

I can't repro on my side:

$ docker buildx imagetools create \
  --tag crazymax/test:buildx-3061 \
  --annotation "index:org.opencontainers.image.created=2025-03-12T10:31:27.347Z" \
  --annotation "index:org.opencontainers.image.description=Fluentd aggregator OCI image based on the default Fluentd OCI image." \
  --annotation "index:org.opencontainers.image.licenses=Apache-2.0" \
  --annotation "index:org.opencontainers.image.revision=9a9f5df4395e0ed164155282919649bfae8d80c7" \
  --annotation "index:org.opencontainers.image.source=https://github.com/stevehipwell/fluentd-aggregator" \
  --annotation "index:org.opencontainers.image.title=fluentd-aggregator" \
  --annotation "index:org.opencontainers.image.url=https://github.com/stevehipwell/fluentd-aggregator" \
  --annotation "index:org.opencontainers.image.version=pr-410" \
  crazymax/test:foo
#1 [internal] pushing docker.io/crazymax/test:buildx-3061
#1 0.000 pushing sha256:9a37a85e91c9136ba3bb55a7c17ea6224d2a1dae423123028728acffb647fc65 to docker.io/crazymax/test:buildx-3061
#1 DONE 0.4s

Annotations are set as expected: https://explore.ggcr.dev/?image=crazymax%2Ftest%3Abuildx-3061

[stevehipwell]

Thanks for the quick response @crazy-max. RE the primary issue I've noticed that my mediaType doesn't match yours, I'm getting application/vnd.docker.distribution.manifest.list.v2+json which I wasn't expecting. For the manifests I was using type=image,push-by-digest=true,name-canonical=true,push=true as my output config but even when I switch to the standard push: true on the GitHub action I see the same thing.

It's also worth noting that the documented pattern for annotations (--annotation "index:org.opencontainers.image.authors=dvdksn") errors as the " need to only be around the value side of the = (so should be --annotation index:org.opencontainers.image.authors="dvdksn").

No this is documented correctly, it should be treated as a single unit. This aligns with the conventional format of key-value flags. But your format works with cobra because it's lenient. It collects all arguments following the --annotation flag into a list, meaning both forms result in the same internal representation. But that leniency doesn't mean it's the cleaner option. So both work, but the first form is cleaner and more correct because it reduces ambiguity and aligns with broader conventions.

I get the following error when wrapping the whole annotation in quotes.

ERROR: failed to parse annotations: invalid annotation type "\"index", expected type and optional platform in square brackets

[stevehipwell]

@crazy-max I've disabled attestations, so that might allow you to reproduce?

[stevehipwell]

@crazy-max without attestations enabled I need to set oci-mediatypes=true in the output of the build push action to get OCI media types. I couldn't see an mention of this being required anywhere in the docs, so I think it's something that might want adding.