error when merging multi-platform manifest(s) to custom tls config registry after successfully pushing for individual platforms

### Contributing guidelines

- [X] I've read the [contributing guidelines](https://github.com/docker/buildx/blob/master/.github/CONTRIBUTING.md) and wholeheartedly agree

### I've found a bug and checked that ...

- [X] ... the documentation does not mention anything about my problem
- [X] ... there are no open or closed issues that are related to my problem

### Description

It looks as if the `imagetools.Opt` passed to the [`itpull := imagetools.New(imageopt)` line](https://github.com/docker/buildx/blob/f8c6a97edce937c9b6576fb04408e61f787bea20/build/build.go#L595) is lacking the necessary `RegistryConfig` to connect to a private registry signed by a CA that isn't included in the system ca-certificates BUT that individual builders are able to push to without issue (meaning, they are configured properly ... the build pulls from the private build cache registry successfully, honoring the private registry cache importer).

### Expected behaviour

Can successfully merge multi-platform manifests for blobs that have already been pushed to a private registry.

### Actual behaviour

**_Cannot_** successfully merge multi-platform manifests for blobs that have already been pushed to a private registry.

### Buildx version

github.com/docker/buildx v0.13.1+dweomer.1 5decc6f3cbed2286772783c6e89faf7d3b3bb985

### Docker info

```text
Client:
   Version:           25.0.4
   API version:       1.44
   Go version:        go1.21.8
   Git commit:        1a576c5
   Built:             Wed Mar  6 16:32:02 2024
   OS/Arch:           linux/amd64
   Context:           default
  
  Server: Docker Engine - Community
   Engine:
    Version:          26.1.3
    API version:      1.45 (minimum version 1.24)
    Go version:       go1.21.10
    Git commit:       8e96db1
    Built:            Thu May 16 08:33:58 2024
    OS/Arch:          linux/amd64
    Experimental:     false
   containerd:
    Version:          v1.7.15
    GitCommit:        926c9586fe4a6236699318391cd44976a98e31f1
   runc:
    Version:          1.1.12
    GitCommit:        v1.1.12-0-g51d5e94
   docker-init:
    Version:          0.19.0
    GitCommit:        de40ad0
  /usr/bin/docker info
  Client:
   Version:    25.0.4
   Context:    default
   Debug Mode: false
   Plugins:
    buildx: Docker Buildx (Docker Inc.)
      Version:  v0.13.1+dweomer.1
      Path:     /home/runner/.docker/cli-plugins/docker-buildx
  
  Server:
   Containers: 0
    Running: 0
    Paused: 0
    Stopped: 0
   Images: 1
   Server Version: 26.1.3
   Storage Driver: overlay2
    Backing Filesystem: xfs
    Supports d_type: true
    Using metacopy: false
    Native Overlay Diff: true
    userxattr: false
   Logging Driver: json-file
   Cgroup Driver: cgroupfs
   Cgroup Version: 1
   Plugins:
    Volume: local
    Network: bridge host ipvlan macvlan null overlay
    Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
   Swarm: inactive
   Runtimes: io.containerd.runc.v2 runc
   Default Runtime: runc
   Init Binary: docker-init
   containerd version: 926c9586fe4a6236699318391cd44976a98e31f1
   runc version: v1.1.12-0-g51d5e94
   init version: de40ad0
   Security Options:
    seccomp
     Profile: builtin
   Kernel Version: 5.10.130-118.517.amzn2.x86_64
   Operating System: Alpine Linux v3.19 (containerized)
   OSType: linux
   Architecture: x86_64
   CPUs: 8
   Total Memory: 30.9GiB
   Name: ip-10-10-11-198.us-gov-east-1.compute.internal
   ID: 941f2083-c5f9-4f79-8d28-fb49661dfb6c
   Docker Root Dir: /var/lib/docker
   Debug Mode: false
   Experimental: false
   Insecure Registries:
    127.0.0.0/8
   Registry Mirrors:
    https://dhub.cache.svc/
   Live Restore Enabled: false
   Product License: Community Engine
```


### Builders list

```text
Name:          builder-0be5ebbb-1707-47bc-bd90-13edc91cfb1e
  Driver:        kubernetes
  Last Activity: 2024-05-25 13:46:09 +0000 UTC
  Nodes:
  Name:                  builder-0be5ebbb-1707-47bc-bd90-13edc91cfb1e0
  Endpoint:              kubernetes:///builder-0be5ebbb-1707-47bc-bd90-13edc91cfb1e?deployment=buildkit-495d6f30-f49f-491d-a811-0cf9049bccc6-8tfds&kubeconfig=
  Driver Options:        nodeselector="category=build" tolerations="key=category,value=build"
  Status:                running
  BuildKit daemon flags: --allow-insecure-entitlement=network.host
  BuildKit version:      v0.13.2
  Platforms:             linux/amd64*, linux/amd64/v2*, linux/amd64/v3*, linux/amd64/v4*, linux/386*, linux/arm64, linux/riscv64, linux/ppc64, linux/ppc64le, linux/s390x, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
  Labels:
   org.mobyproject.buildkit.worker.executor:         oci
   org.mobyproject.buildkit.worker.hostname:         builder-0be5ebbb-1707-47bc-bd90-13edc91cfb1e0-6df88cfdc6-88rrb
   org.mobyproject.buildkit.worker.network:          host
   org.mobyproject.buildkit.worker.oci.process-mode: sandbox
   org.mobyproject.buildkit.worker.selinux.enabled:  false
   org.mobyproject.buildkit.worker.snapshotter:      overlayfs
  GC Policy rule#0:
   All:           false
   Filters:       type==source.local,type==exec.cachemount,type==source.git.checkout
   Keep Duration: 48h0m0s
   Keep Bytes:    488.3MiB
  GC Policy rule#1:
   All:           false
   Keep Duration: 1440h0m0s
   Keep Bytes:    46.57GiB
  GC Policy rule#2:
   All:        false
   Keep Bytes: 46.57GiB
  GC Policy rule#3:
   All:        true
   Keep Bytes: 46.57GiB
  Name:                  builder-0be5ebbb-1707-47bc-bd90-13edc91cfb1e1
  Endpoint:              kubernetes:///builder-0be5ebbb-1707-47bc-bd90-13edc91cfb1e?deployment=buildkit-9e169c6c-af0d-46c4-9ad3-1589f6a15580-5lp28&kubeconfig=
  Driver Options:        nodeselector="category=build-arm64" tolerations="key=category,value=build-arm64"
  Status:                running
  BuildKit daemon flags: --allow-insecure-entitlement=network.host
  BuildKit version:      v0.13.2
  Platforms:             linux/arm/v6*, linux/arm/v7*, linux/arm64*
  Labels:
   org.mobyproject.buildkit.worker.executor:         oci
   org.mobyproject.buildkit.worker.hostname:         builder-0be5ebbb-1707-47bc-bd90-13edc91cfb1e1-cb9c654df-kxqgr
   org.mobyproject.buildkit.worker.network:          host
   org.mobyproject.buildkit.worker.oci.process-mode: sandbox
   org.mobyproject.buildkit.worker.selinux.enabled:  false
   org.mobyproject.buildkit.worker.snapshotter:      overlayfs
  GC Policy rule#0:
   All:           false
   Filters:       type==source.local,type==exec.cachemount,type==source.git.checkout
   Keep Duration: 48h0m0s
   Keep Bytes:    488.3MiB
  GC Policy rule#1:
   All:           false
   Keep Duration: 1440h0m0s
   Keep Bytes:    46.57GiB
  GC Policy rule#2:
   All:        false
   Keep Bytes: 46.57GiB
  GC Policy rule#3:
   All:        true
   Keep Bytes: 46.57GiB
```


### Configuration

```dockerfile
FROM library/alpine:edge
RUN echo 'unable to share this but the same dockerfile merges just fine to ghcr.io'
```

### Build logs

```text
#26 exporting to image
#26 ...
#27 exporting to image
#27 exporting layers
#27 ...
#26 exporting to image
#26 exporting layers 65.5s done
#26 exporting manifest sha256:d9be2cdb45c5b07b54691e153ad5b6b4c8d527356500323f9fea81df300876c5 done
#26 exporting config sha256:17675bb9b8dc515066bc0f326b2d548dfe1232579f588a1cbdbf5c45a7f726cd done
#26 exporting attestation manifest sha256:8c81834da520a243b0fd108537c970d94c4bee5e270a6bfcd74b9c8a38854e5f 0.0s done
#26 exporting manifest list sha256:dc6732014a6873697cadcc9531c31b7504193fbecac1411ea3491d8118152bcb done
#26 pushing layers
#26 pushing layers 3.8s done
#26 pushing manifest for build.cache.svc/my-project/my-image
#26 pushing manifest for build.cache.svc/my-project/my-image 0.0s done
#26 DONE 69.4s
#28 exporting cache to registry
#28 preparing build cache for export
#28 writing layer sha256:08b1720df82a0beee132289941ac9ee2eba74a7d2ad637c1a8352366d751fb25 done
#28 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 done
#28 writing layer sha256:561cb69653d56a9725be56e02128e4e96fb434a8b4b4decf2bdeb479a225feaf done
#28 writing layer sha256:8f665685b215c7daf9164545f1bbdd74d800af77d0d267db31fe0345c0c8fb8b done
#28 writing layer sha256:9361d72813976e1175ddb2fbce2e5f0ab01e71a419990d64e71bc36946edd884 done
#28 writing layer sha256:96ad531c39c935bc6319f19f3be8f9f4a6faa15ded833ad2bd50a95a0d95e8d2 done
#28 writing layer sha256:e5fca6c395a62ec277102af9e5283f6edb43b3e4f20f798e3ce7e425be226ba6 done
#28 writing layer sha256:f56be85fc22e46face30e2c3de3f7fe7c15f8fd7c4e5add29d7f64b87abdaa09 done
#28 writing layer sha256:fc07f0dda8ec1c1acc98ab6a4673371611db7184cff56ddef0eba11523eec347 done
#28 writing config sha256:5bf508bda394326c3229d6ad06bcb6bded9357713a60be9a5056503b68adbadf 0.0s done
#28 writing cache manifest sha256:e681f494875749ceb3083097acf67ee72e298cc01dce5ab63e5f856b65cbf12c
#28 preparing build cache for export 0.1s done
#28 writing cache manifest sha256:e681f494875749ceb3083097acf67ee72e298cc01dce5ab63e5f856b65cbf12c 0.0s done
#28 DONE 0.1s
#27 exporting to image
#27 exporting layers 68.0s done
#27 exporting manifest sha256:c3092a12a16f9d5411701e95592b1f0d0d64b24ff810727cf911128403848f11 done
#27 exporting config sha256:459fb84f04c080c7a977c605b777e903e4a135002b442ede68aed725320f5880 done
#27 exporting attestation manifest sha256:c13a9929f49c119f9dccbeeeb763a84548efa08dea86a7546ad9a128dbc5e9c5 0.0s done
#27 exporting manifest list sha256:fc95710499e7bb88684294ec76c284fa9c73444468793654a5c68ecb3b059397 done
#27 pushing layers
#27 pushing layers 3.7s done
#27 pushing manifest for build.cache.svc/my-project/my-image
#27 pushing manifest for build.cache.svc/my-project/my-image 0.0s done
#27 DONE 71.7s
#29 exporting cache to registry
#29 preparing build cache for export
#29 writing layer sha256:4cf6a83c0e2af3c780abcda02cc33f9e812fdcb40b610ed1838281cc9ab94ec8 done
#29 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 done
#29 writing layer sha256:5a63f40ac9bbdfab87854860e46116e14c81556e0d159437bcbd13ec83848687
#29 preparing build cache for export 0.1s done
#29 writing layer sha256:5a63f40ac9bbdfab87854860e46116e14c81556e0d159437bcbd13ec83848687 done
#29 writing layer sha256:683339ce8d6b9be2ca150a8de67b895e20ea5594b91d3911c95b0b8fea3e314c done
#29 writing layer sha256:686172e40c38722891b4004f55f6447548c8367968ac523a612591e0d92f9db3 done
#29 writing layer sha256:c41833b44d910632b415cd89a9cdaa4d62c9725dc56c99a7ddadafd6719960f9 done
#29 writing layer sha256:e83c0d77c542c0ae16eda4f948bdc6e84b0a82b8a00068b7eeb5a5a743b1b453 done
#29 writing layer sha256:ed43d91b02ce995d68736bc3af861c28500f6109fcb8d62179c71ffa023ce97a done
#29 writing layer sha256:fc1eefa94020698f74056fc3449798c2319f23cb42221d278064fa8f8ea616c0 done
#29 writing config sha256:95ee56b834bf8aa0dde7ef40d4fe16146f00da17d3c14ca69fabb7aafe8f9e87 0.0s done
#29 writing cache manifest sha256:777b29ca996df891e166c85a82232c6da4b94c19470a3d5ca32c0641144ede04 0.0s done
#29 DONE 0.1s
#30 merging manifest list build.cache.svc/my-project/my-image:my-tag,build.cache.svc/my-project/my-image:sha-cc220b522f58843a818603b89cf6195fd4b30643,build.cache.svc/my-project/my-image:latest
#30 ERROR: httpReadSeeker: failed open: failed to do request: Get "https://build.cache.svc/v2/my-project/my-image/manifests/sha256:fc95710499e7bb88684294ec76c284fa9c73444468793654a5c68ecb3b059397": tls: failed to verify certificate: x509: certificate signed by unknown authority
------
 > merging manifest list build.cache.svc/my-project/my-image:my-tag,build.cache.svc/my-project/my-image:sha-cc220b522f58843a818603b89cf6195fd4b30643,build.cache.svc/my-project/my-image:latest:
------
```


### Additional info

This is driven via github actions on a private runner, leveraging:
- https://github.com/marketplace/actions/docker-metadata-action
- https://github.com/marketplace/actions/docker-setup-buildx
- https://github.com/marketplace/actions/build-and-push-docker-images
- https://github.com/dweomer/buildx/releases/tag/v0.13.1%2Bdweomer.1

See also:
- #2462 

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

error when merging multi-platform manifest(s) to custom tls config registry after successfully pushing for individual platforms #2476

dweomer
openedon May 25, 2024

Contributing guidelines

I've found a bug and checked that ...

Description

Expected behaviour

Actual behaviour

Buildx version

Docker info

Builders list

Configuration

Build logs

Additional info

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

error when merging multi-platform manifest(s) to custom tls config registry after successfully pushing for individual platforms #2476

Description

dweomeropenedon May 25, 2024

Contributing guidelines

I've found a bug and checked that ...

Description

Expected behaviour

Actual behaviour

Buildx version

Docker info

Builders list

Configuration

Build logs

Additional info

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

dweomer
openedon May 25, 2024