Secrets fail to set the `uid`, `gid` and `mode` specified in `docker-compose.yml`

[sanchitbapat]

Description

Docker secrets specified using the long syntax for the docker-compose.yml file fail to set the specified uid, gid and mode.

Also, from the docs, the default value of the uid and gid fields should be the user that runs the container however the value remains whatever was set on the host machine.

Steps to reproduce the issue:

1. Create a Dockerfile

❯ cat Dockerfile
FROM ubuntu:20.04

RUN adduser tester -u 1005 --disabled-password
CMD ["/bin/bash", "-c", "ls -al /run/secrets/target_secret_file.txt"]

2. Create a docker-compose.yml

❯ cat docker-compose.yml
services:
  secrets-tester:
    build: .
    secrets:
      - source: some_secret_file
        target: target_secret_file.txt
        uid: "1005"
        gid: "1005"
        mode: 0440

secrets:
  some_secret_file:
    file: somefile.txt

3. Create a text file for secrets (somefile.txt)

❯ cat somefile.txt
Text from a secret file

4. Run the service docker compose run secrets-tester

Describe the results you received:
Received Output:

❯ docker compose run secrets-tester
-rw-r--r-- 1 1000 1000 24 Jul 13 16:47 /run/secrets/target_secret_file.txt

Describe the results you expected:
Expected Output:

❯ docker compose run secrets-tester
-r--r----- 1 1005 1005 24 Jul 13 16:47 /run/secrets/target_secret_file.txt

Additional information you deem important (e.g. issue happens only occasionally):
Same behavior is observed in these cases:

1. Without creating the tester user
2. With same source and target names for the file in docker-compose.yml
3. Using the uid and gid for root
4. Using random values for uid and gid
5. Different values for mode

Output of docker compose version:

❯ docker --version
Docker version 20.10.17, build 100c701

❯ docker compose version
Docker Compose version v2.6.0

Output of docker info:

❯ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
  compose: Docker Compose (Docker Inc., v2.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 2
  Running: 0
  Paused: 0
  Stopped: 2
 Images: 1
 Server Version: 20.10.14
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.102.1-microsoft-standard-WSL2
 Operating System: Ubuntu 20.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 7.763GiB
 Name: DESKTOP-IGS6AOM
 ID: MBVL:76QX:UWDQ:AK7Z:BNGW:PYGL:EXFG:ZHGD:JM53:RBFZ:4SQV:5I7Z
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Additional environment details:

[ev-adias]

Same thing happens to me. I'm trying to use secrets to workaround having to set the permissions on the host file, by setting the mode in the secret, but it end up with the exact same permissions of the host.

This issues comes up for me when using a keyfile for MongoDB replica authentication. The keyfile must not be world-readable, and in *nix this is easy to fix, in Windows (docker-desktop) it's not, so I was hoping that using secrets might fix the problem. It seems not

[markkrj]

As bind mounts do not support different permissions (uid, gid and mode), I suspect that in order to support this, the content of such secrets must be copied into the container (as in #9553), or, use POSIX ACLs (more complex). Am I right @ndeloof?
But why does the compose documentation mention such options if they are not supported in the first place?

[rijnhard]

well secrets and configs arent writeable (as per specification) so no reason why the implementation cant actually be a copy.
unless the idea is that they can change during runtime...

[mario-gravel]

I'm using a secret for a SSH Private Key that must have a 0400 mode. Setting the uid, gid and mode is a requirement.

    secrets:
      - source: ssh_host_ecdsa_key
        target: /etc/ssh/ssh_host_ecdsa_key
        mode: 0400

2022-12-14 20:41:23 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2022-12-14 20:41:23 @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
2022-12-14 20:41:23 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2022-12-14 20:41:23 Permissions 0777 for '/etc/ssh/ssh_host_ed25519_key' are too open.
2022-12-14 20:41:23 It is required that your private key files are NOT accessible by others.
2022-12-14 20:41:23 This private key will be ignored.
2022-12-14 20:41:23 sshd: no hostkeys available -- exiting.

[ndeloof]

Same issue applies both to Compose v1 and v2
the root cause is that both use a bind mount to "inject" secret inside container, but bind mount doesn't allow a distinct UID/GID/mode to be exposed inside container.

A possible alternative I can imagine is to replace this approach with a copy (just like we do with support for secret set by environment). The sole drawback is that, doing so, changes to secret on host won't apply to secret inside container. Doesn't seem to be a major issue to me, but let me know if this would be a blocker for your usage.

[mario-gravel]

In my own opinion, having to rebuild the image to change a secret isn't an option. Major issue? Not now because we haven't reach the production, we're still developing. But once in production, it will be a security concern.

The access to the container repository isn't as secure as the access to the production host. Somebody pulling the image will have access to the private keys. Not a good thing at all !

There must be control over the uid, gid and mode for a secret file.

[ndeloof]

having to rebuild the image to change a secret isn't an option

This is not my proposal: compose would copy the secret into the container (not image) after creation, and before start. This is how secret based on environment is implemented already.

[mario-gravel]

Oh, my bad ! Sorry 'bout that !

[mario-gravel]

So if a secret change, we need to restart the container. Am I right ?

[ndeloof]

yes indeed

[mario-gravel]

Does it implies a redeploy of the container or just the starting phase will be enough ?

It would be great if a refresh can be triggered manually (ex: docker compose refresh-secrets) to redo the copy in the container. Having a watchdog to do this automatically may consume resources unnecessary. Secrets don't change often but being able to update them without a restart can be a good solution.

[ndeloof]

we could copy secret again at any time, or have a dedicated command for this purpose, but the more obvious one would be to trigger a container restart