apparmor failed to apply profile: write /proc/self/attr/exec: invalid argument: unknown.

[alenpaulvarghese]

• This is a bug report
• This is a feature request
• I searched existing issues before opening this one

Expected behavior

docker run should be able to run the container

Actual behavior

Running docker run hello-world gives this error

docker: Error response from daemon: OCI runtime create failed: container_linux.go:367: starting container process caused: process_linux.go:495: container init caused: apply apparmor profile: apparmor failed to apply profile: write /proc/self/attr/exec: invalid argument: unknown.
ERRO[0001] error waiting for container: context canceled 

I Installed docker using pacman package manager in manjaro.

Output of docker version :

Client:
 Version:           20.10.3
 API version:       1.41
 Go version:        go1.15.7
 Git commit:        48d30b5b32
 Built:             Tue Feb  2 02:34:18 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.3
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.15.7
  Git commit:       46229ca1d8
  Built:            Tue Feb  2 02:33:45 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b.m
 runc:
  Version:          1.0.0-rc93
  GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info :

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-tp-docker)

Server:
 Containers: 7
  Running: 0
  Paused: 0
  Stopped: 7
 Images: 1
 Server Version: 20.10.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 269548fa27e0089a8b8278fc4fc781d7f65a939b.m
 runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.10.13-1-MANJARO
 Operating System: Manjaro Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 7.628GiB
 Name: Asus-Rog
 ID: SLHK:5R5I:BVZW:CV7V:KJZF:EXXR:ZTUV:UCLJ:AIJU:NPK2:RPGO:DHGH
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio weight support
WARNING: No blkio weight_device support

[jeduden]

Same issue with Arch Linux. Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-tp-docker)

Server:
 Containers: 2
  Running: 0
  Paused: 0
  Stopped: 2
 Images: 2
 Server Version: 20.10.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 269548fa27e0089a8b8278fc4fc781d7f65a939b.m
 runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.10.13-arch1-1
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 15.25GiB
 Name: spectre
 ID: QR6D:UJ3C:ZIKT:QD44:ZIKW:AEKW:INEM:CEG6:JV7X:XAE7:NPL3:FP76
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio weight support
WARNING: No blkio weight_device support

[idoqo]

What fixed it for me was adding apparmor=1 lsm=lockdown,yama,apparmor,bpf to my kernel parameters. (ArchLinux guide here)

Based on these comments, It seems there were changes to the Linux Security Module that affected how Apparmor is applied from kernel version 5.10.13.

[alenpaulvarghese]

@idoqo So this issue should be resolved by docker?

[jeduden]

@idoqo So this issue should be resolved by docker?

The issue is resolved by changing apparmor related configuration as idoqo pointed out.

I guess the only action for docker is to add a reference to this issue and fix somewhere in the docs.

[alenpaulvarghese]

@jeduden Ok thank you :)

[XeR]

linux.git/Documentation/admin-guide/LSM/index.rst mentions the following:

Process attributes associated with "major" security modules should
be accessed and maintained using the special files in /proc/.../attr.
A security module may maintain a module specific subdirectory there,
named after the module. /proc/.../attr/smack is provided by the Smack
security module and contains all its special files. The files directly
in /proc/.../attr remain as legacy interfaces for modules that provide
subdirectories.

If my understanding is correct, /proc/self/attr/ is the "legacy interface".
When AppArmor is configured as the major security module, it contains the content of /proc/self/attr/apparmor, in particular the exec file.
The change to the kernel's command line sets the major security module to apparmor.

% ls -l /proc/1/attr/apparmor /proc/1/attr
/proc/1/attr:
total 0
dr-xr-xr-x 2 root root 0 Jan  1 00:00 apparmor
-rw-rw-rw- 1 root root 0 Jan  1 00:00 current
-rw-rw-rw- 1 root root 0 Jan  1 00:00 exec
-rw-rw-rw- 1 root root 0 Jan  1 00:00 fscreate
-rw-rw-rw- 1 root root 0 Jan  1 00:00 keycreate
-r--r--r-- 1 root root 0 Jan  1 00:00 prev
dr-xr-xr-x 2 root root 0 Jan  1 00:00 smack
-rw-rw-rw- 1 root root 0 Jan  1 00:00 sockcreate

/proc/1/attr/apparmor:
total 0
-rw-rw-rw- 1 root root 0 Jan  1 00:00 current
-rw-rw-rw- 1 root root 0 Jan  1 00:00 exec
-r--r--r-- 1 root root 0 Jan  1 00:00 prev

I guess the only action for docker is to add a reference to this issue and fix somewhere in the docs.

I think that Docker should use /proc/self/attr/apparmor/exec instead of /proc/self/attr/exec.
What if there is an other program that expects the major security module to be an other LSM?

[AkihiroSuda]

The corresponding code is in runc, not in Docker. So further discussion should happen in runc repo opencontainers/runc#2801

PR is here: opencontainers/runc#2803

[cyphar]

While we can work around this in runc, I would suggest this be reported to the Arch Linux kernel maintainers because it's a regression in their packaging (they broke another package by changing the kernel configuration). Yeah we should write to /proc/self/attr/apparmor/... if it exists, but the legacy interfaces shouldn't stop working either.