don't relabel mqueue with user ns and make selinux and user ns can be used at the same time

[keloyang]

after mount mqueue with user namespace,we will have a view like the following:

root@6996292ba929:/# ls /dev/ -Zn|grep mqueue
drwxrwxrwt. 2 65534 65534 unconfined_u:object_r:user_tmpfs_t:s0

we can see the uid is 65534,this make sys_lsetxattr fail when kernel has a kuid_has_mapping check,so that we can't start a container with selinux and user ns at the same time.
proc and sys also have 65534 uid after mount ,and they are not relabeled.So we can remove the relabel for mqueue,If so, selinux and user ns can be used at the same time.

Signed-off-by: yangshukui yangshukui@huawei.com

[mrunalp]

@rhatdan PTAL

[rhatdan]

What is the error that is blocking the relabel. Relabel should not be governed via DAC permissions.

[rhatdan]

@runcom Have you seen this problem?

[keloyang]

docker daemon --selinux-enabled --userns-remap=1000

After mount,mqueue、proc、sysfs dir ’s uid and gid are 65534,I don't know whether it is ok,but relabel failed in kuid_has_mapping.Can mqueue、proc、sysfs be the same and not be relabeled ?

[rhatdan]

/proc ad sysfs are fine, but mqueue has a bad label by default. If you turn off labeling just for mqueue, what label does it get? tmpfs_t?

Inside the container

ls -lZd /dev/mueue

kuid_has_mapping? Is this a kernel call?

[keloyang]

mqueue get a user_tmpfs_t label by default, why mqueue has a bad label ,can we modify the default label for mqueue?
sys_lsetxattr eventually use kuid_has_mapping:
sys_lsetxattr-->path_setxattr-->setxattr-->vfs_setxattr-->security_inode_setxattr-->call_int_hook-->selinux_inode_setxattr-->inode_owner_or_capable-->kuid_has_mapping

[rhatdan]

And why does kuid_has_mapping block?

[keloyang]

after mount , /dev/mqueue' s uid is 65534 in user ns, not 0 which the general file(e.g. /home) get,I think it's the reason. So if docker relabel proc(sysfs), it will fail as well.

[rhatdan]

Right that is the bug if /dev/mqueue is not followin user namespace, its UID should not be -1 it should be the dockerroot uid inside of the container. Docker is setting the SELinux label, and I believe docker is running as UID=0, so it not being able to set the label is curious. If you disable SELinux inside the container, can the UID=dockerroot inside of the container create content in /dev/mqueu?

[keloyang]

@rhatdan ,thanks. do you means dockerroot is the root inside of the container ? use user ns and disable SELinux, the root inside of the container can't create content in /dev/mqueu.

[rhatdan]

Exactly. The problem here, I believe is in the kernel, or docker in that /dev/mqueue is not being owned by dockerroot, docker should be changing the ownership of the /dev/mqueue to dockerroot, and changing the SELinux label. If this happens after usernamespace has been changed then the processes will not be allowed to write to /dev/mqueueu.

[runcom]

@runcom Have you seen this problem?

Never seen this, also I don't understand what the error is 😕 The only thing I remember with selinux and userns is this BZ https://bugzilla.redhat.com/show_bug.cgi?id=1312665 which is said to be fixed upstream here in runc

[mrunalp]

I'll look into this on Monday

Sent from my iPhone

On Apr 16, 2016, at 8:32 AM, Antonio Murdaca notifications@github.com wrote:

@runcom Have you seen this problem?

Never seen this, also I don't understand what the error is The only thing I remember with selinux and userns is this BZ https://bugzilla.redhat.com/show_bug.cgi?id=1312665 which is said to be fixed upstream here in runc

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

[mrunalp]

I tried to reproduce this on docker master and looks like it is broken with selinux on Fedora 23.

[root@localhost ~]# /root/gosrc/src/github.com/docker/docker/bundles/latest/dynbinary/docker run -it --rm fedora bash
bash: /dev/null: Permission denied
bash: /dev/null: Permission denied
bash: /dev/null: Permission denied
bash: /dev/null: Permission denied
bash: /dev/null: Permission denied
bash: /dev/null: Permission denied
bash: /dev/null: Permission denied
bash: /dev/null: Permission denied
bash: /dev/null: Permission denied
bash: /dev/null: Permission denied
[root@6148bf60a004 /]# ls -l /dev

[mrunalp]

@rhatdan Does #725 fix that?

[rhatdan]

No not related.

What does ausearch -m avc -ts recent show?

[mrunalp]

time->Mon Apr 18 20:04:25 2016
type=AVC msg=audit(1461024265.812:429): avc:  denied  { write } for  pid=5306 comm="bash" name="null" dev="tmpfs" ino=37548 scontext=system_u:system_r:svirt_lxc_net_t:s0:c201,c251 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=chr_file permissive=0
----
time->Mon Apr 18 20:04:25 2016
type=AVC msg=audit(1461024265.812:430): avc:  denied  { write } for  pid=5306 comm="bash" name="null" dev="tmpfs" ino=37548 scontext=system_u:system_r:svirt_lxc_net_t:s0:c201,c251 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=chr_file permissive=0
----
time->Mon Apr 18 20:04:25 2016
type=AVC msg=audit(1461024265.812:431): avc:  denied  { write } for  pid=5306 comm="bash" name="null" dev="tmpfs" ino=37548 scontext=system_u:system_r:svirt_lxc_net_t:s0:c201,c251 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=chr_file permissive=0
----
time->Mon Apr 18 20:04:25 2016
type=AVC msg=audit(1461024265.812:432): avc:  denied  { write } for  pid=5306 comm="bash" name="null" dev="tmpfs" ino=37548 scontext=system_u:system_r:svirt_lxc_net_t:s0:c201,c251 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=chr_file permissive=0

[rhatdan]

Did you set the mountlabel?

[mrunalp]

Looks like the rootfs is labeled correctly

[root@localhost rootfs]# ls -lZa
total 68
drwxr-xr-x. 18 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Apr 18 20:04 .
drwx------.  3 root root unconfined_u:object_r:var_run_t:s0                   140 Apr 18 20:04 ..
lrwxrwxrwx.  1 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251    7 Sep 10  2015 bin -> usr/bin
dr-xr-xr-x.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Sep 10  2015 boot
drwxr-xr-x.  4 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Apr 18 20:04 dev
-rwxr-xr-x.  1 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251    0 Apr 18 20:04 .dockerenv
drwxr-xr-x. 47 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Apr 18 20:04 etc
drwxr-xr-x.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Mar  4 00:19 home
lrwxrwxrwx.  1 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251    7 Sep 10  2015 lib -> usr/lib
lrwxrwxrwx.  1 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251    9 Sep 10  2015 lib64 -> usr/lib64
drwx------.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Mar  4 00:17 lost+found
drwxr-xr-x.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Sep 10  2015 media
drwxr-xr-x.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Sep 10  2015 mnt
drwxr-xr-x.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Sep 10  2015 opt
drwxr-xr-x.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Mar  4 00:17 proc
dr-xr-x---.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Mar  4 00:19 root
drwxr-xr-x.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Mar  4 00:17 run
lrwxrwxrwx.  1 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251    8 Sep 10  2015 sbin -> usr/sbin
drwxr-xr-x.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Sep 10  2015 srv
drwxr-xr-x.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Mar  4 00:17 sys
drwxrwxrwt.  7 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Mar  4 00:19 tmp
drwxr-xr-x. 12 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Mar  4 00:17 usr
drwxr-xr-x. 18 root root system_u:object_r:svirt_sandbox_file_t:s0:c201,c251 4096 Mar  4 00:18 var

[rhatdan]

But what does /dev inside the container look like?

[mrunalp]

@rhatdan I can't access /dev from inside the container.

# From inside:
[root@9dd410a563bb /]# ls -ltrZ dev/
ls: cannot open directory dev/: Permission denied
[root@9dd410a563bb /]# ls -l /dev/
ls: cannot open directory /dev/: Permission denied

# From outside:
[root@localhost rootfs]# ls -lZa dev/
total 0
drwxr-xr-x.  4 root root system_u:object_r:svirt_sandbox_file_t:s0:c126,c739  43 Apr 18 20:33 .
drwxr-xr-x. 18 root root system_u:object_r:svirt_sandbox_file_t:s0:c126,c739 260 Apr 18 20:33 ..
-rwxr-xr-x.  1 root root system_u:object_r:svirt_sandbox_file_t:s0:c126,c739   0 Apr 18 20:33 console
drwxr-xr-x.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c126,c739   6 Apr 18 20:33 pts
drwxr-xr-x.  2 root root system_u:object_r:svirt_sandbox_file_t:s0:c126,c739   6 Apr 18 20:33 shm

[rhatdan]

Do you have the mountlabel defined in the containers json? If you executed setenforce 0 on the host you would be able to see the /dev. And it would not have the correct labels. With runc, you need to specify the mountlabel, so that it will setup any internal tmpfs based mount points with the correct labels. The label on the image rootfs has to be set outside of runc.

[mrunalp]

@rhatdan This is with docker not runc hence it seems like a regression to me.
I set setenforce to 0 and got the labels from /dev/ inside the container.

total 0
crw-------. 1 root root unconfined_u:object_r:user_devpts_t:s0              136,   3 Apr 19  2016 console
lrwxrwxrwx. 1 root root unconfined_u:object_r:user_tmp_t:s0                       11 Apr 19 03:33 core -> /proc/kcore
lrwxrwxrwx. 1 root root unconfined_u:object_r:user_tmp_t:s0                       13 Apr 19 03:33 fd -> /proc/self/fd
crw-rw-rw-. 1 root root unconfined_u:object_r:user_tmp_t:s0                   1,   7 Apr 19 03:33 full
crw-rw-rw-. 1 root root unconfined_u:object_r:user_tmp_t:s0                  10, 229 Apr 19 03:33 fuse
drwxrwxrwt. 2 root root unconfined_u:object_r:user_tmp_t:s0                       40 Apr 19 03:33 mqueue
crw-rw-rw-. 1 root root unconfined_u:object_r:user_tmp_t:s0                   1,   3 Apr 19 03:33 null
lrwxrwxrwx. 1 root root unconfined_u:object_r:user_tmp_t:s0                        8 Apr 19 03:33 ptmx -> pts/ptmx
drwxr-xr-x. 2 root root unconfined_u:object_r:devpts_t:s0                          0 Apr 19 03:33 pts
crw-rw-rw-. 1 root root unconfined_u:object_r:user_tmp_t:s0                   1,   8 Apr 19 03:33 random
drwxrwxrwt. 2 root root system_u:object_r:svirt_sandbox_file_t:s0:c126,c739       40 Apr 19 03:33 shm
lrwxrwxrwx. 1 root root unconfined_u:object_r:user_tmp_t:s0                       15 Apr 19 03:33 stderr -> /proc/self/fd/2
lrwxrwxrwx. 1 root root unconfined_u:object_r:user_tmp_t:s0                       15 Apr 19 03:33 stdin -> /proc/self/fd/0
lrwxrwxrwx. 1 root root unconfined_u:object_r:user_tmp_t:s0                       15 Apr 19 03:33 stdout -> /proc/self/fd/1
crw-rw-rw-. 1 root root unconfined_u:object_r:user_tmp_t:s0                   5,   0 Apr 19 03:33 tty
crw-rw-rw-. 1 root root unconfined_u:object_r:user_tmp_t:s0                   1,   9 Apr 19 03:33 urandom
crw-rw-rw-. 1 root root unconfined_u:object_r:user_tmp_t:s0                   1,   5 Apr 19 03:33 zero

[rhatdan]

Yes if this is docker, then it is a regression.

[mrunalp]

@rhatdan We are missing context= in the mounts as you suspected. I am looking into a fix.

[mrunalp]

MountLabel support is missing in the spec. I have created a PR for it here opencontainers/runtime-spec#393