Do not pass all host devices in /dev when launching with '--privileged'

[zhiminghufighting]

Description of problem

When i launch an android image with --privileged parameter, all device under host /dev directory will be passed into the container directory /dev, the behavior doesn't make sense;

Expected result

Don't pass all host devices into kata container even with --privileged parameter;

Actual result

Here is thels info under kata container directory /dev:

root@5ff9d5d81457:/dev# ls
agpgart          dri    input         lp3                 null    raw   shm       tap21605  tty14  tty21  tty29  tty36  tty43  tty50  tty58  tty8    urandom  vcs6   vfio
autofs           fb0    kmsg          mapper              nvram   rtc0  snapshot  tty       tty15  tty22  tty3   tty37  tty44  tty51  tty59  tty9    usbmon0  vcsa   vga_arbiter
bsg              fd     kvm           mcelog              oldmem  sda   snd       tty0      tty16  tty23  tty30  tty38  tty45  tty52  tty6   ttyS0   vcs      vcsa1  vhci
btrfs-control    fd0    loop-control  mem                 port    sda1  sr0       tty1      tty17  tty24  tty31  tty39  tty46  tty53  tty60  ttyS1   vcs1     vcsa2  vhost-net
console          full   loop0         mqueue              ppp     sda2  stderr    tty10     tty18  tty25  tty32  tty4   tty47  tty54  tty61  ttyS2   vcs2     vcsa3  vmci
cpu              fuse   lp0           net                 ptmx    sda3  stdin     tty11     tty19  tty26  tty33  tty40  tty48  tty55  tty62  ttyS3   vcs3     vcsa4  vsock
cpu_dma_latency  hpet   lp1           network_latency     pts     sg0   stdout    tty12     tty2   tty27  tty34  tty41  tty49  tty56  tty63  uhid    vcs4     vcsa5  zero
crash            hwrng  lp2           network_throughput  random  sg1   tap20463  tty13     tty20  tty28  tty35  tty42  tty5   tty57  tty7   uinput  vcs5     vcsa6
root@5ff9d5d81457:/dev# 

Here is the host view of docker ps:

[root@centos-k8s014 home]# docker ps
CONTAINER ID        IMAGE                                COMMAND             CREATED             STATUS              PORTS               NAMES
5ff9d5d81457        sgjesse/legacy-android-base:latest   "/bin/bash"         About an hour ago   Up About an hour                        gracious_wright
619d72f2ad28        android:NAe000047                    "/android-entry"    About an hour ago   Up About an hour                        friendly_ritchie
822fa322b90c        android:NAe000047                    "/android-entry"    About an hour ago   Up About an hour                        dazzling_fermi

[zhiminghufighting]

Please see the related issue and root cause here:
#1551

[zhiminghufighting]

any idea about this issue?

[jodh-intel]

Hi @zhiminghufighting - I've updated the description and reformatted the details to make this easier to understand. Please can you provide details of which devices you would not expect to see?

[jodh-intel]

Related: #1558.

/cc @amshinde.

[amshinde]

@zhiminghufighting I am not sure why you are seeing this behaviour, before I go take a look, can you tell me what is the specific android image that you are using?

[leoluk]

This is highly unexpected and undocumented behavior that just fell on my feet during a CTF competition that uses Kata Containers for isolation :)

I passed --privileged=true such that I could run Docker containers inside the VM, but this also resulted in full host access that some players used to break out of the VMs.

Applied a local hotfix (no-op filterDevices), but I suggest the default behavior be changed.

[grahamwhaley]

@leoluk - saw your PR over at kata-containers/documentation#452, thx.
Heh, I guess it is a question of 'when is --privileged not really --privileged... I have a feeling we may never please all the users all the time on this one ;-)
/cc @amshinde for thoughts and input.

[zhiminghufighting]

@amshinde I use a android based container which is running in Tencent cloud gaming.(This android based image also is made by intel AIS team for Tencent cloud gaming)
I tried in many conditions and found it only can be found in this condition : Host - Ubuntu + Kata + android based container;
If the Host OS is changed to Centos(7.3), it will never happen. i am not sure if it is related with OS policy?
By the way, the Ubuntu OS is 18.04; Kata is 1.6; i haven't tried on other combination.

@jodh-intel Thanks for your comment and corrections.