NVIDIA GPU support

[3XX0]

Hello, author of nvidia-docker here.

As many of you may know, we recently released our NVIDIA Docker project in our effort to enable GPUs in containerized applications (mostly Deep Learning). This project is currently comprised of two parts:

• A Docker volume plugin to mount NVIDIA driver files inside containers at runtime.
• A small wrapper around Docker to ease the deployment of our images.

More information on this here

While it has been working great so far, now that Docker 1.12 is coming out with configurable runtime and complete OCI support, we would like to move away from this approach (which is admittedly hacky) and work on something which is better integrated with Docker.

The way I see it would be to provide a prestart OCI hook which would effectively trigger our implementation and configure the cgroups/namespaces correctly.

However, there are several things we need to solve first, specifically:

1. How to detect if a given image needs GPU support
   Currently, we are using a special label com.nvidia.volumes.needed, but it is not exported as an OCI annotation (see Clarify how OCI configuration ("config.json") will be handled #21324)
2. How to pass down to the hook which GPU should be isolated
   Currently, we are using an environment variable NV_GPU
3. How to check whether the image is compatible with the current driver or not.
   Currently, we are using a special label XXX_VERSION

All of the above could be solved using environment variables but I'm not particularly fond of this idea (e.g. docker run -e NVIDIA_GPU=0,1 nvidia/cuda)

So is there a way to pass runtime/hook parameters from the docker command line and if not would it be worth it? (e.g. --runtime-opt)

[flx42]

+@cpuguy83 we briefly discussed that with you during DockerCon 16.

[thaJeztah]

ping @mlaventure @crosbymichael wdyt?

[3XX0]

Worth noting that opencontainers/runtime-spec#483 might affect this

[justincormack]

@3XX0 I guess another option would be to have a patched version of runc that has different options and knows about GPUs?

The image detection is a similar issue we have for multiarch in general, and there are flags on hub, but not well exposed yet. This might also work for driver version, let me find the spec.

[3XX0]

Yes I thought about it too but I would rather not have to track upstream runc and we would still hit the same problems since the runtime config in Docker is daemon-wide and static.

Thanks, It would be greatly appreciated.

[flx42]

I just saw that #24750 was closed and redirected here.
I believe we could already have basic GPU support with Docker Swarm if we were able to add devices when using docker service create. Is it on the roadmap?

Related: NVIDIA/nvidia-docker#141

[NanXiao]

@3XX0 @flx42 I am the original author of #24750 . I am not sure this question is appropriate to post, if not, please forgive me. If I just want to implement Docker Swarm to orchestrate Swarm cluster supporting GPU, not caring about whether to use native Docker or nvidia-docker, could you give some comments or suggestions here? Thanks in advance!

[justincormack]

@flx42 I think you may be able to bind mount in device nodes with docker service --mount but it is not very well documented yet as I think the CLI is still being finalised; I am fairly sure the API allows bind mounts though.

[cpuguy83]

--mount type=bind,source=/host/path,target=/container/path