NVIDIA support with docker driver

[chrisroat]

The doc page on using NVIDIA GPUs covers the none and kvm2 drivers. The none driver is considered advanced, while the kvm2 requires a spare GPU.

Is support for the docker driver feasible/planned?

[afbjorklund]

It should be possible, as long as the outer privileged container is launched with nvidia-docker: NVIDIA/nvidia-docker#375

But it is not something that would work out-of-the-box. If it was a simple option (like --runtime), it could be considered.

Running on Linux, that is.

Not for the Docker VM...

[chrisroat]

Yes, I was considering running on Linux and I am using nvidia-docker.

I'm not overly familiar with minikube and how to adjust the launch of the "outer privileged container". My use case is a helm chart which I run on GKE w/GPUs, which I want to iterate locally on.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[pythonwood]

nvidia + docker is easy now. install-guide

docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
Sat Sep 18 04:14:32 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.63.01    Driver Version: 470.63.01    CUDA Version: 11.4     |
...

It is so good if I can use gpu in minikube

[sharifelgamal]

We don't currently have the bandwidth to implement a feature like this, but I would be happy to review a PR that does this.

[d4l3k]

I've added the beginnings of support at #15927. It works E2E but has the caveat that you need a custom kicbase image with an exactly matching NVML version. Reviews welcome :)

[anthonyalayo]

Thanks @d4l3k ! I think a lot of people will appreciate this. What is left for support?

[d4l3k]

@anthonyalayo need a maintainer to chime in on approach and then update the PR to better handle the Nvidia dependencies

[anthonyalayo]

Could a maintainer comment here?