CUDA Version: N/A

[mayday0208]

Description

nerdctl run -it --rm --gpus capabilities=utility,device=GPU-3a23c669-1f69-c64e-cf85-44e9b07e7a2a nvidia/cuda:9.0-base nvidia-smi

when i use python pytorch,torch.cuda.is_available(),return false.

Steps to reproduce the issue

1.nerdctl run -it --rm --gpus capabilities=utility,device=GPU-3a23c669-1f69-c64e-cf85-44e9b07e7a2a nvidia/cuda:9.0-base nvidia-smi
2.install python3, pip install torch
3.python,import torch,retrun false

Describe the results you received and expected

can't use cuda

What version of nerdctl are you using?

0.19.0

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

No response

Host information

No response

[junnplus]

Try sudo nerdctl run --rm --gpus '"capabilities=compute,utility"' --debug nvidia/cuda:11.0-base nvidia-smi

[junnplus]

It seems that nerdctl and docker are a bit different:

[N] ❯ sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
Fri Apr 29 07:43:33 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.60.02    Driver Version: 510.60.02    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 35%   28C    P8     9W / 220W |    701MiB /  8192MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       717      G                                     386MiB |
|    0   N/A  N/A      4349      G                                     105MiB |
|    0   N/A  N/A      4445      G                                      60MiB |
|    0   N/A  N/A     11212      G                                     142MiB |
+-----------------------------------------------------------------------------+

[N] ❯ sudo nerdctl run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
Fri Apr 29 07:43:54 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.60.02    Driver Version: 510.60.02    CUDA Version: N/A      |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 35%   28C    P8     9W / 220W |    701MiB /  8192MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       717      G                                     386MiB |
|    0   N/A  N/A      4349      G                                     105MiB |
|    0   N/A  N/A      4445      G                                      60MiB |
|    0   N/A  N/A     11212      G                                     142MiB |
+-----------------------------------------------------------------------------+

compute is required for cude.

[N] ❯ sudo nerdctl run --rm --gpus '"capabilities=compute,utility"' nvidia/cuda:11.0-base nvidia-smi
Fri Apr 29 07:48:03 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.60.02    Driver Version: 510.60.02    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 35%   28C    P8     9W / 220W |    701MiB /  8192MiB |      9%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       717      G                                     386MiB |
|    0   N/A  N/A      4349      G                                     105MiB |
|    0   N/A  N/A      4445      G                                      60MiB |
|    0   N/A  N/A     11212      G                                     142MiB |
+-----------------------------------------------------------------------------+

[mayday0208]

It seems that nerdctl and docker are a bit different:

[N] ❯ sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
Fri Apr 29 07:43:33 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.60.02    Driver Version: 510.60.02    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 35%   28C    P8     9W / 220W |    701MiB /  8192MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       717      G                                     386MiB |
|    0   N/A  N/A      4349      G                                     105MiB |
|    0   N/A  N/A      4445      G                                      60MiB |
|    0   N/A  N/A     11212      G                                     142MiB |
+-----------------------------------------------------------------------------+

[N] ❯ sudo nerdctl run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
Fri Apr 29 07:43:54 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.60.02    Driver Version: 510.60.02    CUDA Version: N/A      |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 35%   28C    P8     9W / 220W |    701MiB /  8192MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       717      G                                     386MiB |
|    0   N/A  N/A      4349      G                                     105MiB |
|    0   N/A  N/A      4445      G                                      60MiB |
|    0   N/A  N/A     11212      G                                     142MiB |
+-----------------------------------------------------------------------------+

compute is required for cude.

[N] ❯ sudo nerdctl run --rm --gpus '"capabilities=compute,utility"' nvidia/cuda:11.0-base nvidia-smi
Fri Apr 29 07:48:03 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.60.02    Driver Version: 510.60.02    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 35%   28C    P8     9W / 220W |    701MiB /  8192MiB |      9%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       717      G                                     386MiB |
|    0   N/A  N/A      4349      G                                     105MiB |
|    0   N/A  N/A      4445      G                                      60MiB |
|    0   N/A  N/A     11212      G                                     142MiB |
+-----------------------------------------------------------------------------+

thanks! now i can use cuda!