docker-compose.yaml GPU example: capabilities [gpu, utility] omits compute, so libcuda.so.1 is not injected

[alaschgari]

LocalAI version:
quay.io/go-skynet/local-ai:master-gpu-nvidia-cuda-13 (sha256:02e823f5f25ddef3f4edb94a8960e506daa4f71595d7c75b041d52b6eaaf425b, LocalAI bcf196d), docker-compose.yaml at the same commit.

Environment, CPU architecture, OS, and Version:
Docker Desktop 29.7.2 (Compose 5.5.1) on Windows 11, WSL2 kernel 6.18.33.2-microsoft-standard-WSL2, x86_64. NVIDIA RTX 5070 Ti, driver 616.92.

Describe the bug
The commented NVIDIA example in docker-compose.yaml recommends:

    # environment:
    #   NVIDIA_DRIVER_CAPABILITIES: "compute,utility"
    # init: true
    # deploy:
    #   resources:
    #     reservations:
    #       devices:
    #         - driver: nvidia
    #           count: 1
    #           capabilities: [gpu, utility]

With capabilities: [gpu, utility], the container gets only the utility driver libraries. docker inspect shows "Capabilities":[["gpu","utility"]], and /usr/lib/x86_64-linux-gnu contains libnvidia-ml.so.1 and libdxcore.so but no libcuda.so.1. nvidia-smi works inside the container, so the setup looks fine, but every CUDA backend fails to start with ImportError: libcuda.so.1: cannot open shared object file. In my setup, also setting NVIDIA_DRIVER_CAPABILITIES=compute,utility in environment: (as the example suggests) did not change this.

Changing it to capabilities: [gpu, compute, utility] fixes it. libcuda.so.1 then shows up in /usr/lib/x86_64-linux-gnu and in ldconfig -p, and CUDA backends load.

To Reproduce

1. Enable the legacy driver: nvidia example from docker-compose.yaml as written, with a CUDA image such as master-gpu-nvidia-cuda-13.
2. docker compose up -d
3. docker exec <container> sh -c 'ldconfig -p | grep libcuda.so.1' returns nothing. Loading a model on a Python CUDA backend fails with the libcuda.so.1 error above.

Expected behavior
Following the compose example gives a container where CUDA backends work.

Logs

ERROR Failed to load model ... error=failed to load model with internal loader: grpc service not ready: backend process exited with code 1: ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

Additional context
Suggested fix: use capabilities: [gpu, compute, utility] in the examples, and mention that compute is what brings in libcuda. I only tested the legacy driver: nvidia variant, on Docker Desktop/WSL2. I did not test the CDI (nvidia.com/gpu) variant, which may treat capabilities differently. It's worth checking the docs pages that show the same snippet as well.

[localai-org-maint-bot]

I am picking this up. Latest master still omits compute from the legacy NVIDIA Compose device reservation. I will correct that example and the matching documentation, with a note explaining why CUDA needs compute alongside utility.