fix(local-ai): enable NVIDIA GPU passthrough - #8462
Draft
bitsandbots wants to merge 1 commit into
Draft
Conversation
bitsandbots
marked this pull request as ready for review
July 17, 2026 18:57
bitsandbots
force-pushed
the
fix/local-ai-enable-nvidia-gpu-v2
branch
from
July 17, 2026 19:11
1bfb96e to
fe0a1d6
Compare
The local-ai community container only requests /dev/dri, which is the Mesa/AMD/Intel Vulkan mechanism. On NVIDIA-only hosts this leaves Vulkan with no usable device, so every backend silently falls back to llvmpipe (CPU), causing image generation to take minutes and blow past integration_openai's request timeout. Add enable_nvidia_gpu, matching the plex, jellyfin, and memories community containers, so AIO sets Runtime=nvidia and the proper DeviceRequests when the host operator has NEXTCLOUD_ENABLE_NVIDIA_GPU set. /dev/dri stays for Mesa-only hosts, matching memories' pattern of combining both. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Cory <bitsandbots@gmail.com>
bitsandbots
force-pushed
the
fix/local-ai-enable-nvidia-gpu-v2
branch
from
July 17, 2026 19:17
fe0a1d6 to
0dfefd2
Compare
8 tasks
Collaborator
|
@bitsandbots did you test and verified that this works locally? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
local-aicommunity container only requests/dev/dri, which is the Mesa/AMD/Intel Vulkan device passthrough mechanism.llvmpipe(CPU-only software rendering).integration_openai'simage_request_timeout(240s default) and causing Assistant sticker/image tasks to fail outright.plex,jellyfin, andmemoriesalready solve this for their own GPU-accelerated use cases via"enable_nvidia_gpu": true, whichDockerActionManagerturns intoRuntime: nvidia+ the correctDeviceRequestswhen the host operator has setNEXTCLOUD_ENABLE_NVIDIA_GPU=trueon the mastercontainer.local-ai.jsonwas missing this flag.Change
"enable_nvidia_gpu": truetocommunity-containers/local-ai/local-ai.json, alongside the existing"devices": ["/dev/dri"](same pattern asmemories.json, so Mesa-only hosts are unaffected and NVIDIA hosts now get real acceleration).Testing
Reproduced and fixed on an NVIDIA RTX 5070 host:
vulkaninfoinside the container showed onlyllvmpipe(PHYSICAL_DEVICE_TYPE_CPU); a 512x512 SD1.5 image generation took 140s+ (cold) and still exceeded the timeout under normal load.NEXTCLOUD_ENABLE_NVIDIA_GPU=trueon the mastercontainer + hasnvidia-container-toolkitinstalled/configured):vulkaninfocorrectly reports theNVIDIA GeForce RTX 5070(PHYSICAL_DEVICE_TYPE_DISCRETE_GPU), and the same image generation completed in 4.5s once the backend was warm, withnvidia-smiconfirming the stablediffusion-ggml process resident on the GPU.local-ai.jsonstill validates againstphp/containers-schema.json(theenable_nvidia_gpuboolean property already exists in the schema, used by the other GPU-enabled community containers).Notes
NEXTCLOUD_ENABLE_NVIDIA_GPU=trueon the mastercontainer (per the existing hardware-acceleration docs in the main readme) and has the NVIDIA Container Toolkit installed. Mesa/AMD/Intel-only hosts are unaffected since/dev/dripassthrough is untouched.🤖 Drafted with Claude Code