feat(localai): select the accelerator image variant during setup - #171
Merged
Conversation
Detect NVIDIA, AMD, Intel and Vulkan capability on the host and offer the matching LocalAI image, instead of always installing the CPU one. Bump LocalAI to v4.7.1.
The detection shells out to nvidia-smi and docker info. Running it silently right after the LocalAI URL prompt reads as a hang.
This was
linked to
issues
Aug 3, 2026
Closed
This was referenced Aug 3, 2026
Closed
14 tasks
marko-boehm
added a commit
that referenced
this pull request
Aug 3, 2026
## Summary Completes the curated `[1.0.0]` section in `CHANGELOG.md` before the milestone is integrated into `main`. Two gaps remained after #168 wrote the section: the release date was still a placeholder, and #171 was not represented at all — neither the accelerator-variant selection during setup nor the LocalAI bump that came with it. ## Linked issues None — release-artefact maintenance for milestone "v1.0.0 – Lean Core & Extension Platform". ## Type of change - [ ] Bug fix (`fix:`) - [ ] New feature (`feat:`) - [x] Documentation (`docs:`) - [ ] Refactor / chore (`refactor:` / `chore:`) - [ ] CI / build (`ci:` / `build:`) - [ ] Breaking change ## What changed - `## [1.0.0] - 2026-07-26` -> `## [1.0.0] - 2026-08-03` (actual release-cut date) - Features: added the LocalAI accelerator image variant (CPU / NVIDIA / AMD / Intel / Vulkan) selected during setup and proposed by a host hardware probe - Maintenance: added the LocalAI bump to `v4.7.1` The section's existing curation is kept as is: it lists user-visible features, maintenance and documentation only. Fixes to features that never shipped outside the milestone stay out. ## Test plan - [x] `[1.0.0]` section now covers every user-visible change on `releases/1.0.0` up to `b1d03ec` - [x] Entries verified against the branch: `src/ai/localai/docker-compose.yml` pins `localai/localai:v4.7.1`; the variant selection lives in `tools/lib/gpu_detect.py` and `tools/lib/sh/setup.sh` - [x] Keep-a-Changelog structure and heading order unchanged - [x] Documentation-only change, no code or configuration touched ## Checklist - [x] Title follows [Conventional Commits](https://www.conventionalcommits.org/) (e.g. `feat: …`, `fix(scope): …`) - [x] Documentation updated (README / docs / CHANGELOG) where applicable - [x] No secrets, credentials, or `.env` files committed - [x] Self-reviewed the diff
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
LocalAI was pinned to the CPU image on every host, with no GPU devices passed through. Setup
now detects what the host can run — NVIDIA (CUDA), AMD (ROCm), Intel (SYCL), Vulkan — and asks
which image to install, listing the driver prerequisites before the choice. Also raises the
LocalAI pin to v4.7.1.
Only the variant is stored (
LOCALAI_IMAGE_VARIANT). The version stays pinned insrc/ai/localai/docker-compose.ymland the accelerator tag is recomposed from it on everyrender, so
papaia-ctl upgrademoves GPU installs forward exactly like CPU ones — storing afull tag would have frozen the LocalAI version in the config dir via sticky reuse.
The image swap and device passthrough land in a generated
overrides/docker-compose.localai-gpu.override.yml, following the write-or-delete idiom ofgenerate_ssl_cert_override; the core compose file is never mutated. Selecting CPU deletes theoverride again.
Three deliberate design points:
the preselected default only — an operator may configure the stack before installing the
driver or container runtime. Choosing an undetected variant prints a warning.
fork
nvidia-smi. The override generator runs on every render (and therefore on everypapaia-ctl start) and is subprocess-free.nvidia-smianddocker info,which is slow enough on a cold host that a silent wait reads as a hang.
Vulkan is the one variant whose device set is not fixed — upstream documents a different set
per vendor — so it emits the union of the device nodes actually present.
The wizard step looks like this:
Linked issues
Closes #169
Closes #170
Type of change
feat:)Test plan
python -m pytest tests— 441 passed (25 new: detection, override generation, resolvepass, sticky defaults)
shellcheck --severity=warning tools/papaia-ctl tools/lib/sh/*.sh— cleanyamllint .— cleanset -euo pipefail; all return 0 and the expected variantNVIDIA GeForce RTX 4060 Laptop GPU, CUDA 13→recommends
nvidia-cuda-13setup -y --localai-variant=auto→LOCALAI_IMAGE_VARIANT=nvidia-cuda-13plus an override carrying
localai/localai:v4.7.1-gpu-nvidia-cuda-13intel→ tag and/dev/driupdate; switch tocpu→ override removed--localai-variant=bogus→ exits 2 with the valid values listeddocker compose configagainst base + override resolves the expected image,devices,group_addand the NVIDIA reservationv9.9.9, asserted in the tests)succeeds
Not covered: an inference run on AMD, Intel or Vulkan hardware — none available. For those
three the resolved
docker compose configis the evidence.Checklist
.env.example,models.txt,THIRD_PARTY_LICENSES.md; the deadLOCALAI_IMAGEhint is gone).envfiles committed