Skip to content

fix(gallery): correct meta-backend definitions for platform auto-selection#10299

Merged
mudler merged 1 commit into
masterfrom
fix/gallery-meta-backends
Jun 13, 2026
Merged

fix(gallery): correct meta-backend definitions for platform auto-selection#10299
mudler merged 1 commit into
masterfrom
fix/gallery-meta-backends

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

Follows the same root cause as #10291 (opus): backends that ship per-platform images must be meta backends (a capabilities: map and no uri) so the right variant is auto-selected per platform. An audit of backend/index.yaml found several that were misdefined, so on the affected platform the wrong (or a non-existent) backend is selected.

Fixes

Category 1 - base was a concrete with a metal- sibling (identical to the opus bug)

silero-vad, piper, kitten-tts, local-store (each + -development): the anchor carried a uri and a separate metal-<name> existed, so on macOS Load("<name>") got the CPU build / failed. Converted each anchor to a meta ({default: cpu-<name>, metal: metal-<name>}), added the cpu-<name> concrete (keeping the original image tag), renamed <name>-development -> cpu-<name>-development, and added the <name>-development meta.

silero-vad is the VAD-on-Mac failure in #9813.

Category 2 - mlx family had uri AND capabilities (capabilities silently ignored)

mlx, mlx-vlm, mlx-audio, mlx-distributed (+ -development): because the anchor had a uri, IsMeta() was false and the capabilities map was ignored - Load("mlx") always got the metal-darwin-arm64 image (broken on CUDA/Linux), and the metal-<name> capability target did not even exist. Removed the uri/mirrors from each anchor (now real metas) and added the missing metal-<name> / metal-<name>-development concretes. Verified every other capability target exists.

Relevant to the mlx-vlm install problems in #9810.

Category 3 - dangling capability targets (install fails on those platforms)

  • diffusers nvidia-l4t-cuda-12: repointed nvidia-l4t-arm64-diffusers (missing) -> nvidia-l4t-diffusers (exists).
  • kokoro nvidia-l4t-cuda-12: repointed nvidia-l4t-arm64-kokoro (missing) -> nvidia-l4t-kokoro (exists).
  • coqui nvidia-cuda-13: removed the key (cuda13-coqui does not exist) so cuda13 falls back to nvidia/default.

Left unchanged (flagged for maintainer)

  • locate-anything: its capabilities reference *-locate-anything-cpp concretes that do not exist anywhere (no images, no entries) - it appears to be a broken/unpublished WIP entry. Not touched; please confirm whether its concretes should be published or the entry removed.

Verification

  • Referential-integrity audit: from 62 problems down to the documented residue (the only remaining flags are locate-anything, and opus/opus-development which are fixed in fix(gallery): make opus a meta backend for platform auto-selection (#9813) #10291 and absent from this branch).
  • go test ./core/gallery/... green.
  • backend/index.yaml parses; every meta capability target (except documented locate-anything) resolves to an existing concrete.

Net change is backend/index.yaml only.

Assisted-by: claude:claude-opus-4-8 [Claude Code]

@localai-bot

Copy link
Copy Markdown
Collaborator Author

Registry verification: queried quay.io/go-skynet/local-ai-backends for every image tag these restructured metas resolve to (all metal-/cpu-/cuda12-/cuda13-/nvidia-l4t- variants of silero-vad, piper, kitten-tts, local-store, mlx, mlx-vlm, mlx-audio, mlx-distributed - latest and master - plus the diffusers/kokoro repoints). All 82 distinct tags are published. No meta now points at a missing image; the change only re-points tags that already existed in the index.

locate-anything confirmed: no image is published under *-locate-anything or *-locate-anything-cpp for any platform - it is genuinely unpublished/WIP, hence left untouched here.

@mudler
mudler force-pushed the fix/gallery-meta-backends branch from 3e2d580 to 4a348de Compare June 13, 2026 08:08
@localai-bot

Copy link
Copy Markdown
Collaborator Author

Update: locate-anything is now wired up in this PR (the earlier "left for maintainer" note is superseded - confirmed with @mudler it was an accidental omission).

The meta locate-anything existed but its concrete entries were never added, so it was un-installable on every platform. Added the full concrete set (8 platform variants x release+development = 16 entries) plus the locate-anything-development meta, mirroring rfdetr-cpp exactly.

Image tags were grounded against the published quay.io tags, not assumed:

Capability Concrete Image variant master- (dev) latest- (release)
default cpu-locate-anything-cpp cpu-… published at next release
nvidia / nvidia-cuda-12 cuda12-… gpu-nvidia-cuda-12-… published at next release
nvidia-cuda-13 cuda13-… gpu-nvidia-cuda-13-… published at next release
nvidia-l4t / …-cuda-12 nvidia-l4t-arm64-… nvidia-l4t-arm64-… published at next release
nvidia-l4t-cuda-13 cuda13-nvidia-l4t-arm64-… nvidia-l4t-cuda-13-arm64-… published at next release
intel intel-sycl-f32-… gpu-intel-sycl-f32-… published at next release
vulkan vulkan-… gpu-vulkan-… published at next release

(Plus an intel-sycl-f16 concrete - the image is published and rfdetr-cpp ships the same orphan variant.) All master- images verified present on quay; latest- will publish with the next tagged release, exactly like every other backend. The -development meta resolves to the working master- images today.

Referential-integrity audit now reports zero missing capability targets across the whole index.

Backends that ship per-platform images must be meta backends (a capabilities
map and NO uri) so the right variant is auto-selected per platform - mirroring
llama-cpp/whisper. Several entries were misdefined; fixed here:

- Concrete base + metal sibling (could not select the Apple Silicon variant):
  silero-vad, piper, kitten-tts, local-store (+ their -development). Converted
  each anchor to a meta and added the cpu-<name> concrete.
- mlx family (mlx, mlx-vlm, mlx-audio, mlx-distributed + -development): anchor
  had both a uri AND a capabilities map, so IsMeta() was false and the map was
  ignored (always resolved to the metal-darwin image); the metal-<name> target
  did not exist. Removed the uri and added the missing metal-<name> concretes.
- Dangling capability targets: diffusers/kokoro nvidia-l4t-cuda-12 repointed to
  the existing nvidia-l4t-<name> concrete; coqui nvidia-cuda-13 key removed
  (no cuda13-coqui image).
- locate-anything: the meta existed but its concrete entries were never added,
  so it was un-installable on every platform. Added the full concrete set plus
  the locate-anything-development meta, mirroring rfdetr-cpp. Image tags grounded
  against the published quay.io tags.
- trl (cuda12/13): repointed the stale 'cublas-cuda12/13-trl' image tags to the
  actually-published 'gpu-nvidia-cuda-12/13-trl' tags (fixes #9236).

Assisted-by: claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
mudler force-pushed the fix/gallery-meta-backends branch from 4a348de to a9c656b Compare June 13, 2026 08:31
@mudler
mudler merged commit 0eca930 into master Jun 13, 2026
59 checks passed
@mudler
mudler deleted the fix/gallery-meta-backends branch June 13, 2026 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants