feat(gallery): fall back to mirrors and a cached index when the primary source fails

[localai-bot]

A LocalAI instance that cannot fetch its gallery index cannot list or install anything. Today there is exactly one URL per gallery and no recovery: if GitHub's raw endpoint is slow, rate-limited, or unreachable from the network the instance runs on, the model list is empty and stays empty.

This adds a mirrors list to gallery configuration, plus a last-known-good copy on disk, so a gallery has somewhere to fall back to.

[
  {
    "name": "localai",
    "url": "https://my-internal-mirror.corp/index.yaml",
    "mirrors": ["github:mudler/LocalAI/gallery/index.yaml@master"]
  }
]

Sources are tried in order — primary, then each mirror — and a candidate that just failed is kept out of the rotation for 10 minutes so the next listing does not pay its timeout again. If every source fails, the last index that was successfully fetched and parsed is served from disk with a warning. This is useful to anyone running an internal or air-gapped mirror, and the offline tier helps even a stock install: a laptop that has listed the gallery once can list it again on a plane.

Also included: outbound HTTP requests from pkg/downloader now send a User-Agent (LocalAI/<version> (<os>; <arch>)). Previously they sent none, which some CDNs and proxies treat as a bot.

Defaults are unchanged

core/config/runtime_settings_startup.go is untouched — DefaultGalleriesJSON and DefaultBackendGalleriesJSON still point at github:mudler/LocalAI/..., with no mirrors. This PR is mechanism only. A later, separate PR may point the defaults at an index host with GitHub as the mirror; keeping that out of here means this change can be judged on its own and reverted independently.

Behaviour changes a reviewer should weigh

pkg/downloader now treats an HTTP error status as an error

ReadWithAuthorizationAndCallback never looked at the response status. A 404 page or a 502 from a proxy was handed to the caller as a successful read, and the caller parsed the HTML into an empty result. There are 5 non-test callers of the affected path, so this is a real behaviour change: a gallery or model-config URL that has quietly been 404ing will now surface as an error instead of an empty list that gets cached for an hour.

That is the intent. Mirror fallback cannot be built on top of a read that reports failure as success — a dead primary would look fine and no mirror would ever be tried. But some users will see a misconfiguration that was previously invisible turn into a visible failure. It is isolated in its own commit (fix(downloader): treat an HTTP error status as a failed read) if it needs to be discussed or reverted separately.

A whole-transfer deadline on gallery index fetches

Each candidate attempt is now bounded at 120 seconds end to end. Before, only the response headers were bounded — a source that accepted the connection and then dribbled bytes forever would hang the fetch indefinitely, which in practice is the more common GitHub failure mode than an outright error.

120s is deliberately loose. The default index is ~2.2 MB, so it tolerates a sustained ~19 KB/s. Anything slower than that could not go on to download a model anyway, and a tighter value would push healthy-but-slow links into the 10-minute cooldown.

The offline cache

Successful fetches are written to <MODELS_PATH>/../cache/gallery/, one file per gallery URL (name is a SHA-256 of the URL). It is never expired and is safe to delete at any time — a missing cache just means the fallback chain ends one tier earlier.

Only a body that parses as a gallery index is cached. A captive portal or a proxy returning an HTML error page with HTTP 200 therefore cannot poison it.

Known limitations, called out on purpose

Mirrors do not cover a .ref gallery URL. A .ref is resolved to its target before the fallback chain is entered, so if the .ref itself is unreachable neither the mirrors nor the offline cache help. Documented in docs/content/features/model-gallery.md rather than papered over.

Mirror URLs are not run through validateGalleryConfigURL. That validator guards GetGalleryConfigFromURL, where the URL arrives in a request body. Mirrors arrive through the same operator-controlled channel as the primary URL — LOCALAI_GALLERIES or the admin-gated POST /api/settings — and the index fetch has never validated the primary. A mirror is no more privileged than the URL it backs up, so validating mirrors alone would buy nothing while breaking the LAN and air-gapped deployments mirrors exist for. file:// mirrors stay confined to the models directory by the downloader's existing basePath check. The reasoning is in a comment at galleryCandidates so it does not have to be rediscovered.

config.Gallery is no longer comparable with ==. It gained a slice. Gallery.Equal and GalleriesEqual were added and core/config/runtime_settings_registry.go updated to use them. One subtlety: Equal compares the Verification pointer by value, where == compared pointer identity. That is a change, and it is the more correct one here — the registry diffs two independent JSON parses of the same settings, so structurally identical policies were previously reported as different. Covered by tests in core/config/gallery_test.go.

A pre-existing UI bug a maintainer should know about

While updating the gallery placeholders in Settings.jsx to mention mirrors, it became clear the fields they annotate do nothing. The Settings page posts galleries_json and backend_galleries_json; neither key exists anywhere in the Go tree. Gallery edits made from the React UI are silently discarded while the toast reports success. This dates to aee4611ab and is not fixed here — it is a separate bug with its own blast radius — but the field this PR documents is currently a no-op from the UI, and that seemed worth surfacing rather than leaving in a commit message.

Also minor and pre-existing: several ginkgo suites use os.MkdirTemp("", …) as a models path, which now makes the sibling cache directory land in /tmp/cache. This branch contains that for core/gallery with a package TestMain; core/http/endpoints/localai still exhibits it.

Testing

go vet and go test -count=1 -race pass over ./internal/..., ./pkg/downloader/..., ./pkg/oci/..., ./core/config/..., ./core/gallery/.... New unit tests cover the User-Agent format, the downloader status check, Gallery.Equal, candidate ordering and deduplication, cooldown behaviour, and each fallback tier including cache write, cache read, and the parse gate on caching.

Verified by hand with a deliberately dead primary (the discard port) and a working mirror: the primary fails fast with a warning, the mirror serves the listing. Re-run with every source dead, the on-disk copy serves it.

Attribution

Per AGENTS.md, AI involvement is attributed with Assisted-by: trailers. There is no Co-Authored-By and no Signed-off-by from the assistant — the human submitter adds their own sign-off if DCO is required.

🤖 Generated with Claude Code