feat: add LiteLLM as an AI gateway provider

[prodmanpd]

What does this PR do?

Adds LiteLLM as a first class chat provider for the AI SRE agent, registered in the eino model builder registry alongside openai, deepseek, qwen, ollama, claude, and gemini. Set agent.ai.provider: litellm to route detect and analyze through a LiteLLM proxy, one OpenAI compatible endpoint that fans out to 100+ upstream providers, reusing the already vendored eino-ext/components/model/openai with no new dependency.

Changes:

• pkg/agent/ai/eino/provider.go: buildLiteLLMChatModel plus the "litellm" registry entry.
• pkg/agent/ai/eino/provider_test.go: registry, exported validator, and egress tests for litellm.
• pkg/config/agent.go, pkg/agent/aisettings.go, pkg/agent/ai/eino/holder.go: provider list doc comments.
• config/config.yaml, pkg/config/default_config.yaml: provider comment plus a litellm usage note.
• src/agent/configuration.md: provider table row plus a LiteLLM gateway section.

Why?

This adds gateway support so operators can centralise keys, spend caps, caching, and fallbacks in one proxy while Versus keeps a single stable config. The proxy speaks the OpenAI chat completions wire format, so it slots into the registry as a real discrete builder rather than a generic base url toggle, which means SupportedProviders() (the enterprise write boundary validator) and the runtime fail fast path both cover it for free. Prior art: #160 proposed an OpenAI compatible base_url field naming Gemini, LiteLLM, and OpenRouter; the AI layer then moved to the Eino framework with discrete first class builders, and this PR follows that direction with a real buildLiteLLMChatModel.

How to test

Sample config:

agent:
  ai:
    enable: true
    provider: litellm
    api_key: ${LITELLM_VIRTUAL_KEY}   # proxy virtual key (Bearer)
    model: gpt-5                       # a LiteLLM model alias from your proxy config
    temperature: 0.2
    max_tokens: 1024

model is the alias defined in your proxy config (not a raw provider id); the endpoint defaults to a local sidecar at http://localhost:4000/v1.

Unit tests, go test -race ./pkg/agent/ai/eino/...:

--- PASS: TestChatModel_RegistryBuildsAllProviders/litellm
--- PASS: TestSupportedProvidersExported
--- PASS: TestChatModel_LiteLLM_EgressBearerAndJSONMode
ok  github.com/VersusControl/versus-incident/pkg/agent/ai/eino

The egress test spins up an httptest OpenAI compatible server and asserts that the virtual key rides the Bearer Authorization header, that JSON mode sends response_format:{type:json_object}, and that the reply round trips through detect.ParseFinding.

Live end to end through a real LiteLLM proxy. NewToolCallingChatModel(provider=litellm) builds buildLiteLLMChatModel, which calls the OpenAI SDK, which calls the LiteLLM proxy, which calls the upstream:

LIVE litellm response via http://localhost:4000/v1 (model=azure/gpt-4o-mini): "LITELLM_OK"
--- PASS: TestLive_LiteLLM (1.05s)

Proxy side confirmation of the same call (Azure OpenAI upstream through the proxy):

content: LITELLM_OK
model: azure/gpt-4o-mini
usage: {'completion_tokens': 6, 'prompt_tokens': 16, 'total_tokens': 22, ...}

Bug caught and fixed during the deep dive: the proxy path shares the OpenAI SDK, which validates the beta limitation client side, so for reasoning families (gpt-5.*, o series) an explicit temperature is rejected before the request leaves. Naively forwarding the configured temperature broke a gpt-5 alias. The builder now reuses the same isFixedSamplingModel omission as buildOpenAIChatModel, so an operator aliasing gpt-5 through the proxy does not have to set temperature: -1 by hand. The egress test asserts that temperature is absent from the wire for a gpt-5 alias.

Build, vet, and format are clean: go build ./..., go vet ./pkg/agent/ai/eino/... ./pkg/config/..., and gofmt -l. go test -race ./pkg/... is green for every package that builds locally, including pkg/agent/ai/eino, pkg/agent/ai/detect, pkg/agent/ai/analyze, pkg/config, and pkg/core. pkg/controllers and pkg/routes only fail to build locally because they //go:embed ui/dist, which needs the frontend build and is untouched by this PR.

Type of change

• Bug fix (non-breaking)
• New feature (non-breaking)
• Breaking change (config / API / default behavior)
• Documentation only
• Refactor (no functional change)
• CI / build / chore

Checklist

• go test ./... passes locally
• go vet ./... is clean
• Code is gofmt'd
• Added or updated tests for the change
• Updated user-facing docs under src/ if behavior changes
• Updated ROADMAP.md if this closes a roadmap item
• No secrets, tokens, or webhook URLs introduced in source / YAML
• No new third-party dependencies (or justified in the description)

[hoalongnatsu]

Thanks for the PR @prodmanpd; we'll review it

[hoalongnatsu]

Hi @prodmanpd

Blocker

B1 — The proxy endpoint is not configurable; LiteLLM is locked to localhost

buildLiteLLMChatModel defaults BaseURL to http://localhost:4000/v1 and
otherwise uses req.baseURL — but that field is a test-only seam
(pkg/agent/ai/eino/provider.go:37 — "test-only Options.BaseURL; "" uses the
provider default"), and there is no base_url mapstructure field anywhere in
AgentAIConfig. An operator therefore cannot point this provider at any proxy
other than a local sidecar.

That undercuts the main reason teams adopt LiteLLM: a shared, centralised
gateway (https://litellm.internal.corp) holding the keys, spend limits and
fallbacks. The ollama/qwen precedent does not apply — their defaults are the
canonical endpoints for those backends, whereas a LiteLLM address is inherently
deployment-specific.

The documentation compounds it by omission. src/agent/configuration.md says
"The endpoint defaults to a local proxy at http://localhost:4000/v1 (the
common sidecar deployment)", which reads as though a knob exists.

Ask: add an operator-settable base_url (env AGENT_AI_BASE_URL) wired
through AgentAIConfig → chatModelRequest, or — if that is out of scope for
this PR — state the limitation explicitly in the docs and the config comments.
The knob would also benefit self-hosted OpenAI-compatible backends (vLLM,
LocalAI).

---

Should-fix

S1 — UI provider list not updated, breaking a stated invariant

ui/src/lib/api.ts declares the AIProvider union and AI_PROVIDERS array with
a comment stating it "mirrors the OSS chat-model registry
(eino.SupportedProviders)". litellm is missing from both.

The enterprise runtime override derives its supported set from
eino.SupportedProviders(), so the server already accepts litellm — but the
AI-settings dropdown cannot offer it. Net effect: LiteLLM is YAML-only, with no
runtime provider switch, and the documented mirror invariant is now false.

HEADER_AUTH_PROVIDERS correctly needs no change — LiteLLM authenticates with a
Bearer virtual key, so the per-org runtime key override applies as-is.

S2 — Embeddings gap for an operator who standardises on LiteLLM

embedderBuilders (pkg/agent/ai/eino/embedder.go:43) is {openai, ollama, gemini}. Setting provider: litellm and using runbook RAG fails hard with
"unsupported ai provider "litellm" for embeddings".

Unlike deepseek/claude — deliberately unwired because they have no embedding
component — LiteLLM does proxy OpenAI-compatible /v1/embeddings, so this is
an avoidable gap. It is trivially wired by reusing buildOpenAIEmbedder with the
same endpoint default. At minimum, document that RAG needs a different embedding
provider.