[Models] Phase 2 — ollama backend #629

Scope

Ship the first real ModelBackend implementation in core: an ollama backend that talks to a local or remote Ollama HTTP API. Validates the Phase 1 interface against something non-trivial without external dependencies in CI.

What ships:

• ollama backend class implementing ModelBackend with embed, generate, generateStream.
• Config-driven backend resolution (backend: ollama, host: ..., model: ...).
• Streaming via Ollama's chunked response → AsyncIterable<GenerateChunk> per Phase 1's contract.
• AbortSignal honored from BackendOpts — propagated into the underlying fetch() so dropped-client cancellation aborts upstream.
• Per-call accounting written through hdb_model_calls via Phase 1's writer.
• Integration tests against a real local Ollama (matching the source-portability promise).

API surface

Implements the interface from Phase 1 (#628). Capability negotiation:

capabilities(): ModelCapabilities {
  return {
    embed: true,
    generate: true,
    stream: true,
    tools: false,        // Ollama tool support is evolving; not yet portable enough for v1
    adapters: false,     // No multi-LoRA serving in Ollama
  };
}

Configuration (extends Phase 1's backend registry shape):

models:
  embedding:
    default:
      backend: ollama
      host: localhost:11434
      model: <your-embedding-model>     # e.g. nomic-embed-text
  generative:
    fast:
      backend: ollama
      host: localhost:11434
      model: <your-generative-model>    # e.g. llama-3.2-3b

Multiple ollama backends can coexist (different host and model per logical name). The registry indexes them by logical name from config.

Implementation notes

• HTTP client: native fetch(). No new SDK dependency in core.
• Ollama endpoints used:
  • POST /api/embeddings for embed
  • POST /api/generate (non-streaming) and POST /api/chat (chat-shape) for generate
  • Same with stream: true for generateStream
• Token-count fields when Ollama returns them (prompt_eval_count, eval_count) map to TokenUsage.promptTokens / completionTokens / embeddingTokens.
• gpu_ms is not reported by Ollama; left undefined in the accounting record.
• inputType: 'document' | 'query' on EmbedOpts: Ollama doesn't currently consume this distinction in its embedding API, but some models (nomic-embed-text v1.5+) prepend the appropriate prefix at the application layer. Backend handles the prefix injection when the model is one that uses it; documented in code as a per-model behavior table.
• signal: AbortSignal from BackendOpts is passed straight into the fetch() call; client disconnect → fetch aborts → upstream HTTP closes.

Files

Backend registration: when Harper boots and reads the models config, backend: ollama instantiates OllamaBackend with the provided host and model, registers under the logical name in the backend registry.

Acceptance criteria

• OllamaBackend implements ModelBackend per Phase 1's interface.
• scope.models.embed() configured with backend: ollama produces a vector from a running Ollama instance.
• scope.models.generate() produces a completion from a running Ollama instance.
• scope.models.generateStream() yields content deltas via the AsyncIterable<GenerateChunk> shape from Phase 1.
• Per-call accounting records backend: 'ollama', model, token counts (when Ollama reports them), latency, success.
• AbortSignal from BackendOpts cancels in-flight requests when a client disconnects.
• Capability negotiation correctly reports tools: false, adapters: false.
• inputType: 'document' | 'query' produces different vectors for models that distinguish (e.g. nomic-embed-text v1.5+ when configured).
• Integration tests pass against a local Ollama instance (CI provisioning to be settled in PR review — likely a container in the integration shard).
• CI green (unit + integration + 3 Node versions).

Out of scope

• Tool calls via Ollama — tools: false advertised; Ollama tool-call support is uneven across models and not portable enough for v1.
• LoRA adapter selection — adapters: false; multi-LoRA serving is FAB-503 territory via vLLM.
• Streaming cancellation beyond AbortSignal propagation (no explicit "cancel" endpoint on Ollama).
• Custom HTTP transport (mTLS, proxies, etc.) — uses default fetch() config for now; extensions ship as backend-config options later.

Stacks on

• Phase 1 ([Models] Phase 1 — Foundation: scope.models interfaces, hdb_model_calls, TestBackend #628) — needs the ModelBackend interface, backend registry, and hdb_model_calls writer.

Hard prerequisites

• Expose request.signal (AbortSignal) for Resource methods #513 (request.signal exposed on Resource methods) — Phase 1 reads ctx.signal from ALS; this phase consumes it through BackendOpts.signal.

Branch & PR conventions

• Branch: feat/models-ollama-backend
• PR base: main (after Phase 1 merges).
• Closes this issue via Closes #<self>; references Add unified model-access API (scope.models) #510 via Tracking: #510.

Smoke test

# Prerequisites:
# - Ollama running locally on :11434 with a model pulled (e.g. `ollama pull nomic-embed-text`)
# - Harper config has models.embedding.default = { backend: ollama, host: localhost:11434, model: nomic-embed-text }

# In a Resource method:
class EmbedTest extends Resource {
  async post(_target, body, _request) {
    return await scope.models.embed(body.text);
  }
}

curl -X POST http://localhost:9926/EmbedTest/ \
  -H 'Content-Type: application/json' \
  -d '{"text": "hello world"}'

# Expected: a real 768-dim (or model-appropriate) Float32Array
# Verify: SELECT * FROM system.hdb_model_calls WHERE backend = 'ollama' ORDER BY $createdtime DESC LIMIT 1
#   shows the call with method='embed', model, latency_ms, embedding_tokens, success=true.

Tracking

Part of #510. Sub-issue 2 of 6.

🤖 Generated with Claude Code