How to use a local LLM with OAuth instead of API Key

[nikhilsimha85]

The hindsight only has HINDSIGHT_API_LLM_API_KEY for API key in configuration. What should I use if I want to pass OAuth token

[nicoloboschi]

@nikhilsimha85 this is not supported right now. What provider you needed to use OAuth token with? GCP, AWS, Azure?

[nikhilsimha85]

AWS. Cant we have a config for auth token just like how Anthropic accepts either api key or auth token

[nikhilsimha85]

@nicoloboschi Also, how do i work with ollama.
docker run --rm -it --pull always -p 8888:8888 -p 9999:9999
-e HINDSIGHT_API_LLM_PROVIDER=${HINDSIGHT_API_LLM_PROVIDER:-ollama}
-e HINDSIGHT_API_LLM_BASE_URL=${HINDSIGHT_API_LLM_BASE_URL:-http://localhost:11434/v1}
-e HINDSIGHT_API_LLM_MODEL=${HINDSIGHT_API_LLM_MODEL:-deepseek-r1}
-v $HOME/.hindsight-docker:/home/hindsight/.pg0
ghcr.io/vectorize-io/hindsight:latest

gives an error:

File "/app/api/hindsight_api/engine/llm_wrapper.py", line 172, in verify_connection
raise RuntimeError(f"LLM connection verification failed for {self.provider}/{self.model}: {e}") from e
RuntimeError: LLM connection verification failed for ollama/deepseek-r1: Connection error.

Curl below works fine

curl http://localhost:11434/api/chat -d '{
"model": "deepseek-r1",
"messages": [
{"role": "user", "content": "What is the capital of France?"}
]
}'

[nicoloboschi]

@nikhilsimha85 since you're running in docker, you need to point to the host machine, not localhost.

on macos, this should be host.docker.internal

> docker run --rm -it -p 8888:8888 -p 9999:9999 -e HINDSIGHT_API_LLM_PROVIDER=ollama -e HINDSIGHT_API_LLM_BASE_URL=http://host.docker.internal:11434/v1 -e HINDSIGHT_API_LLM_MODEL=gemma3:270m ghcr.io/vectorize-io/hindsight:latest



  ▄▄       ▄▄
 ▄ ▀▄ ▄▄▄ ▄▀ ▄
▀▀▄▄▄▄▀▀▀▄▄▄▄▀▀
   ▄▄▄ ▄ ▄▄▄
 ▄▀ ▀▀▄▄▄▀  ▀▄

  Hindsight: Agent Memory That Works Like Human Memory

2026-01-23 09:37:27,522 - INFO - hindsight_api.config - Database: pg0
2026-01-23 09:37:27,522 - INFO - hindsight_api.config - LLM: provider=ollama, model=gemma3:270m
2026-01-23 09:37:27,522 - INFO - hindsight_api.config - Embeddings: provider=local
2026-01-23 09:37:27,522 - INFO - hindsight_api.config - Reranker: provider=local
2026-01-23 09:37:27,522 - INFO - hindsight_api.config - Graph retriever: bfs
/app/api/.venv/lib/python3.11/site-packages/fastmcp/server/server.py:285: DeprecationWarning: Providing `stateless_http` when creating a server is deprecated. Provide it when calling `run` or as a global setting instead.
  self._handle_deprecated_settings(
2026-01-23 09:37:34,750 - INFO - hindsight_api.api - HTTP REST API enabled
2026-01-23 09:37:34,750 - INFO - hindsight_api.api - MCP server enabled at /mcp/
Starting Hindsight API...
  URL: http://0.0.0.0:8888
  Database: pg0
  LLM: ollama / gemma3:270m
  Embeddings: local
  Reranker: local
  MCP: enabled at /mcp

INFO:     Started server process [7]
INFO:     Waiting for application startup.
2026-01-23 09:37:34,919 - INFO - docket.worker - Starting worker 'f64cf864571c#7' with the following tasks:
2026-01-23 09:37:34,919 - INFO - docket.worker - * trace(message: str, ...)
2026-01-23 09:37:34,919 - INFO - docket.worker - * fail(message: str, ...)
2026-01-23 09:37:34,919 - INFO - docket.worker - * sleep(seconds: float, ...)
2026-01-23 09:37:34,920 - INFO - mcp.server.streamable_http_manager - StreamableHTTP session manager started
2026-01-23 09:37:34,920 - INFO - hindsight_api.api - MCP lifespan started
2026-01-23 09:37:34,920 - INFO - root - Metrics initialized - available at /metrics endpoint
2026-01-23 09:37:34,922 - INFO - hindsight_api.engine.llm_wrapper - Verifying LLM: provider=ollama, model=gemma3:270m, base_url=http://host.docker.internal:11434/v1...
2026-01-23 09:37:36,167 - INFO - hindsight_api.pg0 - Starting embedded PostgreSQL (name=hindsight, port=auto)...
2026-01-23 09:37:36,414 - INFO - hindsight_api.engine.llm_wrapper - LLM verified: ollama/gemma3:270m