feat(cost): cache-aware token and cost accounting; meter DeepSeek and OpenRouter

docs/guides/cost-tracking.md

@@ -20,13 +20,23 @@ Activates when a provider's key env var is set:
 | OpenAI | `OPENAI_API_KEY` | `OPENAI_BASE_URL` |
 | Mistral | `MISTRAL_API_KEY` | `MISTRAL_BASE_URL` |
 | Moonshot (Kimi) | `MOONSHOT_API_KEY` | `MOONSHOT_API_BASE` |
+| DeepSeek | `DEEPSEEK_API_KEY` | `DEEPSEEK_BASE_URL`, and `DEEPSEEK_SEARCH_BASE_URL` (the DeepSeek harness's web search) |
+| OpenRouter | `OPENROUTER_API_KEY` | `OPENROUTER_BASE_URL` and `OPENROUTER_API_BASE` (both, one port) |
 
 **Claude Code** (`claude -p`) is always intercepted (OAuth, no key env var). With no key
 set and no always-intercept provider, cost tracking is a no-op.
 
+A base-URL var you set yourself is where the proxy forwards to, so a solution pointed at
+a gateway or a compatible endpoint is still metered — see the Claude Code note below.
+
 ## Provider support
 
 - **Anthropic, OpenAI, Claude Code** — work out of the box (the SDK auto-reads the base URL).
+- **Claude Code on an Anthropic-compatible endpoint** — set `ANTHROPIC_BASE_URL` to it as
+  that vendor's guide says (DeepSeek: `https://api.deepseek.com/anthropic`; OpenRouter:
+  `https://openrouter.ai/api`). trap forwards there, keeps the path, and prices each call
+  at the rates of the model the endpoint reports — DeepSeek's for `deepseek-flash`, not
+  Anthropic's.
 - **Mistral** — needs one line, as its SDK doesn't auto-read the env var:
   ```python
   client = Mistral(api_key=os.environ.get("MISTRAL_API_KEY"),
@@ -37,13 +47,46 @@ set and no always-intercept provider, cost tracking is a no-op.
   No standard SDK auto-reads a Moonshot base-URL var, so a solution calling the OpenAI SDK
   directly should read the same var explicitly
   (`base_url=os.environ.get("MOONSHOT_API_BASE")`).
+- **DeepSeek** — the DeepSeek harness reads `DEEPSEEK_BASE_URL` (from the environment it
+  inherits, not a `.env` file) and works out of the box. Its `web_search` tool is a second
+  billed call, in Anthropic format, to `DEEPSEEK_SEARCH_BASE_URL`; trap redirects that too
+  and files it under `deepseek`. A direct OpenAI-SDK caller reads
+  `base_url=os.environ.get("DEEPSEEK_BASE_URL")`.
+- **OpenRouter** — clients disagree on the var: OpenRouter's TypeScript SDK reads
+  `OPENROUTER_BASE_URL`, litellm (and so Aider) reads `OPENROUTER_API_BASE`. trap points
+  both at the same port, so either works out of the box; a direct OpenAI-SDK caller reads
+  `base_url=os.environ.get("OPENROUTER_BASE_URL")`.
+- **Codex CLI** — not metered out of the box: current releases ignore `OPENAI_BASE_URL`
+  (the base URL is the `openai_base_url` config key), and prefer a WebSocket transport the
+  proxy does not read.
 - **AWS Bedrock / Google Vertex** — unsupported (SDK-level auth, no redirectable base URL); the run still works, cost is just absent.
 
 ## In report.json
 
-Each case carries a `cost` object — per-model breakdown plus aggregate `prompt_tokens`,
-`completion_tokens`, `cost_usd`, `calls` — or `null` if the solution made no LLM calls.
-The terminal table shows per-case aggregates; per-model detail lives in `report.json`.
+Each case carries a `cost` object — or `null` if the solution made no LLM calls. Its
+`by_model` list has one entry per (provider, model), each with:
+
+| Field | Meaning |
+|---|---|
+| `prompt_tokens` | **uncached** input tokens |
+| `cache_read_tokens` | input tokens read from the provider's prompt cache |
+| `cache_write_tokens` | input tokens written to the cache (every TTL) |
+| `completion_tokens` | output tokens (reasoning included) |
+| `cost_usd` | what the calls cost, cache included |
+| `cache_cost_usd` | the part of `cost_usd` spent on cache reads and writes |
+| `calls` | API calls |
+
+The four token counts are **disjoint**: the input a model was sent is
+`prompt_tokens + cache_read_tokens + cache_write_tokens`. That is Anthropic's own split;
+vendors that report cached tokens as part of the prompt (OpenAI, DeepSeek, OpenRouter,
+Moonshot, Mistral) have them taken out of `prompt_tokens`, so the counts compare across
+vendors. It is also the split the site uses (`input` / `cache_read` / `cache_creation`).
+`cache_cost_usd` is inside `cost_usd`, never on top of it. The `cost` object also sums
+every field over its models; the terminal table shows those per-case sums, with
+`cache_rd` / `cache_wr` columns when any case used the cache.
+
+`cost_usd` and `cache_cost_usd` are `null` when unknown — see below. A report written
+before cache accounting loads with cache counts of 0 and `cache_cost_usd` `null`.
 
 ## Pricing
 
@@ -66,15 +109,51 @@ is ordered specific-first, so the first matching prefix is the most specific one
 server — no CLI release. `TRAPSTREET_URL` redirects the fetch (e.g. at UAT),
 `TRAP_PRICING_CACHE` relocates the cache file.
 
-Models absent everywhere (or local servers like Ollama/vLLM) still get token
-counts, but `cost_usd` is `null` — an unknown cost, deliberately distinct from `0.0`.
+### Cache rates
+
+Cache prices are served like every other price. A row may carry `cache_read_per_mtok`,
+`cache_write_per_mtok` (a standard write — Anthropic's 5-minute TTL) and
+`cache_write_1h_per_mtok` (Anthropic's 1-hour write). All three are optional, and a client
+that predates them ignores them. The CLI parses and computes; it holds no prices of its own.
+
+**Interim fallback.** Until the site serves those columns, a rate a row lacks is filled from
+the vendor's documented multiple of the row's input rate. The table lives in
+`src/trap/cost/interim_cache_rates.py`, each entry with its source, and is deleted once the
+site serves the rates. It fills per rate — a rate the row serves always wins — and only for
+native model ids: a routed id (`vendor/model`, e.g. on OpenRouter) is priced from served
+rates or not at all.
+
+| Model family | Cache read | Cache write |
+|---|---|---|
+| Claude | 0.1x (0.025x on Fable 5.1 / Mythos 5.1) | 1.25x (5-minute), 2x (1-hour) |
+| GPT-5.6+, GPT-6 | 0.1x | 1.25x |
+| GPT-5 – 5.5 | 0.1x | no write fee |
+| GPT-4.1, o3, o4-mini | 0.25x | no write fee |
+| GPT-4o, o1, o3-mini | 0.5x | no write fee |
+| `-pro` models (gpt-5-pro … gpt-5.5-pro, o1-pro, o3-pro) | 1x (no cached discount) | no write fee |
+| DeepSeek (`deepseek-flash`, `-v4-flash`, `-v4-pro`) | 0.02x, 0.02x, 1/30x | no write fee |
+| Kimi (`kimi-k3`, `-k2.7-code`, `-k2.6`) | 0.1x, 0.2x, 0.168x | no write fee |
+
+**Unknown is never priced as zero, or as full input.** `cost_usd` is `null` when the model
+is absent from the table (or a local server like Ollama/vLLM), and also when a call used
+the cache on a model with no cache rate — none served, and no interim multiple. Either would
+be a wrong number, and a wrong number is worse than a missing one; the token counts are
+recorded either way.
+
+Not modelled: DeepSeek's off-peak discount (it halves both rates, so the ratio holds; the
+row's base rate decides), long-context and batch/flex/priority tiers, and the TTL of cache
+writes OpenRouter reports (one total, priced at the standard write rate).
 
 ## Proxy internals
 
-- **One port per provider.** trap starts a separate proxy server per active provider, each
-  bound to a random localhost port (port `0` → OS-assigned). Because a port serves exactly one
-  provider, the proxy never has to detect the provider per request. No TLS interception is
-  needed — the proxy just forwards over HTTPS and tees the response to read `usage`.
+- **One port per provider endpoint.** trap starts a separate proxy server per active registry
+  entry, each bound to a random localhost port (port `0` → OS-assigned). Because a port serves
+  exactly one endpoint, the proxy never has to detect the provider per request. No TLS
+  interception is needed — the proxy just forwards over HTTPS and tees the response to read
+  `usage`.
+- **Streams are metered from their last word.** Usage in a stream is cumulative, so the proxy
+  keeps the last usage it sees (Anthropic's `message_delta` restates `message_start` per field;
+  the Responses API nests it in `response.completed`), never a sum.
 - **Upstream URLs compensate for SDK path quirks.** SDKs differ in whether they keep the `/v1`
   path prefix when the base URL is overridden, so each provider's configured upstream must
   match:
@@ -83,3 +162,7 @@ counts, but `cost_usd` is `null` — an unknown cost, deliberately distinct from
   - Mistral SDK keeps `/v1` → upstream `https://api.mistral.ai` (no suffix)
   - Moonshot (Kimi), via the OpenAI SDK, drops `/v1` → upstream `https://api.moonshot.ai/v1`
     (a `.cn` account sets `MOONSHOT_API_BASE=https://api.moonshot.cn/v1`)
+  - The DeepSeek harness appends `/chat/completions` with no `/v1` → upstream
+    `https://api.deepseek.com`; its web search appends `/messages` → upstream
+    `https://api.deepseek.com/anthropic/v1`
+  - OpenRouter clients' bases carry `/api/v1` → upstream `https://openrouter.ai/api/v1`

docs/reference/evaluation-api.md

@@ -87,9 +87,13 @@ POST /api/v2/runs/{run}/context
   "timing": { "started_at": "…", "finished_at": "…", "solver_ms": 41200 },
   "usage": { "by_model": [ { "model": "gpt-5", "provider": "openai",
                              "input": 12000, "output": 900, "calls": 12,
+                             "cache_read": 56000, "cache_creation": 0,
                              "cost_usd_reported": 0.031 } ] } }
 ```
 
+`input` is the uncached input; tokens read from or written to the prompt cache go in
+`cache_read` / `cache_creation`, not in `input`.
+
 ### 2. Claim a case
 
 ```http

src/trap/cost/interim_cache_rates.py

@@ -0,0 +1,91 @@
+"""INTERIM cache-rate fallback — delete this module once ``GET /api/pricing`` serves cache rates.
+
+The site's price table is the one source of cache prices, as of every other price: a row's
+``cache_read_per_mtok`` / ``cache_write_per_mtok`` / ``cache_write_1h_per_mtok``. Until the
+site serves them, a rate a row lacks is taken here, as the vendor's documented multiple of
+that row's own input rate. This is vendor pricing copied into the CLI, and it goes stale
+when a vendor reprices — which is why it is a stopgap, not a second source:
+
+- a rate the row serves always wins; this only fills a missing one;
+- native model ids only — a routed id ("vendor/model", OpenRouter's shape) gets nothing
+  here, since what a route charges is only known to the served table;
+- a family absent here has no known cache price, and cached tokens on it price as unknown.
+
+To retire it: once the site serves the three columns, delete this file and
+``tests/test_interim_cache_rates.py``, and drop the ``fallback`` argument from
+:meth:`trap.cost.pricing.PriceRow.cache_rates`.
+"""
+
+from __future__ import annotations
+
+from typing import NamedTuple
+
+
+class CacheMultipliers(NamedTuple):
+    """A vendor's cache prices as multiples of the model's own input rate."""
+
+    read: float
+    write: float  # a standard cache write (Anthropic: the 5-minute TTL)
+    write_1h: float  # a 1-hour cache write; only Anthropic reports these, elsewhere == write
+
+
+def _reads_at(read: float, write: float = 1.0) -> CacheMultipliers:
+    """A vendor with no 1-hour TTL (every one but Anthropic); ``write=1.0`` = no write fee."""
+    return CacheMultipliers(read=read, write=write, write_1h=write)
+
+
+_ANTHROPIC = CacheMultipliers(read=0.1, write=1.25, write_1h=2.0)
+_ANTHROPIC_5_1 = CacheMultipliers(read=0.025, write=1.25, write_1h=2.0)
+
+# Documented multiples as of 2026-09-11, per model family. First prefix wins, so a more
+# specific family precedes the one it extends.
+_MULTIPLES: tuple[tuple[str, CacheMultipliers], ...] = (
+    # https://platform.claude.com/docs/en/about-claude/pricing — "5m cache writes: 1.25x base
+    # input price · 1h cache writes: 2x · Cache hits & refreshes: 0.1x (0.025x on Claude Fable
+    # 5.1 and Claude Mythos 5.1)".
+    ("claude-fable-5-1", _ANTHROPIC_5_1),
+    ("claude-mythos-5-1", _ANTHROPIC_5_1),
+    ("claude-", _ANTHROPIC),
+    # https://developers.openai.com/api/docs/pricing (cached ÷ input rate) and the prompt-caching
+    # guide: "For GPT-5.6 and later, cache writes cost 1.25x the standard, uncached input-token
+    # rate... subsequent reads cost only 0.1x"; every earlier model has "No additional
+    # cache-write charge". The -pro models have no cached price at all ("GPT-5.5 Pro does not
+    # offer a cached input discount"), so a cached token there is plain input. gpt-realtime is
+    # left out: it prices cached audio (0.0125x) and text (0.1x) apart.
+    ("gpt-6", _reads_at(0.1, write=1.25)),
+    ("gpt-5.6", _reads_at(0.1, write=1.25)),
+    ("gpt-5-pro", _reads_at(1.0)),
+    ("gpt-5.2-pro", _reads_at(1.0)),
+    ("gpt-5.4-pro", _reads_at(1.0)),
+    ("gpt-5.5-pro", _reads_at(1.0)),
+    ("gpt-5", _reads_at(0.1)),
+    ("gpt-4.1", _reads_at(0.25)),
+    ("gpt-4o", _reads_at(0.5)),
+    ("o1-pro", _reads_at(1.0)),
+    ("o3-pro", _reads_at(1.0)),
+    ("o1", _reads_at(0.5)),
+    ("o3-mini", _reads_at(0.5)),
+    ("o3", _reads_at(0.25)),
+    ("o4-mini", _reads_at(0.25)),
+    # https://api-docs.deepseek.com/quick_start/pricing (effective 2026-09-10), USD/Mtok input
+    # cache hit vs miss: deepseek-flash 0.006 vs 0.30, deepseek-v4-pro 0.044 vs 1.32 (off-peak
+    # halves both, keeping the ratio); deepseek-v4-flash is "billed at the Flash price". A miss
+    # is plain input — DeepSeek charges no cache write.
+    ("deepseek-v4-pro", _reads_at(0.044 / 1.32)),
+    ("deepseek-v4-flash", _reads_at(0.006 / 0.30)),
+    ("deepseek-flash", _reads_at(0.006 / 0.30)),
+    # https://platform.kimi.ai/docs/pricing/chat, USD/Mtok cache hit vs miss: kimi-k3 0.30 vs
+    # 3.00, kimi-k2.7-code 0.19 vs 0.95 (-highspeed 0.38 vs 1.90), kimi-k2.6 0.16 vs 0.95; no
+    # cache write or storage fee.
+    ("kimi-k3", _reads_at(0.1)),
+    ("kimi-k2.7-code", _reads_at(0.2)),
+    ("kimi-k2.6", _reads_at(0.16 / 0.95)),
+)
+
+
+def interim_multiples(model: str) -> CacheMultipliers | None:
+    """The documented multiples for a native ``model`` id (lowercased), or None — always
+    None for a routed ``vendor/model`` id."""
+    if "/" in model:
+        return None
+    return next((multiples for prefix, multiples in _MULTIPLES if model.startswith(prefix)), None)

src/trap/cost/pricing.py

@@ -17,6 +17,12 @@
 models at 0 or a stale predecessor's rate): a price update is a data change on the
 server, and :meth:`PriceTable.cost_of` reports an unknown model as ``None``, never
 as a wrong number.
+
+Cache rates are served the same way: ``cache_read_per_mtok``, ``cache_write_per_mtok`` and
+``cache_write_1h_per_mtok`` on a row. A rate a row does not serve yet is filled from the
+INTERIM fallback in :mod:`trap.cost.interim_cache_rates` (a stopgap, deleted once the site
+serves them), and cached tokens neither can price make the call's cost unknown rather than
+guessed. This module parses and computes; it holds no prices of its own.
 """
 
 from __future__ import annotations
@@ -26,26 +32,60 @@
 import time
 from importlib.resources import files
 from pathlib import Path
-from typing import Literal
+from typing import Literal, NamedTuple
 
 import httpx
 from pydantic import BaseModel, ValidationError, field_validator
 
+from trap.cost.interim_cache_rates import CacheMultipliers, interim_multiples
+from trap.models.cost import CallUsage
+
+
+class CallCost(NamedTuple):
+    """What one call cost, in USD. ``cache_usd`` is the part of ``usd`` spent on cache
+    reads and writes — included in ``usd``, not on top of it."""
+
+    usd: float
+    cache_usd: float
+
 
 class PriceRow(BaseModel):
     """One priced model-id prefix. Extra fields on the wire (``provider``, ``note``, …)
-    are ignored — only the prefix and its two rates are used."""
+    are ignored — only the prefix and its rates are used. The cache rates are optional:
+    a table served before the site priced cache prices without them, and an older client
+    ignores them."""
 
     model_prefix: str
     input_per_mtok: float
     output_per_mtok: float
+    cache_read_per_mtok: float | None = None
+    cache_write_per_mtok: float | None = None  # a standard (Anthropic: 5-minute) cache write
+    cache_write_1h_per_mtok: float | None = None  # a 1-hour cache write (Anthropic)
 
     @field_validator("model_prefix")
     @classmethod
     def _normalise(cls, value: str) -> str:
         # matched against a lowercased model id, so store the prefix lowercased
         return value.lower()
 
+    def cache_rates(
+        self, fallback: CacheMultipliers | None
+    ) -> tuple[float | None, float | None, float | None]:
+        """(read, write, 1-hour write) in USD per Mtok. Each is the rate the row serves, else
+        the interim ``fallback`` multiple of the row's input rate, else None (unknown)."""
+        base = self.input_per_mtok
+
+        def rate(served: float | None, multiple: float | None) -> float | None:
+            if served is not None:
+                return served
+            return None if multiple is None else multiple * base
+
+        return (
+            rate(self.cache_read_per_mtok, fallback and fallback.read),
+            rate(self.cache_write_per_mtok, fallback and fallback.write),
+            rate(self.cache_write_1h_per_mtok, fallback and fallback.write_1h),
+        )
+
 
 class PriceTable(BaseModel):
     """The /api/pricing wire shape, shared by the server response, the on-disk cache, and
@@ -74,19 +114,32 @@ def parse(cls, payload: object) -> PriceTable | None:
     def is_fresh(self, ttl_seconds: float) -> bool:
         return time.time() - self.fetched_at < ttl_seconds
 
-    def cost_of(self, prompt_tokens: int, completion_tokens: int, model: str | None) -> float | None:
-        """The USD cost of one API call against this table, or ``None`` when the model is
-        unknown or absent — an unknown cost is not a zero cost, and is reported as JSON null
-        rather than a misleading number. First matching prefix wins (order is load-bearing;
-        see :meth:`PriceCatalogue._default`)."""
-        if model is None:
+    def cost_of(self, usage: CallUsage) -> CallCost | None:
+        """What one API call cost against this table, or ``None`` when that is unknown —
+        the model is absent or unpriced, or it has cached tokens at a rate neither its row
+        nor its vendor's documented multipliers give. An unknown cost is not a zero cost,
+        and is reported as JSON null rather than a misleading number. First matching prefix
+        wins (order is load-bearing; see :meth:`PriceCatalogue._default`)."""
+        if usage.model is None:
+            return None
+        name = usage.model.lower()
+        row = next((row for row in self.prices if name.startswith(row.model_prefix)), None)
+        if row is None:
             return None
-        name = model.lower()
-        for row in self.prices:
-            if name.startswith(row.model_prefix):
-                per_mtok = prompt_tokens * row.input_per_mtok + completion_tokens * row.output_per_mtok
-                return per_mtok / 1_000_000
-        return None
+        read, write, write_1h = row.cache_rates(interim_multiples(name))
+        cache = 0.0
+        for tokens, rate in (
+            (usage.cache_read_tokens, read),
+            (usage.cache_write_tokens - usage.cache_write_1h_tokens, write),
+            (usage.cache_write_1h_tokens, write_1h),
+        ):
+            if not tokens:
+                continue
+            if rate is None:
+                return None
+            cache += tokens * rate
+        plain = usage.prompt_tokens * row.input_per_mtok + usage.completion_tokens * row.output_per_mtok
+        return CallCost(usd=(plain + cache) / 1_000_000, cache_usd=cache / 1_000_000)
 
 
 class PriceCatalogue:
@@ -160,7 +213,8 @@ def _write_cache(cls, table: PriceTable) -> None:
         try:
             path = cls._cache_path()
             path.parent.mkdir(parents=True, exist_ok=True)
-            path.write_text(table.model_dump_json(indent=2))
+            # exclude_none: a cache rate the server did not give is left out, not written as null
+            path.write_text(table.model_dump_json(indent=2, exclude_none=True))
         except OSError:
             pass