billing: 1-hour TTL cache writes are billed at the 5-minute 1.25x rate, undercharging by 37.5%

[mastermanas805]

While reading through the cost math I noticed EffectiveInputCost multiplies all cache-creation tokens by 1.25. That's Anthropic's 5-minute cache-write rate, but a 1-hour write costs 2x. We do forward ttl: "1h" when a client sends it, and nothing downstream distinguishes the two, so those turns get billed at about 62% of what Anthropic actually charges us.

It's not only a telemetry number either, since this feeds internal/billing/service.go:292.

In internal/router/catalog/cost.go:

return (float64(fresh) +
	float64(cacheCreation)*1.25 +          // 5m rate, applied to every cache write
	float64(cacheRead)*p.EffectiveCacheReadMultiplier()) / 1_000_000 * pricePer1M

There's no cache-write field on Pricing, and the function only takes a single cacheCreation int, so there's currently nowhere to put the distinction even if a caller knew it.

The prompt caching docs spell the rates out:

• 5-minute cache write tokens are 1.25 times the base input tokens price
• 1-hour cache write tokens are 2 times the base input tokens price
• Cache read tokens are 0.1 times the base input tokens price

What I saw

I ran this against a local stack with a mock Anthropic upstream that reports a cache write of 1M tokens, all on the 1h tier. Routing went through normally (X-Router-Decision: cluster:v0.75 ... model=claude-opus-5 provider=anthropic), and this is the row we stored:

decision_model | provider  | input_tokens | cache_creation_tokens | actual_input_cost_usd
claude-opus-5  | anthropic | 10           | 1000000               | 6250050

Working it through: (10 x $5/MTok) + (1,000,000 x 1.25 x $5/MTok) is $6.25005, which matches the stored 6250050 micros exactly. At the 1h rate it should have been $10.00005. So we're short $3.75 on that single turn, and because the fresh-token part reconciles precisely there's no rounding hiding in there — the entire gap is the multiplier.

Worth noting cache_creation_tokens only stores the total, so the row can't be re-costed later once the split is gone.

How a request actually ends up here

The Claude Agent SDK defaulted to a 1-hour TTL at one point (#188), so clients really do send this. On our side validateAnthropicCacheControl accepts ttl: "1h" and even has an ordering rule specific to it.

The mock logged what it received as cache_control_ttls=['5m(default)', '1h'] — the client's 1h breakpoint on system, plus the 5m one we inject on the last message block. Mixed-TTL requests look like the normal case rather than an edge case.

Anthropic does send the breakdown back, we just never look at it: grep -rn "ephemeral_5m\|ephemeral_1h" --include="*.go" . comes back empty, and every call site reads cache_creation_input_tokens on its own.

One caveat on the above: a mock only proves how we cost a payload, not that Anthropic emits that exact payload. For that part I'm going off the documented field names and multipliers.

Fixing it

The response already carries the split, and the aggregate we read today is just the sum of the two:

"cache_creation": {"ephemeral_5m_input_tokens": 148, "ephemeral_1h_input_tokens": 100}

So it's roughly: pull ephemeral_1h_input_tokens out alongside the aggregate, give EffectiveInputCost somewhere to put it, and cost it as 5m x 1.25 + 1h x 2.0. The one thing I'd be careful about is falling back to 1.25x whenever the breakdown is missing — Bedrock and Vertex payloads may only carry the aggregate, and assuming those are all 5m keeps today's behaviour instead of swinging us into over-billing.

A couple of things I didn't want to decide on my own: whether the split is worth persisting in telemetry (that's a migration) or is only needed at debit time, and whether these multipliers belong on Pricing next to CacheReadMultiplier instead of sitting as constants in cost.go. Let me know which way you'd prefer and I'm glad to put up a PR.

[vignesh-chaturvedi]

Both of your open questions have a precedent in the code that I think settles them.

Multipliers on Pricing, not constants. Pricing already carries
CacheReadMultiplier with a DefaultCacheReadMultiplier fallback and an
EffectiveCacheReadMultiplier() accessor. Write multipliers mirror that
one-for-one. There's a second argument for it: the 1.25 is applied to every
provider today, but in internal/observability/otel/usage.go only the Anthropic
branch ever sets cacheCreation —> OpenAI and Google set cacheRead only. So the
multiplier is already Anthropic-only by accident. Putting it on the binding makes
that explicit instead of load-bearing coincidence.

Debit-time first, telemetry persistence as a follow-up. You're right that
cache_creation_tokens storing only the total means rows can't be re-costed
later, and that's worth fixing. But the migration pulls in db/queries/, sqlc
regeneration, the analytics export schema and the check_migrations gate, which
is a lot of surface next to a multiplier fix. Two PRs seems better than one that
has to argue both.

Your 1.25 fallback when the breakdown is absent is the right call —> Bedrock and
Vertex may only carry the aggregate, and assuming 5m keeps today's behavior rather
than swinging into over-billing.

Are you still planning to put this up? I've started on the debit-time half and am
happy to either hand it over or open it with you credited - whichever you prefer.
Just don't want us both writing the same patch.