PreferredModels can poison the semantic cache — a hit serves another model's body while headers report the newly-scored model

[rohith500]

Summary

ComputeKnobsHash never incorporates PreferredModels or the winning model.
When a per-installation preference changes which model wins a close scoring
call, the semantic cache doesn't isolate on that — a near-duplicate prompt
can return a cached response body from a different model than the one the
live routing decision (and x-router-model header) reports.

Why this is a bug, not a tradeoff

PR #214 introduced clusterVersion + knobsHash into the cache bucket key
specifically to fix this exact class of problem, and named it explicitly:

Cache fix (latent bug, pre-existing this work)
bucketKey gains clusterVersion + knobsHash. Without this, requests
differing only on knobs share cache buckets…

That's the maintainers' own stated principle: any axis that can change the
routing winner must be part of cache-bucket isolation, or sharing a bucket
across it is a bug. PreferredModels (introduced later, #521) is exactly
such an axis — confirmed below — and was never folded in.

No ADR, comment, or PR discussion argues for excluding preferences from cache
isolation as an intentional tradeoff. "Soft nudge" language in #521 describes
the scorer's quality floor (a preference won't override a clearly-better
model) — it is not a statement that cache correctness may be sacrificed once
a preference does win.

Where

• internal/router/cache/cache.go: bucketKey{format, clusterID, clusterVersion, knobsHash} — no model or preference dimension.
• internal/router/cluster/knobs.go: ComputeKnobsHash(alpha, speedWeight, outputCostRatio, expectedOutputTokens, perModelVerbosity) — no model
  identity, no preferences.
• internal/router/cluster/scorer.go ~618-636: priorityBonus built from
  req.PreferredModels; ~1208-1212: added into blendScoresV2's per-model
  score inside the top-P cluster loop (so a rank-0 bonus of ~0.15 compounds
  across clusters — enough to flip a real close call, not just a tie).
  EffectiveKnobsHash (~754-762) is computed separately and never sees this.
• internal/proxy/service.go writeCachedResponse (~1436-1453): headers are
  intentionally overwritten from the live decision ("x-router-* headers come
  from the live decision so the client sees an accurate routing trace"), but
  the body is written unconditionally from the cached entry. The header
  overwrite is correct on its own; the bug is that the body isn't co-isolated
  with the same decision axis the headers now reflect.

Reproduction (mechanism-level, verified independently twice)

Using the existing scorer test fixtures (newV2BundleForTest, same setup as
TestScorer_PreferredModelSoftNudge) and the real cache.Cache — no mocks:

1. Same prompt, no PreferredModels → wins claude-opus-4-7 (score 0.55).
   EffectiveKnobsHash = 12184787796704120736.
2. Same prompt, PreferredModels = ["claude-haiku-4-5"] → wins
   claude-haiku-4-5 (score 0.60). EffectiveKnobsHash — identical.
3. cache.Store under (1)'s key, cache.Lookup under (2)'s key → hit,
   returns (1)'s (Opus's) body while (2)'s live decision says Haiku.

Verified independently twice against current HEAD (two separate runs,
same result both times).

Severity (honest assessment)

Medium, not critical:

• preferred_models defaults to {} — inert unless an installation sets it.
• Requires a near-duplicate prompt clearing the cache's cosine threshold
  (default 0.95).
• The sticky-pin STAY path bypasses the cache entirely (Metadata == nil
  there) — this only affects the scorer-fresh routing path, not every turn
  of a pinned session.
• Realistic trigger: an installation changes its preference list and replays
  a similar prompt, or two installations sharing the same cache scope
  (same org external_id) have different preferences.

Real correctness bug when it fires — wrong model's output silently served —
but not a cross-org isolation failure and not continuous on organic traffic.

Suggested fix direction

Both patterns exist in the codebase already:

• Gate (like feat: subscription-aware routing (discount covered models by observed quota headroom) #495/router: serve subscription-only below the overdraft floor instead of 402 #747): exclude cacheEligible when
  len(req.PreferredModels) > 0. Simpler, safe, but disables caching
  entirely for any installation using preferences.
• Fold (like feat(router): runtime-tunable routing knobs via v2 bundle format #214/feat(router): bandit foundation — propensity + reward logging, gated explorer #338, via mixModel): incorporate the resolved
  priorityBonus or winning model into EffectiveKnobsHash. Preserves
  cache hit rate for preference-using installations.

Fold is likely preferable given #214's own precedent, but deferring to
maintainer judgment on the tradeoff.

Related (not duplicate)

• feat(router): per-installation model priority ranking (soft preference bonus) #521 — introduced PreferredModels; Bugbot raised this exact gap on the
  PR twice, unaddressed, no human reply.
• feat(router): runtime-tunable routing knobs via v2 bundle format #214 — established the "winner-affecting axis must be in the cache key"
  principle this gap violates.
• feat: subscription-aware routing (discount covered models by observed quota headroom) #495, router: serve subscription-only below the overdraft floor instead of 402 #747 — same eligibility-gating pattern for other model-choice axes.
• feat(router): bandit foundation — propensity + reward logging, gated explorer #338 — same hash-folding pattern (mixModel) for the bandit-explore case.
• Semantic cache key omits cluster version → cross-version response contamination #17 (closed) — earlier instance of the same class of bug.

No open issue or PR currently tracks this.