[Feature] Prefix stability validator — detect dynamic content bleeding into cached prefixes #48

Problem

Prompt caching breaks when dynamic content appears before the cache_control breakpoint. This is the single most common cause of 0% cache hit rates in production, and it's invisible without tooling.

Examples of dynamic content that silently invalidates caches:

# Looks fine. Breaks caching.
system_prompt = f"""
You are a helpful assistant.
Request ID: {request_id}          # changes every request
Current time: {datetime.now()}    # changes every request
User: {user.name}                 # changes per user
"""

The cache_control breakpoint is set after the system prompt. The prefix hash changes on every request. Hit rate: 0%. No error, no warning.

Proposed: distill cache validate

A static analysis + runtime check that detects prefix instability.

Static analysis (pre-flight)

Scan message templates for patterns that produce non-deterministic output before a cache_control marker:

• datetime, time.time(), uuid, random
• f-string interpolation of variables not in a fixed allowlist
• Dictionary iteration (non-deterministic ordering in some Python versions)

Runtime check

Compare prefix hashes across N consecutive requests to the same call site. If the hash changes on every request, emit a warning with a token diff showing what changed:

WARNING cache-prefix-unstable src/agent/reviewer.py:201
  Prefix hash changed on 890/890 requests (0% stability)
  First differing byte at offset 142:
  - "Request ID: a3f2c1d4"
  + "Request ID: b7e9f2a1"
  Likely cause: dynamic interpolation before cache_control breakpoint

Integration point

This fits naturally as a validation step in distill analyze or as a new distill cache validate subcommand. Could also run as a CI check — fail the build if prefix stability drops below a threshold.

Why this fits Distill

Distill already has a distill analyze command that inspects context for duplicates. Prefix stability is the same class of analysis applied to the caching layer — find what's causing unnecessary token spend before it reaches the LLM.

Scope

• Python SDK instrumentation
• Static AST scanner for common dynamic patterns
• Runtime prefix hash comparison with diff output
• CI-friendly exit codes