perf(gradient): reuse exact visible-content token counts across turns #1737

Parent incident: #1708

Priority: Early independent fix

Confirmed code gap

The reported provenance cache is warm (2,300 hits / 2 misses), but gradient's estimateMessage/estimateParts repeatedly call synchronous BPE for visible text and tool outputs. estimateTokens caches tokenizer instances, not string counts. Compression logged about 18.9 s after prewarm in the new trace; this identifies an expensive stage, not proof that BPE owns all of it.

Baseline: gradient estimators and fitting, tokenizer, existing derived-count cache.

Implementation plan

1. Measure count calls, unique input bytes and BPE time in the gradient fitting paths, using the current benchmark fixture and representative large outputs.
2. Introduce a bounded exact-count cache for visible content and generated/stripped variants. Key it by the actual counted bytes, encoding/tokenizer version and relevant representation policy; message ID alone is insufficient.
3. Reuse immutable part/message counts and already validated source-checkpoint metadata across fitting stages and appended turns. Count changed/new content once rather than resumming by retokenizing strings.
4. Persist only bounded disposable hashes/counts when appropriate, reusing the source-window/cache durability rules. Preserve no-store behavior, ownership and database-generation guards.
5. Retain the existing budget/calibration policy and exact fallback on cache miss. Cached per-part arithmetic must match current counting boundaries; BPE counts are not generally additive across arbitrary concatenation boundaries.

Acceptance criteria

• Warm continuation and fresh-process checkpoint resume avoid BPE over unchanged retained content.
• Token estimates, selected layer/window and serialized model request match the uncached reference across compression stages.
• Changed output, reasoning/provenance, stripping, encoding or tokenizer version invalidates the appropriate counts.
• Cache capacity is bounded by entries/bytes and eviction cannot break correctness; no hidden full-history retention.
• Report CPU/time and BPE-byte reduction on the same workload, including a small-session control.

Dependencies and boundary

Independent extension of merged #1710/#1726 and #1717. One caching PR; no tokenizer replacement, heuristic counting, generic streaming parser or change to context budgets. If cold misses remain a bottleneck, record that separately.

Hard prerequisites

None. This issue can be implemented independently.