Skip to content

Reorder PromptRenderer so the highest-value context lands nearest the decode boundary #174

Description

@kiki830621

Type

refactor

Problem

PromptRenderer.render emits classes in the order names → terms → phrases, so
the highest-value content sits at the front of every rendered prompt.

Whisper conditions on the prompt through the <|startofprev|> channel
immediately preceding <|startoftranscript|>, and content nearest that boundary
is widely reported to carry outsized influence. OpenAI's reference decoder makes
the same judgement structurally — when an over-long initial_prompt must be
truncated it keeps the tail:

# whisper/decoding.py, DecodingTask._get_initial_tokens
tokens = [self.tokenizer.sot_prev] + prompt_tokens[-(self.n_ctx // 2 - 1):] + tokens

So the renderer currently puts the names — the exact content a context directory
exists to inject — in the position with the least conditioning weight, in every
prompt, not only when truncation happens.

Background

#164 reversed the engine-side clamp (design D4) to keep the prefix, because the
old suffix-keeping clamp dropped names outright on overflow. That fixed the
outright-drop, but it fixed it on the content-priority axis while the recency
argument lives on the position axis. Four independent review lenses judged D4 in
#164's verify round 1; none argued for reverting it, and three independently
derived the same durable remedy, which is also what PipelineWiringTests.swift
already says in a comment but does not implement:

reorder the renderer so the highest-value items land at the tail, not re-flip
the clamp against its own priority ordering

Expected

Highest-value items land nearest the decode boundary and survive truncation.
Concretely: renderer emits phrases → terms → names (or otherwise places names
last), while the clamp keeps whichever end now holds the names.

Both mechanisms then agree, and the context-calibration spec sentence "the
clamp SHALL preserve the highest-priority items" stays satisfied.

Notes

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions