You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Both Whisper-family engines declare .supported(maxTokens: 224), but only one of
them enforces anything.
WhisperKitEngine tokenizes, clamps with clampedPromptTokens (keeping the
prefix, per #164 design D4), and hands the pipeline an exact token array. WhisperCppEngine passes the rendered string through verbatim as a CLI argument:
iflet prompt {
arguments +=["--prompt", prompt]}
There is no tokenizer on that path — the context-calibration spec acknowledges
this ("a conservative character heuristic on the whisper-cli path") — so whenever
the character estimate undershoots the real token count, truncation is entirely
up to whisper-cli's internals. As a port of the same reference decoding
algorithm, it most likely keeps the suffix:
prompt_tokens[-(self.n_ctx//2-1):]
which is the opposite of what #164 documents as the project's clamp policy.
Impact
Two backends declaring the same 224-token capability behave differently under
overflow, and one behaves opposite to the documented policy.
The context-calibration sentence "the clamp SHALL preserve the highest-priority
items" has no enforcement point and no test on this path
(PipelineWiringTests' clamp test only exercises WhisperKitEngine).
Type
bug
Problem
Both Whisper-family engines declare
.supported(maxTokens: 224), but only one ofthem enforces anything.
WhisperKitEnginetokenizes, clamps withclampedPromptTokens(keeping theprefix, per #164 design D4), and hands the pipeline an exact token array.
WhisperCppEnginepasses the rendered string through verbatim as a CLI argument:There is no tokenizer on that path — the
context-calibrationspec acknowledgesthis ("a conservative character heuristic on the whisper-cli path") — so whenever
the character estimate undershoots the real token count, truncation is entirely
up to whisper-cli's internals. As a port of the same reference decoding
algorithm, it most likely keeps the suffix:
which is the opposite of what #164 documents as the project's clamp policy.
Impact
overflow, and one behaves opposite to the documented policy.
context-calibrationsentence "the clamp SHALL preserve the highest-priorityitems" has no enforcement point and no test on this path
(
PipelineWiringTests' clamp test only exercisesWhisperKitEngine).which is why this is filed rather than fixed inside context 的 token 預算應是 per-model 參數而非全域寫死 200:7 個 engine 只有 2 個吃 prompt,其餘照樣截斷且照印 injected(N) #164.
Notes
whisper-cli's suffix-keeping becomes correct for free, and the WhisperKit
clamp is the one that would need to follow.