Skip to content

WhisperCppEngine enforces no prompt clamp, so its overflow behaviour contradicts the documented direction #176

Description

@kiki830621

Type

bug

Problem

Both Whisper-family engines declare .supported(maxTokens: 224), but only one of
them enforces anything.

WhisperKitEngine tokenizes, clamps with clampedPromptTokens (keeping the
prefix, per #164 design D4), and hands the pipeline an exact token array.
WhisperCppEngine passes the rendered string through verbatim as a CLI argument:

if let prompt {
    arguments += ["--prompt", prompt]
}

There is no tokenizer on that path — the context-calibration spec acknowledges
this ("a conservative character heuristic on the whisper-cli path") — so whenever
the character estimate undershoots the real token count, truncation is entirely
up to whisper-cli's internals. As a port of the same reference decoding
algorithm, it most likely keeps the suffix:

prompt_tokens[-(self.n_ctx // 2 - 1):]

which is the opposite of what #164 documents as the project's clamp policy.

Impact

Notes

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions