models : fix n_text_ctx taken from max_length instead of max_target_positions - #3987
Open
axiom-of-choice wants to merge 1 commit into
Open
Conversation
…ositions n_text_ctx sizes the decoder positional embedding, so it must be the architecture (max_target_positions), not a generation default. Both are 448 on OpenAI's checkpoints, so this only shows on a fine-tune that sets max_length, where the header disagrees with the tensor and the model fails to load. Subsumes ggml-org#2477 and ggml-org#2840, which used max_target_positions only as a fallback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3986.
n_text_ctxsizes the decoder positional embedding (src/whisper.cpp:1797), so it must bemax_target_positions, the row count of thedecoder.positional_embeddingtensor this scriptwrites. It was reading
max_length, a generation default. The audio side of the header alreadyuses
max_source_positions, andconvert-h5-to-coreml.pyalready readsmax_target_positions.Both fields are 448 on OpenAI's checkpoints, so this is invisible until a fine-tune sets
max_length.Measured with a synthetic checkpoint (
max_target_positions=1024,max_length=448) andwhisper-clibuilt from this tree:decoder.positional_embedding has wrong size, load fails.binbyte-identical#2477 and #2840 used
max_target_positionsas a fallback for a missing or non-intmax_length;this makes it the source, so both are subsumed. Their coercion is dropped as unreachable:
from_pretrainedrejects a non-int, negative or weights-disagreeing value before the header iswritten.
Not verified against a real fine-tune. #2778 may be the same bug but has no
config.jsontoconfirm it.