models/convert-h5-to-ggml.py on master (line 125, unchanged since before #2477) writes the ggml header's n_text_ctx from hparams["max_length"]:
# lines 87-95 — #2477 and #2840: max_target_positions as a FALLBACK for when max_length is MISSING
if "max_length" not in hparams or hparams["max_length"] is None:
hparams["max_length"] = hparams.get("max_target_positions", 448)
...
# line 125 — but whenever max_length EXISTS, it is what goes into the header
fout.write(struct.pack("i", hparams["max_length"])) # n_text_ctxmax_length is a generation default. max_target_positions is the architecture — the row count of model.decoder.embed_positions.weight, the tensor this same script writes a few lines later. Fine-tuning frequently sets max_length to something else, and then the header and the tensor disagree:
whisper_model_load: tensor 'decoder.positional_embedding' has wrong size in model file
whisper_model_load: shape: [512, 448, 1], expected: [512, 1024, 1]
whisper_init_with_params_no_state: failed to load model
error: failed to initialize whisper context
The conversion succeeds and the model will not load.
Why it has never shown up in testing
On OpenAI's own checkpoints max_length == max_target_positions == 448, so reading the wrong field returns the right number. The bug is invisible on every model the script is normally tested against, and appears only on a fine-tune whose training run set a different max_length.
Prior issues — all of them the missing case, which is why this one is still open
Both reach for max_target_positions as a default when max_length is missing, rather than as the source. When max_length is present and wrong for this purpose, neither path fires.
🎯 This may be the cause of #2778 (open)
#2778 reports "it generates the output, however, when trying to load it with whisper.cpp it fails loading" for a fine-tuned non-English checkpoint — the same symptom, from the same population (fine-tunes are where max_length gets set). I cannot confirm it: that issue does not include config.json, the loader's output, or a link to the model.
If @the-reporter-of-2778 is still around, this is a one-command check:
python -c "import json;c=json.load(open('config.json'));print(c.get('max_length'), c.get('max_target_positions'))"If the two numbers differ, this is the cause.
Minimal reproduction
python - <<'PY'
import json, urllib.request
repo = "tarteel-ai/whisper-base-ar-quran" # public, Apache-2.0; any fine-tune where the fields differ
c = json.load(urllib.request.urlopen(f"https://huggingface.co/{repo}/resolve/main/config.json"))
print(repo, "max_length =", c.get("max_length"), "| max_target_positions =", c.get("max_target_positions"))
PY
# -> max_length = 1024 | max_target_positions = 448
git clone https://huggingface.co/tarteel-ai/whisper-base-ar-quran
python models/convert-h5-to-ggml.py ./whisper-base-ar-quran/ ./whisper .
./build/bin/whisper-cli -m ggml-model.bin -f samples/jfk.wav # fails to loadSuggested fix — the small one
n_text_ctx = hparams.get("max_target_positions") or hparams.get("max_length") or 448
fout.write(struct.pack("i", n_text_ctx))Reverse the precedence left by #2477 and #2840: prefer the architecture field, keep max_length as the fallback. One line, and it keeps #2477's, #2840's and #1520's cases working.
And a larger one, offered as a suggestion rather than a condition
Read the shape of the tensor being written — model.model.decoder.embed_positions.weight.shape[0] — and assert it equals the header value. That makes the header a derived number instead of a declared one, so the class of bug disappears rather than being fixed once. The small fix above is complete on its own; this is only if you would rather the header could never disagree with the tensor again.
Workaround for anyone who finds this first
Convert from a copy of the model directory whose config.json has max_length set to max_target_positions. Do not edit the published checkpoint — the value is correct for what it means; it is only being read as something else.
models/convert-h5-to-ggml.pyonmaster(line 125, unchanged since before #2477) writes the ggml header'sn_text_ctxfromhparams["max_length"]:max_lengthis a generation default.max_target_positionsis the architecture — the row count ofmodel.decoder.embed_positions.weight, the tensor this same script writes a few lines later. Fine-tuning frequently setsmax_lengthto something else, and then the header and the tensor disagree:The conversion succeeds and the model will not load.
Why it has never shown up in testing
On OpenAI's own checkpoints
max_length == max_target_positions == 448, so reading the wrong field returns the right number. The bug is invisible on every model the script is normally tested against, and appears only on a fine-tune whose training run set a differentmax_length.Prior issues — all of them the missing case, which is why this one is still open
06a1da9daf: the linked model hasmax_lengthabsent.Both reach for
max_target_positionsas a default whenmax_lengthis missing, rather than as the source. Whenmax_lengthis present and wrong for this purpose, neither path fires.🎯 This may be the cause of #2778 (open)
#2778 reports "it generates the output, however, when trying to load it with whisper.cpp it fails loading" for a fine-tuned non-English checkpoint — the same symptom, from the same population (fine-tunes are where
max_lengthgets set). I cannot confirm it: that issue does not includeconfig.json, the loader's output, or a link to the model.If @the-reporter-of-2778 is still around, this is a one-command check:
python -c "import json;c=json.load(open('config.json'));print(c.get('max_length'), c.get('max_target_positions'))"If the two numbers differ, this is the cause.
Minimal reproduction
Suggested fix — the small one
Reverse the precedence left by #2477 and #2840: prefer the architecture field, keep
max_lengthas the fallback. One line, and it keeps #2477's, #2840's and #1520's cases working.And a larger one, offered as a suggestion rather than a condition
Read the shape of the tensor being written —
model.model.decoder.embed_positions.weight.shape[0]— and assert it equals the header value. That makes the header a derived number instead of a declared one, so the class of bug disappears rather than being fixed once. The small fix above is complete on its own; this is only if you would rather the header could never disagree with the tensor again.Workaround for anyone who finds this first
Convert from a copy of the model directory whose
config.jsonhasmax_lengthset tomax_target_positions. Do not edit the published checkpoint — the value is correct for what it means; it is only being read as something else.