test(gemma4): record the HF reference for the 12B text tower

[FeathBow]

Description

Closes #831

The loader can put the 12B text tower on a device, but nothing can say yet whether a forward pass through it would be right. Every comparison ahead — one local layer, the first and last global layer, the full single-token forward, the sub-window prefills — needs the same thing first: a reference produced by the published implementation, checked in, and regenerable from a recorded command. This commits that reference and the generator behind it. The engine for this model line does not exist yet, so nothing reads the fixture today; a gate lands with its first consumer, on the same grounds as #784.

tools/accuracy/dump_gemma4_hf_golden.py produces test_data/gemma4-12b-hf-golden.safetensors from a local checkpoint. Three cases, each named by what it answers: one BOS token, where softmax runs over a single key so the query, its RoPE and the mask cannot affect the result, leaving the value and output projections, the norms, the MLP and the final softcap on the path; nine tokens, the compact multi-token case, causal masking and non-zero positions being live from two tokens on; and 1024 tokens, exactly the sliding window. The first two carry activation probes — at window width those would dwarf the rest of the file and nothing reads them.

The window edge was measured, not derived. Layer 0 is sliding and has no global layer ahead of it, so whether token 0 still reaches the last position of its output reads the boundary directly: at 1023 and at 1024 that position moves when token 0 changes, at 1025 it is bitwise unchanged. The window admits sliding_window keys inclusive of the current position, so sliding_window tokens is the widest prefill that evicts nothing and sliding_window + 1 is the first that evicts.

Probe layers come out of layer_types rather than being hardcoded, both ends of both layer types, so layer-type dispatch is exercised at each end. Cuts are layer boundaries: the input of layer i and the output of layer i-1 are one activation, so adjacent probe layers share a tensor rather than storing it twice — layers 46 and 47 are adjacent, which is why there are eight cuts and not nine. A cut after the final norm keeps the tail diagnosable, splitting a logprob mismatch between the norm, the tied head and the softcap.

Activations come from forward hooks even though output_hidden_states=True works here. That argument returns 49 tensors, but its last entry is the final norm applied to the last layer's output rather than that output itself, so it cannot supply both global_last_out and final_norm_out; hooks also collect eight cuts instead of all 49. They are stored in bf16, the dtype the model computes in — widening to fp32 would store a converted value rather than the reference one, and double the file for no added precision. Sampled ids skip all 24 special and added tokens, which include the image and audio ids that text-only serving has to reject.

Two facts the reference pins are worth stating outside the doc. The embedding scale is 62.0, not sqrt(3840) = 61.9677: the buffer is cast to the weight dtype before the multiply, so bf16 rounding is part of the reference, and the resulting 5.2e-4 relative gap is far too large to pass off as accumulation noise. And each decoder layer ends by scaling its output by layer_scalar, after both residual adds, so that tensor applies to the layer output rather than to either branch.

Metadata is a single sorted-JSON key. safetensors serializes its metadata map in randomized order, so a multi-key block made two runs differ byte for byte while carrying identical tensors and identical metadata as dicts — a fixture that is reproducible looking like one that is not. With one key, regeneration is checked with sha256sum and nothing else.

Test Env

• Single GPU (sm_89, x86_64), 48 GB. Resident weights 22.28 GiB; the window-edge case adds a full-vocabulary logit tensor and its fp32 log-softmax on top of that.
• Checkpoint google/gemma-4-12B-it at revision 707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7. The fixture records sha256 of config.json, generation_config.json and the safetensors header — the header pins the tensor layout without reading 22 GiB of payload, and the revision pins the payload.
• Reference produced with transformers 5.11.0, torch 2.11.0+cu128. The checkpoint declares 5.10.0.dev0, a development build that was never released, so the pin is the release verified to load this architecture rather than a guess at what that build became.

python tools/accuracy/dump_gemma4_hf_golden.py <checkpoint-dir> \
    test_data/gemma4-12b-hf-golden.safetensors \
    --source-repo google/gemma-4-12B-it --revision 707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7

Verification

Regeneration is byte-identical: two runs of the script agree, a third after reformatting the script agrees, and the committed file matches all three at sha256 c30a338d499512e6f0505bd12b184ebb5af9d7536f0b7fc9ea2bdfdb18b1a46d.

The dumper refuses to write rather than emit a wrong reference. It aborts if a case is not bitwise reproducible within the process, if a probe layer is not the layer type it was selected as, or if any logit escapes the declared softcap. All three hold on the committed run: the 12B resolves to sliding 0/46 and global 5/47, and logits peak at 29.5 against a cap of 30.

ruff check and ruff format --check clean on the new script. No Rust changes, so no cargo surface moves.