bake/preview: check_thumbs.py — duplicate-bytes detector (catches scalar-collapse / orchestrator mid-mixup)

[gerchowl]

Motivation

bake/preview/utils/check_thumbs.py matches each thumb against two reference fingerprints (blank_default.png, blank_default_grey.png) per file. It does not check cross-thumb identity.

Result: a 3-material bake that produced byte-identical PNGs across visually distinct materials passed every per-file fingerprint check despite being obviously broken:

$ md5 /tmp/.../bernhard-ambientcg-full369/*.png
MD5 (Fabric_004.png)        = 30b593c563a668eb7712c05a4b4161bc
MD5 (Metal_007.png)         = 30b593c563a668eb7712c05a4b4161bc
MD5 (Metal_Plates_006.png)  = 30b593c563a668eb7712c05a4b4161bc

Three bernhard reference materials, three identical files, no fingerprint hit because none matched the existing references — they matched each other.

Decision / proposed approach

Add a third gate: bucket all baked PNGs by md5; fail if any bucket has size > 1 (or > N for a configurable threshold, default 1 = strict).

def detect_duplicate_renders(out_dir: Path, max_bucket: int = 1) -> list[list[Path]]:
    by_md5: dict[str, list[Path]] = {}
    for p in out_dir.rglob(\"thumb.png\"):
        h = hashlib.md5(p.read_bytes()).hexdigest()
        by_md5.setdefault(h, []).append(p)
    return [paths for paths in by_md5.values() if len(paths) > max_bucket]

This catches three independent failure modes:

1. Scalar-collapse upstream of the renderer (substrate empty / _scalars_for returning {} for many materials → all default-grey).
2. Orchestrator-side mid mixup (same to_threejs payload written for different materials due to closure / loop bug).
3. Texture-binding regression in the renderer (scalars differ but textures don't bind, so all materials reduce to the same scalar-only render).

What already exists

• bake/preview/utils/check_thumbs.py — DISTANCE_THRESHOLD-based fingerprint matcher.
• Two committed reference PNGs at bake/preview/assets/blank_default.png + blank_default_grey.png (re-baked under final orientation in d2742c6).

Scope

P0

• Add detect_duplicate_renders — exact md5 bucketing.
• Wire into check_thumbs.py main as a hard-fail with a clear report ("Metal_007.png and Fabric_004.png are byte-identical").
• One-line unit test: synth two identical PNGs in a tmpdir → asserts the check fails.

P1

• Approximate-duplicate detector (perceptual hash or downsampled L2) for renders that differ by 1-2 LSBs of dither / aaliasing but are otherwise identical materials. Threshold-tuneable.
• Per-source allowlist: some sources may legitimately have identical-bytes renders (e.g. two upstream entries that are the same material, deduped post-bake) — config carve-out.

Pitfalls

• A truly empty substrate produces N identical default-grey PNGs all matching blank_default_grey.png — the existing per-file check already fails those, so the new check would just reinforce. Make sure the new failure mode is reported distinctly so the operator can tell "3 materials defaulted" from "3 materials produced the same non-default render."
• File-size pre-filter: don't md5 every PNG in a 5000-material bake without a size-bucket pre-filter (cheap rejection of obviously-different files first).

Acceptance criteria

• Running check_thumbs.py /tmp/.../bernhard-ambientcg-full369 against the (current) byte-identical-x3 directory exits non-zero with a duplicate-bytes report.
• Running against a healthy bake (every PNG distinct) exits zero with no false positive.
• CI covers a positive and negative case via tmpdir fixtures.

References

• feat(bake): dedicated "thumb" tier — sphere-rendered material previews #361 thumb bake (this gate is the orchestrator's last line of defense).
• feat(tests): 5-layer field-passthrough test architecture (catches #380-class bugs in <1s) #382 — 5-layer test architecture; this is L5 (visual fingerprint), the layer the L3/L4 mocked-catalog tests cannot reach.
• Diagnosis trail: byte-identical x3 from production-substrate fallback while the orchestrator was supposed to hit gerchowl/mat-vis-tst (see client: allow dataset-repo override for MatVisClient (env var + ctor arg) #384).