feat(parity): compare every CPU value with Netflix/vmaf in the dev image and fail on a difference no ADR covers (ADR-1487, ADR-1494)

[lusoris]

Summary

make upstream-parity compares this tree's CPU extractors with Netflix/vmaf at the recorded parity head, every emitted value at %.17g, in the dev container image, and fails on a difference that no recorded deviation covers. It makes the rule of ADR-1487 enforceable: code inherited from Netflix evaluates as Netflix's source does, and a difference needs an ADR.

What it does:

• Builds Netflix/vmaf at the pin (the heading in docs/development/known-upstream-bugs.md, read through scripts/ci/upstream_parity_pin.py) and this tree with the golden build profile, both with the same compilers, both in the dev container image (--container, what the make targets pass).
• Runs one harness (scripts/dev/upstream_parity_harness.c) against each tree through the C API: 16 shared extractors, 169 option variants, 32 model requests, on 31 fixtures, at scalar and default dispatch (the full matrix adds AVX2).
• Attributes every difference to one fragment under scripts/ci/upstream_parity.d/. Fails on a difference no fragment covers, on one above its fragment's bound, on a stale fragment, and on a crash of this tree's harness. Exit 2 means it could not compare; that is never a pass.

Why the dev image, and the heap check

• Netflix's own values depend on the environment. Its ciede2000() calls powf(x, 2): on the host (glibc 2.44) that differs from the product on 119 of 327 frames (fix(ciede): form ciede2000()'s two products in float as upstream does and mirror them in the CUDA, SYCL and HIP twins (ADR-1476) #1892); in the dev image (glibc 2.43) on none. So a comparison is evidence only in one recorded environment. Both result documents record the image id, the compilers and the C library; documents from two environments are not compared; outside the image the guard exits 2 unless --unpinned marks the verdict advisory.
• Some upstream outputs are undefined. Repeating the full matrix on the host with the heap filled moved 48 upstream runs and twice took a difference above the bound it had then (ciede on odd sizes, float_motion scale-1 chroma: 25.2, then 61). --heap-check (in make upstream-parity-full) reruns every request with MALLOC_PERTURB_=170: an output of this tree that changes fails the guard; an upstream output that changes may only be covered with bound: inf. In the image: 1,675 upstream outputs in 45 runs change, none of this tree's.

Result on master

Dev image sha256:43ef1e32cb32 (GCC 15.2.0, glibc 2.43) on ryzen-4090-arc, master 96af5b34e with this PR against Netflix 9e48141b.

	Probe set (make upstream-parity)	Full matrix (make upstream-parity-full)
Runs per tree	1,848	5,349
Values compared	252,162	886,002
Identical	219,592	780,741
Differences covered by a deliberate fragment	19,958	70,826
Differences covered by a pending fragment	6,180	12,690
Not covered / above bound / stale	0 / 0 / 0	0 / 0 / 0
Heap check: upstream outputs that change / this tree's	not run	1,675 in 45 runs / 0
Verdict	PASS	PASS

42 fragments: 37 deliberate (D1 to D15 of the audit, ADR-1494, and the harmonic-mean guard of ADR-1008) and 5 pending. Every fragment's evidence line is the measurement in the image; no bound had to move from the host measurement. ADR-1467's ciede product has no fragment: in the image there is no difference to cover.

A planted change fails it (in the image): 10 * changed to 10.000000000000002 * in one line of core/src/feature/float_psnr.c gave 376 differences not covered, exit 1.

The pending fragments end as designed. Four reverts landed while this PR was in review (#1891, #1892, #1894, #1895); their seven fragments went stale and are removed here. With the last one, the SpEED revert, applied in a scratch copy, the full matrix has 795,000 identical values of 886,002, nothing outside the allowlist, and exactly its 3 pending-revert fragments stale.

Pending fragments and who removes them

Branch	Fragments
fix/speed-upstream-double-math	speed_chroma.float-math, speed_temporal.float-math, model.speed-float-math
port/upstream-motion-five-frame-window	motion.five-frame-window, model.hfr-five-frame-window

Whichever of this PR and one of those branches lands second deletes the fragments.

ADR-1494

adm and float_adm refuse frames below 17x17 (fork PRs #1473, #1770), which no ADR recorded. Measured in the image on the 16x16, 12x9 and 8x8 crops: upstream's integer ADM ends in signal 11 on all three; its float_adm returns values, which at 12x9 and 8x8 change with the heap's contents (adm2 3.94 or 3.38; 1.42 or 3.6e-11). The two error fragments cite it.

Other changes

• testdata/bench_upstream_ab.py builds upstream through the guard and takes its score verdict from the guard's comparison (marked advisory when run on the host). --max-score-delta is gone; the default upstream is the recorded head, not v3.2.0.
• scripts/ci/cross_backend_calibration.py: the fragment line parser is shared (parse_fragment_fields(), check_fragment_adrs()); exact_twins.d loads as before.
• scripts/ci/setup-golden-build.sh honours GOLDEN_NINJA_JOBS.

CI

Not wired, by decision of ADR-1487: a hosted job would need the same image, and the bounds are measured in that image on one host so far. T-UPSTREAM-PARITY-GUARD-HOSTED-JOB-2026-10-02 records what a nightly job needs. The script's tests need no build and run in a pre-commit hook.

Type

• feat — new feature
• build / ci — tooling / infra

Checklist

• Commits follow Conventional Commits.
• Unit tests pass: python3 -B scripts/ci/tests/test_upstream_parity_allowlist.py (25), python3 -B scripts/dev/tests/test_upstream_parity.py (52), pytest scripts/ci/test_calibration.py scripts/ci/test_cross_backend_parity_gate.py (109).
• New .c file carries the licence header (scripts/dev/upstream_parity_harness.c, EUPL-1.2).
• ADR rows live in docs/adr/_index_fragments/; docs/adr/README.md is regenerated.
• Cross-backend diff: not applicable, no extractor, SIMD or GPU source changes.

Bug-status hygiene (ADR-0165)

• docs/state.md updated: T-UPSTREAM-PARITY-PENDING-FRAGMENTS-2026-10-02 and T-UPSTREAM-PARITY-GUARD-HOSTED-JOB-2026-10-02 opened.

Netflix golden-data gate (ADR-0024)

• I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.

Deep-dive deliverables (ADR-0108)

• Research digest — docs/research/1487-upstream-parity-audit.md.
• Decision matrix — docs/adr/1487-upstream-parity-policy-and-guard.md and docs/adr/1494-adm-refuses-frames-below-17.md, ## Alternatives considered.
• AGENTS.md invariant note — scripts/ci/AGENTS.d/upstream-parity.md.
• Reproducer / smoke-test command — below.
• CHANGELOG fragment — changelog.d/added/upstream-parity-guard.md, changelog.d/changed/bench-upstream-ab-parity-guard.md.
• Rebase note — docs/rebase-notes.md, "Upstream parity guard and its allowlist".

Reproducer

docker compose -f dev/docker-compose.yml build dev-mcp   # the pinned image
scripts/test/fetch-test-yuvs.sh           # the three golden pairs
make upstream-parity                      # probe set: PASS, 0 not covered, 0 stale
make upstream-parity-full                 # full matrix with the heap check
python3 -B scripts/ci/tests/test_upstream_parity_allowlist.py
python3 -B scripts/dev/tests/test_upstream_parity.py
scripts/dev/upstream_parity.py --mode probe   # on the host: exit 2, not in the pinned environment
# a planted difference fails: in core/src/feature/float_psnr.c write
#   MIN(10.000000000000002 * log10(...)) and run
#   scripts/dev/upstream_parity.py --container --only F.float_psnr   -> exit 1, 376 not covered

Known follow-ups

• Hosted nightly job in the same image: T-UPSTREAM-PARITY-GUARD-HOSTED-JOB-2026-10-02.
• Where upstream's value is undefined (ciede.odd-size-chroma, float_motion.scale1-stride, speed_temporal.prescale-above-one) the bound is inf; this tree's values there are held by the tests of ADR-1483, ADR-1486 and ADR-1480.
• Not compared: aarch64, clang / icx / MSVC, any C library but the image's, more than one thread, the extractors only this tree has.