Skip to content

fix(adm): compute the CSF weights of float ADM in float, as Netflix does (ADR-1489) - #1894

Merged
lusoris merged 1 commit into
masterfrom
fix/float-adm-barten-upstream-float
Oct 2, 2026
Merged

lusoris merged 1 commit into
masterfrom
fix/float-adm-barten-upstream-float

Conversation

@lusoris

@lusoris lusoris commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follows #1891 (the integer half, ADR-1475, on master as ad0326f); this branch is rebased onto master and holds one commit.

float_adm computed its contrast-sensitivity weights with intermediates Netflix does not use. Two inherited routines were widened from float to double by fork commits that answered a static-analysis finding (cpp/integer-multiplication-cast-to-long) and record no decision:

This PR puts Netflix's arithmetic back (ADR-1489), under the rule that inherited code evaluates as Netflix's source does unless an ADR records a deviation. ADM_OPT_RECIP_DIVISION stays undefined (ADR-1442, deliberate): after this PR that division is the only arithmetic difference between the fork's float_adm and Netflix's on x86, which the third column below shows.

What changed

  • core/src/feature/adm_tools.h: float r, float temp, float Q = 2.0 * params->a * pow(10.0, params->k * temp * temp) / ..., as libvmaf/src/feature/adm_tools.h has them; the finding is answered by a suppression comment that cites the ADR.
  • core/src/feature/barten_csf_tools.h (its 18 locals that never change are const now, which clears the header's clang-tidy findings as C++): each product and quotient is formed in float, as Netflix forms it, and the promotion of the result is written out (pow((double)(p_0 * spatial_frequency), (double)p_1), exp((double)(-barten_mtf_params_b[i] * spatial_frequency)), ...). The reason for the explicit form: the SYCL and Metal twins of integer ADM compile this header as C++, where pow(float, float) and exp(float) are the float functions. Netflix's text compiled as C++ returns other weights than compiled as C on 135 of 144 probed argument sets; with the promotions written out the header returns the same bits in both languages, and an explicit conversion is not what the static-analysis query reports. linear_interpolate() is Netflix's text unchanged.
  • core/src/feature/metal/float_adm_metal.mm: its copy of the step forms the exponent in a named float. The CUDA, HIP and SYCL twins of float_adm call the CPU's adm_csf_rfactor_s() and follow by rebuild.
  • No SIMD file computes a weight: the x86 and NEON kernels of float ADM take them as arguments.
  • core/test/test_float_adm_csf_upstream.c (new, with barten_csf_cxx.cpp): the step equals the float form on 40 steps and the double form is another number; linear_interpolate() equals the float form and the double slope is another number; barten_csf() compiled as C++ returns the bits of the C one on 168 weights; on glibc the 40 steps and the 168 weights have the bits of a Netflix cea2b4d8 build. It fails against master's headers.
  • core/test/test_float_adm_csf_upstream_contract.py (new): reads the two headers and the Metal copy; rejects a double local, a promoted operand, and an implicit promotion in the Barten header, each with a planted regression.

Against Netflix master

Harness of #1891: C API, every collector value at %.17g, 658 frames in the 28 fixtures float_adm accepts, float_adm by default, with debug=true and under 36 option variants, seven models that read float_adm. Three trees: Netflix cea2b4d8; this branch; and Netflix with the plain quotient, which is cea2b4d8 with ADM_OPT_RECIP_DIVISION undefined and the #ifdef __SSE2__ of adm_tools.c forced to its #else branch (DIVS(n, d) ((n) / (d)), what Netflix compiles on ARM; with the macro alone undefined the file has no DIVS on x86 and does not compile). Values identical / values compared; the counts are the same at scalar (cpumask 63), AVX2 (48) and AVX-512 (0) dispatch:

Before, against Netflix After, against Netflix After, against Netflix with the plain quotient
adm2, 658 frames 30 (at most 1.14e-7 apart) 464 (at most 6.8e-8) 658
per-frame values, default and debug=true runs 1867 / 8225 7053 / 8225 8225 / 8225
per-frame values, 36 option variants 17 995 / 71 280 60 085 / 71 280 71 280 / 71 280
vmaf_float_v0.6.1 score, 252 frames 9 (at most 2.0e-5) 179 (at most 1.48e-5) 252
vmaf_float_v0.6.1neg score 13 (at most 2.7e-5) 151 (at most 1.79e-5) 252
frame scores of the seven float models 146 / 1764 1243 / 1764 1764 / 1764

The residual against Netflix is therefore exactly the division of ADR-1442. Outside the table: the three fixtures below 17x17, which the fork refuses (#1770) where Netflix reads outside the band.

What moves in the fork's own output (tree before against tree after):

Metric Frames that move Largest move
float_adm adm2 618 of 658 1.14e-7
float_adm per-scale scores 2.7e-7
float_adm adm2, adm_csf_mode=1 315 of 315 1.5e-7
integer adm, integer_adm2, adm_csf_mode=1 315 of 315 1.6e-7
integer adm per-scale scores, adm_csf_mode=1 2.5e-7
vmaf_float_v0.6.1 frame score 241 of 252 2.2e-5
vmaf_float_v0.6.1neg frame score 245 of 252 2.7e-5

Integer adm in Barten mode moves because it reads the same header (U3). ADR-1472's weight limits and ADR-1325's normalisation are unchanged and its tests pass (test_adm_csf_representable 7 of 7, test_barten_csf 13 of 13, test_barten_csf_coverage); against Netflix that mode stays apart by design, since Netflix's Barten weights wrap in their fixed-point conversion. Integer adm outside Barten mode and the 11 models that do not read float_adm do not move: 82 710 of 82 710 values identical.

Golden gate, SIMD, twins

The harness tables, the aarch64 golden gate and the cpu, cuda, hip and arm64 tidy lanes were measured at 38b7a737d (this change on top of #1891 before both were rebased; the changed lines of every file are the same as now). After the rebase onto master f60ad15 the CPU build and suite, the float ADM and Barten tests, the x86 golden gate, all device tests and gate cells below, and the sycl tidy lane were run again on the pushed head.

  • Netflix golden gate, GCC golden-profile build: x86-64 271 passed, 12 skipped; aarch64 under qemu 271 passed, 12 skipped (four shards: 66 + 68 + 69 + 68 passed, 4 + 3 + 2 + 3 skipped). No assertion edited.
  • CPU suite: 267 of 267 pass (2 skipped). aarch64 under qemu: the 38 ADM and Barten tests pass, test_float_adm_csf_upstream, test_float_adm_dwt2_neon and test_feature_isa_invariance among them.
  • SIMD: test_float_adm_x86 (the test feat(adm): make the x86 float ADM wavelet and CSF kernels exact and dispatch them (ADR-1473) #1874 added) passes: the AVX2 and AVX-512 wavelet and CSF kernels return the reverted scalar code's bits. The harness counts above are identical at the three dispatch levels.
  • CUDA, RTX 4090: test_cuda_float_adm_parity 17 of 17, _parity_large 17 of 17, test_cuda_exact_twins 2 of 2, test_cuda_adm_parity 11 of 11, _parity_large 11 of 11, _small_border, _tiny_frames (8), _wide_rounding, _dwt2_rows (6).
  • HIP, gfx1036: test_hip_float_adm_parity 19 of 19, _parity_large 19 of 19, test_hip_float_adm_math 3 of 3, test_hip_exact_twins 2 of 2, test_hip_adm_exact 9 of 9, test_hip_adm_parity 3, _parity_large 3, _small_border, _tiny_frames (8), _wide_rounding, _dwt2_rows (5), _init_unwind, both first_frame_clear tests.
  • SYCL, Arc A380: test_sycl_float_adm_parity 21 of 21, _parity_large 21 of 21, test_sycl_float_adm_math 7 of 7, test_sycl_exact_twins 2 of 2, test_sycl_adm_parity 7 of 7, _parity_large 7 of 7, _tiny_frames 8 of 8.
  • Parity gate, float_adm and adm cells, tolerance 0 (no fragment under scripts/ci/exact_twins.d/ touched): max difference 0 on the Netflix pair, both checkerboard pairs and a 1920x1080 clip, for each of the three backends (24 cells).
  • Integer adm with adm_csf_mode=1, debug=true, each twin against the CPU extractor of the same build at --precision max: 0 of 864 values differ on the Netflix pair and 0 of 864 on the 1920x1080 clip, per backend. For SYCL this compares the Barten weights of the C++ translation unit with those of the C one.
  • An icx build (Intel's math library): test_float_adm_csf_upstream passes, bit tables included.
  • Metal: source changed to match, not run: no device.

Snapshots

  • CPU: none moves. testdata/scores_cpu_*.json hold the features of vmaf_v0.6.1 (fixed-point, Watson mode): the default-model output of this branch equals that of fix(adm): form the integer ADM quantisation step's exponent in float, as Netflix does (ADR-1475) #1891 value for value. No snapshot holds a float_adm or a Barten-mode value.
  • SYCL, Arc A380: testdata/scores_sycl_a380_{576,640,720,1080,4k}.json are re-recorded here (requested in review of fix(adm): form the integer ADM quantisation step's exponent in float, as Netflix does (ADR-1475) #1891; the device is on this host). They were stale before either PR: the files of 2026-09-26 differ from the new ones on 139, 123, 108, 101 and 107 of 576 shared values by up to 1.0e-4 and lack three metrics the twins emit now. Recorded with testdata/run_sycl_scores.py a380 inside the dev container (image sha256:43ef1e32cb32b148a076ed6dff73b72d7a6566ca3882bf90954b8a34a74761fc, icx 2026.1.1, glibc 2.43, the worktree copied in, /dev/dri passed through, compute runtime 26.35.39758), under the device lock. The image has no ocloc, so the binary was built with -Dsycl_icpx_aot_targets= (kernels compiled at run time), not with the image recipe's ahead-of-time list. The recordings equal the CPU snapshots on 3599 of 3600 values; the one is vmaf of frame 26 at 1280x720 (88.435637 for 88.435634), which is the icx build's math library (T-ICX-LIBIMF-HOST-MATH-2026-10-01). At %.17g the twin equals the CPU extractor of its own build on every value of the five clips, and a second recording run repeats the first.
  • SYCL, B580 and UHD 770: six files, not on this host, left as they are; T-SYCL-SNAPSHOTS-STALE-2026-10-02 now says what reads each recording (nothing gates on any of them; testdata/compare_combined.py, run by hand, reads the A380 ones) and what the office box has to record.

Type

  • feat — new feature
  • fix — bug fix
  • perf — performance improvement
  • refactor — no behavior change
  • docs — documentation only
  • test — test-only
  • build / ci — tooling / infra
  • port — cherry-pick from upstream Netflix/vmaf
  • sycl / cuda / simd — backend-specific

Checklist

  • Commits follow Conventional Commits (the commit-msg hook enforces this).
  • make format && make lint is green locally. (clang-format, the commit hooks, scripts/dev/preflight.sh --stage msvcism. clang-tidy 22.1.8 lanes measured in the dev container with scripts/dev/tidy-lane.sh: cpu 70 = baseline 70; sycl 154 for this PR's files, with the baseline tightened from 172 in this PR (barten_csf_tools.h 18 to 0: the header's 18 misc-const-correctness findings as C++ are fixed, and the three new test sources are listed); cuda 620 = 620; hip 505 against 504, arm64 116 against 115 and, after the rebase, sycl 155 against 154, where the one finding above baseline in each is readability-function-size at core/test/test_iqa_ssim_sub_window.c:281, a file master gained with fix(ssim): give the SIMD kernels the number of windows so float_ssim on a frame smaller than its window is 0, not garbage #1882 (2efce88) and this PR does not touch.)
  • Unit tests pass: python3 scripts/ci/run_meson_test.py -- -C build. (CPU build: 267 of 267, 2 skipped.)
  • If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2. (0 on CUDA, HIP and SYCL: the parity gate's float_adm and adm cells at tolerance 0.)
  • If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap under "Known follow-ups" below. (CUDA, HIP and SYCL by the shared routine and header; the Metal copy edited, not run.)
  • If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header (see CONTRIBUTING.md). (test_float_adm_csf_upstream.c, barten_csf_cxx.cpp, barten_csf_cxx.h: EUPL-1.2.)
  • If this is a breaking change, the commit message uses ! or BREAKING CHANGE: and the migration path is documented below. (Not breaking: no public symbol or option changes. Scores move in the seventh decimal for float_adm, the fifth for the float models.)
  • If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/<NNNN-slug>.md. (1489-float-adm-barten-upstream-float.md.)

Bug-status hygiene (ADR-0165)

  • docs/state.md updated in this PR. T-ADM-CSF-EXPONENT-NOT-UPSTREAM-2026-10-01 moved to Recently closed (both halves) and taken off the RC3 list; T-SYCL-SNAPSHOTS-STALE-2026-10-02 rewritten (consumers, A380 re-recorded, B580 and UHD 770 still stale).

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
  • If I believe a golden value must change, I have explained why below AND pinged @lusoris for a CODEOWNERS exception. (None changes.)

Deep-dive deliverables (ADR-0108)

  • Research digest — docs/research/1489-float-adm-barten-upstream-float.md.
  • Decision matrix — ADR-1489, "Alternatives considered".
  • AGENTS.md invariant note — core/src/feature/AGENTS.d/adm-csf-weights-float.md (new), core/src/feature/metal/AGENTS.md (the copy), and an entry in docs/development/rebase-sensitive-invariants.md.
  • Reproducer / smoke-test command — pasted below under "Reproducer".
  • CHANGELOG fragment — changelog.d/fixed/float-adm-barten-upstream-float.md.
  • Rebase note — docs/rebase-notes.md, top entry: the statements are upstream's again and a sync takes upstream's side; for the Barten header, upstream's arithmetic with the promotions kept explicit.

User documentation: docs/metrics/features.md, new section "float_adm uses Netflix's contrast-sensitivity weights".

Reproducer

meson setup build-cpu core -Denable_cuda=false -Denable_sycl=false -Db_lto=false
ninja -C build-cpu
build-cpu/test/test_float_adm_csf_upstream             # 6 tests run, 6 passed
python3 core/test/test_float_adm_csf_upstream_contract.py
build-cpu/test/test_float_adm_x86                      # SIMD == scalar
make test-netflix-golden                               # 271 passed, 12 skipped
# residual against Netflix = the division only: build Netflix/vmaf cea2b4d8 with
#   adm_options.h: `#define ADM_OPT_RECIP_DIVISION` removed
#   adm_tools.c:   `#ifdef __SSE2__` -> `#if 0`
# and compare every float_adm value through the C API at %.17g (Netflix's CLI prints
# six decimals): identical.

Known follow-ups

@github-actions github-actions Bot added the type:bug Something isn't working label Oct 2, 2026
@lusoris
lusoris force-pushed the fix/float-adm-barten-upstream-float branch 2 times, most recently from b2f8b5f to dc309e1 Compare October 2, 2026 23:24
@lusoris
lusoris marked this pull request as ready for review October 2, 2026 23:24
…oes (ADR-1489) (#1894)

* fix(adm): compute the CSF weights of float ADM in float, as Netflix does (ADR-1489)

Two inherited routines compute the contrast-sensitivity weights of float
ADM, and fork commits had widened both from float to double to quiet a
static-analysis finding (cpp/integer-multiplication-cast-to-long), without
a decision:

- dwt_quant_step() in adm_tools.h (Watson): #552 cast an operand of the
  exponent k * temp * temp, #760 then kept r, temp and Q in double. 32 of
  40 probed steps were not Netflix's bits.
- barten_csf_tools.h (Barten, adm_csf_mode=1): #44 promoted one operand of
  six float products and quotients. 138 of 144 probed weight sets were not
  Netflix's bits.

Both evaluate Netflix's arithmetic again. adm_tools.h has Netflix's three
statements, with a suppression comment for the finding. The Barten header
forms each product and quotient in float and writes the promotion of the
result out: the SYCL and Metal twins of integer ADM compile it as C++,
where pow(float, float) and exp(float) are the float functions, so
Netflix's implicit form returns other weights there (135 of 144 probed
sets) than in C. The Metal copy of the step (float_adm_metal.mm) follows;
the CUDA, HIP and SYCL twins of float_adm call the CPU's routine.

ADM_OPT_RECIP_DIVISION stays undefined (ADR-1442). Against Netflix/vmaf
cea2b4d8 built with the plain quotient, float_adm is identical on every
measured value (8225 of 8225 default-run values, 71280 of 71280 over 36
option variants, 1764 of 1764 frame scores of seven float models; 658
frames, scalar, AVX2 and AVX-512 dispatch): the division is the only
difference left. Against Netflix as built on x86, adm2 is identical on 464
of 658 frames (30 before).

Scores move: float_adm adm2 by at most 1.14e-7, the float models by at
most 2.7e-5 on a frame, fixed-point adm with adm_csf_mode=1 by at most
1.6e-7. Fixed-point adm in its default mode, the default models and the
testdata snapshots do not move.

Twins on this host stay bit-identical to the CPU: float_adm and adm on
CUDA (RTX 4090), HIP (gfx1036) and SYCL (Arc A380), parity tests and the
gate's cells at tolerance 0, and integer adm in Barten mode on each twin.
Metal: source changed, not run (no device).

Snapshots: no CPU snapshot moves (they hold fixed-point features in
Watson mode). testdata/scores_sycl_a380_{576,640,720,1080,4k}.json are
re-recorded, because the files of 2026-09-26 no longer described the
twin: they differ from the new ones on 101 to 139 of 576 shared values by
up to 1.0e-4 and lack three metrics. Recorded with
testdata/run_sycl_scores.py a380 inside the dev container (image
sha256:43ef1e32cb32b148a076ed6dff73b72d7a6566ca3882bf90954b8a34a74761fc,
icx 2026.1.1, glibc 2.43, /dev/dri passed through, Arc A380, compute
runtime 26.35.39758), from this change as it stood at 539e99261 (no file
under core/ differs from this commit). The image has no ocloc, so the
binary was built with -Dsycl_icpx_aot_targets= (kernels compiled at run
time). The recordings equal the CPU snapshots on 3599 of 3600 values; the
one is vmaf of frame 26 at 1280x720 (88.435637 for 88.435634), an icx
build's math library (T-ICX-LIBIMF-HOST-MATH-2026-10-01). The B580 and
UHD 770 recordings stay stale: T-SYCL-SNAPSHOTS-STALE-2026-10-02.

Closes T-ADM-CSF-EXPONENT-NOT-UPSTREAM-2026-10-01.
@lusoris
lusoris force-pushed the fix/float-adm-barten-upstream-float branch from dc309e1 to 96af5b3 Compare October 2, 2026 23:30
@lusoris
lusoris merged commit 96af5b3 into master Oct 2, 2026
3 of 37 checks passed
@lusoris
lusoris deleted the fix/float-adm-barten-upstream-float branch October 2, 2026 23:30
lusoris added a commit that referenced this pull request Oct 2, 2026
…age and fail on a difference no ADR covers (ADR-1487, ADR-1494)

Code inherited from Netflix/vmaf evaluates as Netflix's source does; a
difference needs an ADR. The upstream parity audit of 2026-10-02 found
170,344 of 348,132 values different, six causes of them unintended, and
no gate that would have seen any of them. This makes the rule checkable.

scripts/dev/upstream_parity.py (make upstream-parity, make
upstream-parity-full) builds Netflix/vmaf at the recorded parity head
(read through scripts/ci/upstream_parity_pin.py) and this tree with the
golden build profile, with the same compilers, runs one C API harness
against each and compares every per-frame value, aggregate and pool at
%.17g: 16 shared extractors, their option variants and the shipped
models, on the Netflix pairs and clips derived from them, at scalar and
default dispatch (AVX2 too in the full matrix).

Both trees are built and run in the dev container image (--container):
Netflix's own ciede values differ between glibc 2.43 and 2.44, so a
comparison is evidence only in one recorded environment. The documents
record the image id, compilers and C library; outside the image the
guard refuses to measure unless --unpinned marks the verdict advisory.
--heap-check (in the full target) reruns every request with
MALLOC_PERTURB_=170: an output of this tree that changes fails, and an
upstream output that changes is undefined and may only be covered with
bound inf (ciede on odd sizes and float_motion's scale-1 chroma, which
went flaky on the host).

Every difference is attributed to one fragment under
scripts/ci/upstream_parity.d/ (the exact_twins.d pattern; the line
parser is now shared): 37 deliberate deviations with their ADRs and
bounds measured in the image, 5 pending (the SpEED revert and the
five-frame motion port). The guard fails on a difference no fragment
covers, on one above its bound, on a stale fragment and on a crash of
this tree's harness; exit 2 means it could not compare. The seven
fragments of the reverts that landed meanwhile (fork PRs #1891, #1892,
#1894, #1895) are removed.

ADR-1494 records the ADM extractors' refusal of frames below 17x17
(fork PRs #1473, #1770): upstream's integer ADM ends in signal 11 there,
its float ADM returns values that at 12x9 and 8x8 depend on the heap.

On master 96af5b3 in the image (GCC 15.2.0, glibc 2.43) against
Netflix 9e48141b: full matrix 886,002 values, 780,741 identical, 83,516
differences all covered, none stale; 1,675 upstream outputs depend on
the heap, none of this tree's. Probe set 252,162 values, pass. With the
SpEED revert applied in a scratch copy, exactly its 3 fragments go
stale. A one-ulp change planted in float_psnr.c fails the guard.

testdata/bench_upstream_ab.py builds upstream through the guard and
takes its score verdict from it (advisory on the host); --max-score-delta
is gone and the default upstream is the recorded head. Not a required
check yet (T-UPSTREAM-PARITY-GUARD-HOSTED-JOB-2026-10-02).

This branch was successfully deployed

1 active deployment
github-pages — 96af5b34 Deployed Oct 2, 2026 by lusoris via deploy #4116
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant