feat(adm): make the x86 float ADM wavelet and CSF kernels exact and dispatch them (ADR-1473)

[lusoris]

Summary

The x86 float ADM kernels existed, were called by nothing and tested by nothing, and differed from the scalar code (found in #1864). Maintainer decision (popup): make them exact, test them and wire them. This PR does that for the wavelet and the CSF stage, and removes the two reduction kernels, which cannot return the reference's bits (ADR-1473).

float_adm is 13 to 24 % faster with AVX2 or AVX-512, and no score moves.

What each kernel was, and is

Kernel	Scalar counterpart	Before	Now
float_adm_dwt2_avx2 / _avx512	adm_dwt2_s()	started each four-tap sum at the first product; the scalar starts at +0. Identical on picture data, -0 for +0 on 2.5 % of the outputs of a frame of signed zeros	exact, dispatched
float_adm_csf_avx2 / _avx512	the element loop of adm_csf_s()	multiplied FLOAT_ONE_BY_30 * fabsf(dst) in float; the scalar multiplies in double because the constant is a double literal. 0.9 % of values differ in the last bit	exact, dispatched
float_adm_csf_den_scale_avx2 / _avx512	adm_csf_den_scale_s()	double lane sums; the reference adds fp32 in column order	removed
float_adm_sum_cube_avx2 / _avx512	adm_sum_cube_s()	cubed in float where the scalar cubes in double; and compute_adm() never calls a sum of cubes	removed

What the scalar does (the brief asks): adm_dwt2_s() and its two pass helpers carry a contraction guard (GCC optimize("-ffp-contract=off"), Clang pragma; ADR-1057), and the whole tree builds with strict floating point since #1829. So the scalar multiplies, then adds; nothing is fused. The kernels do the same: _mm256_add_ps(accum, _mm256_mul_ps(c, s)) from a zero vector, no fmadd.

Why the reductions go: the denominator's bits are a sequence of float additions in column order (inner[k] += val * val * val, folded per row; adm_fold3_s() says these accumulators are the golden-gated arithmetic). A kernel that keeps that order adds one value per step and is bound by the same addition chain as the scalar loop; one that does not changes the bits. The stage is 1.9 % of the extractor at 3840x2160. RC7 row T-FLOAT-ADM-X86-SCALAR-STAGES-2026-10-02 holds it together with the decouple and the contrast masking, which never had kernels.

Wiring

adm.c chooses the kernel per call from vmaf_get_cpu_flags(), as float_motion, float_ssim, float_psnr and the others choose theirs:

• adm_dwt2_dispatch() (it already carried the NEON branch) gains AVX-512 and AVX2 branches;
• the CSF stage takes its band kernel through adm_csf_planes_s(..., plane). adm_csf_s() is that function with the scalar adm_csf_plane_s(), which is upstream's element loop with its three statements verbatim.

--cpumask 63 runs the scalar code, 48 AVX2, 0 AVX-512 (perf shows float_adm_dwt2_avx2 / float_adm_csf_avx2 and the _avx512 pair under the two settings). The wavelet kernels now return -ENOMEM like adm_dwt2_s(); they used to return nothing and leave the bands unwritten. The AVX2 wavelet also filters its horizontal pass eight outputs at a time (it was scalar).

Bits

Base: master b34732ae1. Everything at --precision max.

Record	Cases	Result
every adm / float_adm output (debug=true, 21 option sets) and the default and vmaf_v0.6.1 scores, 35 fixtures, scalar / AVX2 / AVX-512	1695	identical to the record of #1872's branch (which is master's arithmetic)
the same on aarch64 GCC under qemu, scalar / NEON	1066	1066 identical before the rebase (base 374e4342a); on this head 1030 identical to the record taken before #1872, and the 36 that differ are the integer adm_csf_mode=1 set, which #1872 changed
float_adm alone: fixture x option-set groups compared across the three dispatch levels	305 groups, 154,494 values	identical on all three
vmaf_float_v0.6.1, vmaf_float_v0.6.1neg, vmaf_float_4k_v0.6.1 on 35 fixtures, per dispatch level	315 (45,495 values)	identical to the base and across the three levels

Fixtures: Netflix 576x324 at 8, 10, 12 and 16 bits, both 1920x1080 checkerboards, BBB 3840x2160 (50 frames), 14 sizes from 17x17 to 129x65 at 8 and 10 bits. Against the x86 record taken before #1872, 1635 cases are identical and the 60 that differ are the integer adm_csf_mode=1 set.

Test

core/test/test_float_adm_x86.c (new, suites fast and simd), 9 tests:

• wavelet, AVX2 and AVX-512, against adm_dwt2_s(): every band compared byte for byte with the stride padding (both sides start from one poison pattern, so a store outside the plane shows); 29 widths from 17 to 576 (around the 8- and 16-wide loops of both passes) at five heights from 17; picture, fractional and 16-bit data; frames of signed zeros; NaN, infinities and denormals;
• CSF, both kernels, against adm_csf_plane_s(): every width from 1 to 70, six input classes, five weights including one that overflows;
• compute_adm() on six sizes from 576x324 to 17x17 with the kernels masked off, with AVX2 and with everything: the twelve doubles it returns have the same bits.

One limit: where two different NaNs meet in a sum, scalar and vector code can return NaNs of different sign or payload (the result is the first operand's, and a compiler may commute an addition). For the special-value class the test compares where the NaNs are, not their payload. A NaN in a frame fails the frame anyway (ADR-1302).

17 planted changes fail the test: a sum started at the first product (vector and scalar, both instruction sets), a fused multiply-add, two taps added in the other order, a horizontal loop one column too far, a wrong deinterleave, a horizontal tap from the wrong elements, the CSF filter as a float product (both kernels, and in the reference), a CSF loop that writes past the plane, swapped index tables in the dispatch. An 18th, the AVX-512 CSF falling back to the scalar function, passes by construction: the bits are the same.

Golden, twins, timing

• Netflix golden gate: 271 passed, 12 skipped on x86-64 GCC and on aarch64 GCC under qemu, on this head.

• x86 --suite=fast: 246 of 246. The three GPU contract tests that pin the reference's text (test_sycl_float_adm_exact_contract, test_cuda_float_adm_exact_contract, test_hip_float_adm_exact_contract) pass: adm_csf_plane_s() keeps upstream's statement verbatim.

• Twins. The CPU's values do not move, so the twins stay exact. Measured:

  Twin	Device	Gate cell float_adm, cpu against device	Device tests
  float_adm_cuda	RTX 4090	0 on the Netflix pair (48 frames), the 10 px checkerboard (3) and BBB 3840x2160 (200), on this head; the 1 px checkerboard too before the rebase	test_cuda_float_adm_parity, _parity_large: 17 of 17 each
  HIP	gfx1036	0 on the Netflix pair (48 frames)	test_hip_float_adm_parity, _parity_large (19 each), test_hip_float_adm_math (3), first-frame test
  SYCL	Arc A380 (xe)	0 on the Netflix pair (48 frames)	test_sycl_float_adm_parity, _parity_large (21 each), test_sycl_float_adm_math (7)

  HIP and SYCL were run on the tree before the rebase onto b34732ae1 (same sources in every file this PR touches).

• Timing. Whole vmaf --feature float_adm run (file reading and picture copy included), one thread, median of five, Ryzen 9 9950X3D, load average 22 to 28:

  Frame	Dispatch	Before	After
  576x324	scalar (--cpumask 63)	1.455 ms	1.450 ms
  576x324	AVX2 (48)	1.457 ms	1.168 ms
  576x324	AVX-512 (0)	1.421 ms	1.106 ms
  1920x1080	scalar	17.53 ms	17.90 ms
  1920x1080	AVX2	16.77 ms	15.17 ms
  1920x1080	AVX-512	17.54 ms	15.12 ms
  3840x2160	scalar	73.86 ms	76.16 ms
  3840x2160	AVX2	73.58 ms	62.38 ms
  3840x2160	AVX-512	75.02 ms	62.85 ms

  Before the change all three rows of a size run the same scalar code; their spread (and the scalar row's before / after difference) is the noise of a loaded host. Profile at 3840x2160: the wavelet goes from 23 % to 11 %; the decouple (27 %) and the contrast masking (20 %) now lead.

NEON

float_adm_dwt2_neon() is dispatched (adm_dwt2_dispatch(), since ADR-1057's update) and does not have the signed-zero defect: every sum starts at vdupq_n_f32(0.0f), and test_float_adm_dwt2_neon has a signed-zero case (it passed under qemu before and after #1865). Nothing to fix.

The other three NEON kernels (float_adm_csf_neon, float_adm_csf_den_scale_neon, float_adm_sum_cube_neon) are built and not dispatched, and by construction have the differences listed above for their x86 counterparts; their unit test compares them with a reference of its own, not with the scalar functions. I did not change them (not measured on aarch64; recorded in the RC7 row).

Type

• feat — new feature
• fix — bug fix
• perf — performance improvement
• refactor — no behavior change
• docs — documentation only
• test — test-only
• build / ci — tooling / infra
• port — cherry-pick from upstream Netflix/vmaf
• sycl / cuda / simd — backend-specific

Checklist

• Commits follow Conventional Commits (the commit-msg hook enforces this).
• make format && make lint is green locally. (clang-format and the commit hooks; HISS baseline unchanged at 242. clang-tidy: the host's version became 23.1.1 during this work, so the lanes could not be measured against their 22.1.8 baselines. Under 23.1.1 the two kernel files measure 0, and the new test, adm.c and adm_tools.c show only what that version reports tree-wide: modernize-redundant-void-arg on f(void) and bugprone-signed-bitwise on flags & VMAF_X86_CPU_FLAG_*, the spelling every extractor uses.)
• Unit tests pass: python3 scripts/ci/run_meson_test.py -- -C build. (--suite=fast on a CPU build: 246 of 246.)
• If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2. (0: scalar, AVX2 and AVX-512 are bit-identical; the GPU twins equal the CPU.)
• If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap under "Known follow-ups" below. (No arithmetic of the reference changes.)
• If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header (see CONTRIBUTING.md). (test_float_adm_x86.c: EUPL-1.2.)
• If this is a breaking change, the commit message uses ! or BREAKING CHANGE: and the migration path is documented below. (Not breaking: no public symbol changes; the removed kernels were internal and had no caller.)
• If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/<NNNN-slug>.md. (1473-float-adm-x86-simd-exact-and-dispatched.md.)

Bug-status hygiene (ADR-0165)

• docs/state.md updated in this PR. T-FLOAT-ADM-X86-KERNELS-NOT-EXACT-NOT-DISPATCHED-2026-10-02 opened and closed; T-FLOAT-ADM-X86-SCALAR-STAGES-2026-10-02 opened for RC7.

Netflix golden-data gate (ADR-0024)

• I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
• If I believe a golden value must change, I have explained why below AND pinged @lusoris for a CODEOWNERS exception. (None changes.)

Deep-dive deliverables (ADR-0108)

• Research digest — no digest needed: the measurements are in ADR-1473 and in this description.
• Decision matrix — ADR-1473, "Alternatives considered".
• AGENTS.md invariant note — core/src/feature/x86/AGENTS.d/float-adm.md (rewritten: what each kernel must match and the rules that keep it exact) and core/src/feature/AGENTS.d/float-adm.md (where the dispatch lives, what stays scalar, where an upstream hunk in adm_csf_s() goes).
• Reproducer / smoke-test command — pasted below under "Reproducer".
• CHANGELOG fragment — changelog.d/changed/float-adm-x86-simd-exact-dispatched.md.
• Rebase note — docs/rebase-notes.md, "x86 float ADM: wavelet and CSF kernels are exact and dispatched; the reduction kernels are gone". The kernel files are fork files (upstream has no float ADM SIMD); adm_tools.c, adm_tools.h and adm.c are upstream mirrors, and the note says where an upstream hunk goes.

User documentation: docs/metrics/features.md, new section "float_adm uses AVX2 and AVX-512, with the same scores".

Reproducer

meson setup build-cpu core -Denable_cuda=false -Denable_sycl=false -Db_lto=false
ninja -C build-cpu
build-cpu/test/test_float_adm_x86          # 9 tests run, 9 passed
Y=python/test/resource/yuv
for mask in 63 48 0; do                    # scalar, AVX2, AVX-512: the three outputs are identical
  build-cpu/tools/vmaf -r $Y/src01_hrc00_576x324.yuv -d $Y/src01_hrc01_576x324.yuv \
    -w 576 -h 324 -p 420 -b 8 --no_prediction --feature float_adm=debug=true \
    --precision max --cpumask $mask --json -o /tmp/float_adm_$mask.json -q
done
make test-netflix-golden                   # 271 passed, 12 skipped

Known follow-ups

• T-FLOAT-ADM-X86-SCALAR-STAGES-2026-10-02 (RC7): the decouple, the denominator reduction and the contrast masking have no SIMD form; the three undispatched NEON kernels.
• The tidy baselines' measured_sources do not list the new test; I did not write baselines from this host after its clang-tidy changed version.