Repository navigation
fix(arm64): NEON parity tests for every uncovered kernel, and 7 defects they found - #1156
Merged
Merged
Conversation
…ts they found
ADR-1057 (a dropped filter tap in adm_dwt2_8_neon) reached master and drifted
the ARM golden because no unit test covered that kernel. Writing one found two
more defects in the same file. This does the same for the other eight
uncovered arm64 files: 18 kernels, 8 new parity tests, 7 defects.
The most serious is in VIF, a core VMAF feature:
vif_statistic_8_neon dropped up to 7 columns from EVERY row. Its horizontal
loop is `for (; j < uiw7; j += 8)` with uiw7 = w > 7 ? w - 7 : 0, and the row
loop closed straight after it -- no scalar tail. integer_vif.c installs the
kernel on `flags & VMAF_ARM_CPU_FLAG_NEON` with NO width guard, so every width
was admitted and the last w % 8 columns never reached num/den. For w <= 7 the
whole row was dropped. The 16-bit NEON twin, vif_statistic_8_avx2 and
vif_statistic_8_avx512 all close this with vif_compute_line_residuals(); the
8-bit NEON kernel was the only one of the four that did not.
This moved the score. Netflix golden content cropped to 348x216 (348 % 8 = 4):
NEON pre-fix vmaf 78.777730 vif_scale0 0.391580
scalar vmaf 78.778275 vif_scale0 0.391416
NEON post-fix vmaf 78.778275 vif_scale0 0.391416
Verified independently of the agent that found it. Any ARM user scoring
content whose width is not a multiple of 8 was affected.
The same kernel also omitted `sigma2_sq = MAX(sigma2_sq, 0)`, which is
width-independent. The scalar clamps before branching and the non-log arm then
accumulates sigma2_sq into num; the NEON epilogue stored the raw subtraction,
so negative values -- routine on near-flat content -- drove num the wrong way.
Symptom: integer_vif_scale0 of 1.0000087, impossible for a VIF ratio. The
16-bit twin and AVX2 both already clamp.
Also fixed: an 8-bit plane row over-read in ciede_preprocess_8_neon, a
zero-clamp sign divergence in ssim_variance_neon, and three float_adm_neon
precision/reduction divergences. Clean on first run, no changes needed:
float_adm_dwt2_neon, motion_neon, float_motion_neon, psnr_neon, and both
vif_subsample_rd kernels and vif_statistic_16_neon.
THE NETFLIX GOLDEN IS UNCHANGED. 576, 1280 and 1920 are all multiples of 8, so
the VIF tail defect cannot fire there; the golden pair scores 76.667831
pre-fix, post-fix and on the scalar path alike, against an assertion of
76.66783025 at places=4. No golden assertion is touched.
Verified: aarch64 106/106 under qemu-aarch64-static (all 10 NEON parity tests
included), x86 105/105, clang-format and check-copyright clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
5 of 9 tasks
lusoris
added a commit
that referenced
this pull request
Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was self-contradictory: a 2026-06-27 note inside the T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open" for a correct re-attempt, while its Recently-closed row already recorded a 2026-08-30 closure — and that closure narrative described only the integer-path dropped-tap defect and cited a branch name (`fix/adr-1057-arm-fma-drift`) instead of a commit. Re-verified against origin/master, with no defective code left to point at: - The float-ADM follow-up the row asked for landed in `a6c4dfffb` (PR #853): the kernel lives in a dedicated non-contracting TU `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with `-ffp-contract=off` (`core/src/meson.build`), and `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar `adm_dwt2_s` carries the matching function-scoped guard, so both sides of the comparison are non-contracting and the 1-ULP FMA gap that motivated the row cannot arise. - The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity tests for the remaining uncovered kernels. - `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar bit-exactness by bit pattern and is registered in the default `fast` suite. The row stays in "Recently closed" (exactly one row for the id), rewritten in past tense with real commit shas, and the stale "stays Open" note is marked superseded so the next session does not re-investigate. Documentation only — no code or behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was self-contradictory: a 2026-06-27 note inside the T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open" for a correct re-attempt, while its Recently-closed row already recorded a 2026-08-30 closure — and that closure narrative described only the integer-path dropped-tap defect and cited a branch name (`fix/adr-1057-arm-fma-drift`) instead of a commit. Re-verified against origin/master, with no defective code left to point at: - The float-ADM follow-up the row asked for landed in `a6c4dfffb` (PR #853): the kernel lives in a dedicated non-contracting TU `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with `-ffp-contract=off` (`core/src/meson.build`), and `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar `adm_dwt2_s` carries the matching function-scoped guard, so both sides of the comparison are non-contracting and the 1-ULP FMA gap that motivated the row cannot arise. - The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity tests for the remaining uncovered kernels. - `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar bit-exactness by bit pattern and is registered in the default `fast` suite. The row stays in "Recently closed" (exactly one row for the id), rewritten in past tense with real commit shas, and the stale "stays Open" note is marked superseded so the next session does not re-investigate. Documentation only — no code or behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was self-contradictory: a 2026-06-27 note inside the T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open" for a correct re-attempt, while its Recently-closed row already recorded a 2026-08-30 closure — and that closure narrative described only the integer-path dropped-tap defect and cited a branch name (`fix/adr-1057-arm-fma-drift`) instead of a commit. Re-verified against origin/master, with no defective code left to point at: - The float-ADM follow-up the row asked for landed in `a6c4dfffb` (PR #853): the kernel lives in a dedicated non-contracting TU `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with `-ffp-contract=off` (`core/src/meson.build`), and `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar `adm_dwt2_s` carries the matching function-scoped guard, so both sides of the comparison are non-contracting and the 1-ULP FMA gap that motivated the row cannot arise. - The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity tests for the remaining uncovered kernels. - `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar bit-exactness by bit pattern and is registered in the default `fast` suite. The row stays in "Recently closed" (exactly one row for the id), rewritten in past tense with real commit shas, and the stale "stays Open" note is marked superseded so the next session does not re-investigate. Documentation only — no code or behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 7, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was self-contradictory: a 2026-06-27 note inside the T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open" for a correct re-attempt, while its Recently-closed row already recorded a 2026-08-30 closure — and that closure narrative described only the integer-path dropped-tap defect and cited a branch name (`fix/adr-1057-arm-fma-drift`) instead of a commit. Re-verified against origin/master, with no defective code left to point at: - The float-ADM follow-up the row asked for landed in `a6c4dfffb` (PR #853): the kernel lives in a dedicated non-contracting TU `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with `-ffp-contract=off` (`core/src/meson.build`), and `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar `adm_dwt2_s` carries the matching function-scoped guard, so both sides of the comparison are non-contracting and the 1-ULP FMA gap that motivated the row cannot arise. - The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity tests for the remaining uncovered kernels. - `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar bit-exactness by bit pattern and is registered in the default `fast` suite. The row stays in "Recently closed" (exactly one row for the id), rewritten in past tense with real commit shas, and the stale "stays Open" note is marked superseded so the next session does not re-investigate. Documentation only — no code or behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 7, 2026
…#1331) The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was self-contradictory: a 2026-06-27 note inside the T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open" for a correct re-attempt, while its Recently-closed row already recorded a 2026-08-30 closure — and that closure narrative described only the integer-path dropped-tap defect and cited a branch name (`fix/adr-1057-arm-fma-drift`) instead of a commit. Re-verified against origin/master, with no defective code left to point at: - The float-ADM follow-up the row asked for landed in `a6c4dfffb` (PR #853): the kernel lives in a dedicated non-contracting TU `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with `-ffp-contract=off` (`core/src/meson.build`), and `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar `adm_dwt2_s` carries the matching function-scoped guard, so both sides of the comparison are non-contracting and the 1-ULP FMA gap that motivated the row cannot arise. - The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity tests for the remaining uncovered kernels. - `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar bit-exactness by bit pattern and is registered in the default `fast` suite. The row stays in "Recently closed" (exactly one row for the id), rewritten in past tense with real commit shas, and the stale "stays Open" note is marked superseded so the next session does not re-investigate. Documentation only — no code or behaviour change. Co-authored-by: Lusoris <lusoris@pm.me> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
21 of 27 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ADR-1057 reached master and drifted
the ARM golden because no unit test covered that kernel. Writing one (#1154)
found two more defects in the same file. This does the same for the remaining
eight uncovered
core/src/feature/arm64/files: 18 kernels, 8 new paritytests, 7 defects fixed.
The serious one: VIF drops columns on ARM
vif_statistic_8_neon's horizontal loop isfor (; j < uiw7; j += 8)withuiw7 = w > 7 ? w - 7 : 0, and the row loop closes straight after it — noscalar tail.
integer_vif.cinstalls the kernel onflags & VMAF_ARM_CPU_FLAG_NEONwith no width guard at all:So every width was admitted and the last
w % 8columns never reachednum/den. Forw <= 7, the entire row was dropped.The 16-bit NEON twin,
vif_statistic_8_avx2andvif_statistic_8_avx512allclose this gap with
vif_compute_line_residuals(). The 8-bit NEON kernel wasthe only one of the four that did not —
grep -c vif_compute_line_residualsgives 1 for
vif_neon.c(the 16-bit kernel) against 2 each for the AVX files.It moves the score. Netflix golden content cropped to 348×216 (348 % 8 = 4):
integer_vif_scale0--cpumask 1)VIF is a core VMAF feature, so any ARM user scoring content whose width is not a
multiple of 8 was getting wrong numbers. Reproduced independently of the agent
that found it.
The same kernel also omitted
sigma2_sq = MAX(sigma2_sq, 0), which iswidth-independent and fires at
w % 8 == 0too. The scalar clamps beforebranching and the non-log arm then accumulates
sigma2_sqintonum; the NEONepilogue stored the raw subtraction, so negative values — routine on near-flat
content, where fixed-point rounding of
mu2outruns the filtereddis²— drovenumthe wrong way. Symptom:integer_vif_scale0of 1.0000087, impossiblefor a VIF ratio. The 16-bit twin (
vmaxq_s32) and AVX2 (_mm256_max_epi32)both already clamp.
The Netflix golden is unchanged
576, 1280 and 1920 are all multiples of 8, so the tail defect cannot fire
there. The golden pair scores
76.667831pre-fix, post-fix and on the scalarpath alike, against an assertion of
76.66783025atplaces=4. No goldenassertion is touched.
Everything else
vif_neonvif_subsample_rd_8/16andvif_statistic_16cleanciede_neonssim_neonssim_variance_neon— fixedfloat_adm_neonfloat_adm_dwt2_neon,motion_neon,float_motion_neon,psnr_neonFive kernel families passing first time is worth stating: the pass was not
looking for something to change.
Type
fix— bug fixtest— the coverage that was missingsimd— backend-specificChecklist
make lintgreen; clang-format and check-copyright clean on all touched files.qemu-aarch64-static(all 10 NEON parity tests), x86 105/105..cfiles carry the license header.Bug-status hygiene
docs/state.md— covered by the per-defect changelog fragments; the VIF entry records the measured pre/post scores.Netflix golden-data gate
assertAlmostEqual(...)score. Golden verified unchanged on ARM:76.667831pre- and post-fix.Deep-dive deliverables
AGENTS.mdinvariant note — the dispatch-granularity-vs-vector-stride invariant is recorded indocs/rebase-notes.mdby fix(arm64): make adm_dwt2_8_neon bit-exact with the scalar kernel #1154; this PR is the same class.changelog.d/fixed/vif-neon-8bit-two-defects.md,changelog.d/added/neon-parity-test-coverage.md, plus three per-file fragments.arm64/files are fork-added; no upstream counterpart.Reproducer
Show the VIF defect moving a real score — crop the golden pair to a width that
is not a multiple of 8, and compare NEON against the scalar path:
qemu-aarch64-static -L /usr/aarch64-linux-gnu build-arm64/tools/vmaf \ --reference r348.yuv --distorted d348.yuv --width 348 --height 216 \ --pixel_format 420 --bitdepth 8 --model version=vmaf_v0.6.1 --json # add --cpumask 1 to force the scalar path and compareKnown follow-ups
w % 8dispatch guard question is worth a sweep:vif_statistic_8_neonhad no guard while the kernel needed one. Other arm64 dispatch sites should
be audited for the same stride-vs-guard mismatch.