Skip to content

fix(arm64): NEON parity tests for every uncovered kernel, and 7 defects they found - #1156

Merged
lusoris merged 1 commit into
masterfrom
feat/neon-parity-coverage
Aug 30, 2026
Merged

lusoris merged 1 commit into
masterfrom
feat/neon-parity-coverage

Conversation

@lusoris

@lusoris lusoris commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Summary

ADR-1057 reached master and drifted
the ARM golden because no unit test covered that kernel. Writing one (#1154)
found two more defects in the same file. This does the same for the remaining
eight uncovered core/src/feature/arm64/ files: 18 kernels, 8 new parity
tests, 7 defects fixed
.

The serious one: VIF drops columns on ARM

vif_statistic_8_neon's horizontal loop is for (; j < uiw7; j += 8) with
uiw7 = w > 7 ? w - 7 : 0, and the row loop closes straight after it — no
scalar tail
. integer_vif.c installs the kernel on
flags & VMAF_ARM_CPU_FLAG_NEON with no width guard at all:

if (flags & VMAF_ARM_CPU_FLAG_NEON) {
    s->vif_statistic_8 = vif_statistic_8_neon;

So every width was admitted and the last w % 8 columns never reached
num/den. For w <= 7, the entire row was dropped.

The 16-bit NEON twin, vif_statistic_8_avx2 and vif_statistic_8_avx512 all
close this gap with vif_compute_line_residuals(). The 8-bit NEON kernel was
the only one of the four that did not — grep -c vif_compute_line_residuals
gives 1 for vif_neon.c (the 16-bit kernel) against 2 each for the AVX files.

It moves the score. Netflix golden content cropped to 348×216 (348 % 8 = 4):

build VMAF integer_vif_scale0
NEON pre-fix 78.777730 0.391580
scalar (--cpumask 1) 78.778275 0.391416
NEON post-fix 78.778275 0.391416

VIF is a core VMAF feature, so any ARM user scoring content whose width is not a
multiple of 8 was getting wrong numbers. Reproduced independently of the agent
that found it.

The same kernel also omitted sigma2_sq = MAX(sigma2_sq, 0), which is
width-independent and fires at w % 8 == 0 too. The scalar clamps before
branching and the non-log arm then accumulates sigma2_sq into num; the NEON
epilogue stored the raw subtraction, so negative values — routine on near-flat
content, where fixed-point rounding of mu2 outruns the filtered dis² — drove
num the wrong way. Symptom: integer_vif_scale0 of 1.0000087, impossible
for a VIF ratio. The 16-bit twin (vmaxq_s32) and AVX2 (_mm256_max_epi32)
both already clamp.

The Netflix golden is unchanged

576, 1280 and 1920 are all multiples of 8, so the tail defect cannot fire
there. The golden pair scores 76.667831 pre-fix, post-fix and on the scalar
path alike, against an assertion of 76.66783025 at places=4. No golden
assertion is touched.

Everything else

file kernels result
vif_neon 4 2 defects (above); vif_subsample_rd_8/16 and vif_statistic_16 clean
ciede_neon 2 8-bit plane row over-read — fixed
ssim_neon 4 zero-clamp sign divergence in ssim_variance_neon — fixed
float_adm_neon 3 3 precision/reduction divergences — fixed
float_adm_dwt2_neon, motion_neon, float_motion_neon, psnr_neon 5 clean first run, no changes needed

Five kernel families passing first time is worth stating: the pass was not
looking for something to change.

Type

  • fix — bug fix
  • test — the coverage that was missing
  • simd — backend-specific

Checklist

  • Commits follow Conventional Commits.
  • make lint green; clang-format and check-copyright clean on all touched files.
  • Unit tests: aarch64 106/106 under qemu-aarch64-static (all 10 NEON parity tests), x86 105/105.
  • SIMD paths touched — bit-exactness against the scalar reference is exactly what the new tests assert.
  • New .c files carry the license header.
  • Not a breaking change.
  • No ADR — bug fixes plus test coverage; no architectural choice.

Bug-status hygiene

  • docs/state.md — covered by the per-defect changelog fragments; the VIF entry records the measured pre/post scores.

Netflix golden-data gate

  • I did not modify any assertAlmostEqual(...) score. Golden verified unchanged on ARM: 76.667831 pre- and post-fix.

Deep-dive deliverables

  • Research digest — no digest needed: each defect's investigation is its test plus the arithmetic argument in the changelog.
  • Decision matrix — no alternatives: only-one-way fix. A SIMD kernel either matches its scalar reference or does not.
  • AGENTS.md invariant note — the dispatch-granularity-vs-vector-stride invariant is recorded in docs/rebase-notes.md by fix(arm64): make adm_dwt2_8_neon bit-exact with the scalar kernel #1154; this PR is the same class.
  • Reproducer / smoke-test command — below.
  • CHANGELOG fragment — changelog.d/fixed/vif-neon-8bit-two-defects.md, changelog.d/added/neon-parity-test-coverage.md, plus three per-file fragments.
  • Rebase note — all touched arm64/ files are fork-added; no upstream counterpart.

Reproducer

meson setup build-arm64 core --cross-file build-aux/aarch64-linux-gnu.ini \
  -Denable_cuda=false -Denable_sycl=false -Denable_dnn=disabled
ninja -C build-arm64 -j4
meson test -C build-arm64 --suite=fast          # 106/106

Show the VIF defect moving a real score — crop the golden pair to a width that
is not a multiple of 8, and compare NEON against the scalar path:

qemu-aarch64-static -L /usr/aarch64-linux-gnu build-arm64/tools/vmaf \
  --reference r348.yuv --distorted d348.yuv --width 348 --height 216 \
  --pixel_format 420 --bitdepth 8 --model version=vmaf_v0.6.1 --json
# add --cpumask 1 to force the scalar path and compare

Known follow-ups

  • The w % 8 dispatch guard question is worth a sweep: vif_statistic_8_neon
    had no guard while the kernel needed one. Other arm64 dispatch sites should
    be audited for the same stride-vs-guard mismatch.

…ts they found

ADR-1057 (a dropped filter tap in adm_dwt2_8_neon) reached master and drifted
the ARM golden because no unit test covered that kernel. Writing one found two
more defects in the same file. This does the same for the other eight
uncovered arm64 files: 18 kernels, 8 new parity tests, 7 defects.

The most serious is in VIF, a core VMAF feature:

  vif_statistic_8_neon dropped up to 7 columns from EVERY row. Its horizontal
  loop is `for (; j < uiw7; j += 8)` with uiw7 = w > 7 ? w - 7 : 0, and the row
  loop closed straight after it -- no scalar tail. integer_vif.c installs the
  kernel on `flags & VMAF_ARM_CPU_FLAG_NEON` with NO width guard, so every width
  was admitted and the last w % 8 columns never reached num/den. For w <= 7 the
  whole row was dropped. The 16-bit NEON twin, vif_statistic_8_avx2 and
  vif_statistic_8_avx512 all close this with vif_compute_line_residuals(); the
  8-bit NEON kernel was the only one of the four that did not.

  This moved the score. Netflix golden content cropped to 348x216 (348 % 8 = 4):
    NEON pre-fix   vmaf 78.777730  vif_scale0 0.391580
    scalar         vmaf 78.778275  vif_scale0 0.391416
    NEON post-fix  vmaf 78.778275  vif_scale0 0.391416
  Verified independently of the agent that found it. Any ARM user scoring
  content whose width is not a multiple of 8 was affected.

  The same kernel also omitted `sigma2_sq = MAX(sigma2_sq, 0)`, which is
  width-independent. The scalar clamps before branching and the non-log arm then
  accumulates sigma2_sq into num; the NEON epilogue stored the raw subtraction,
  so negative values -- routine on near-flat content -- drove num the wrong way.
  Symptom: integer_vif_scale0 of 1.0000087, impossible for a VIF ratio. The
  16-bit twin and AVX2 both already clamp.

Also fixed: an 8-bit plane row over-read in ciede_preprocess_8_neon, a
zero-clamp sign divergence in ssim_variance_neon, and three float_adm_neon
precision/reduction divergences. Clean on first run, no changes needed:
float_adm_dwt2_neon, motion_neon, float_motion_neon, psnr_neon, and both
vif_subsample_rd kernels and vif_statistic_16_neon.

THE NETFLIX GOLDEN IS UNCHANGED. 576, 1280 and 1920 are all multiples of 8, so
the VIF tail defect cannot fire there; the golden pair scores 76.667831
pre-fix, post-fix and on the scalar path alike, against an assertion of
76.66783025 at places=4. No golden assertion is touched.

Verified: aarch64 106/106 under qemu-aarch64-static (all 10 NEON parity tests
included), x86 105/105, clang-format and check-copyright clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@lusoris
lusoris merged commit 6d61106 into master Aug 30, 2026
25 of 61 checks passed
@lusoris
lusoris deleted the feat/neon-parity-coverage branch August 30, 2026 16:45
@lusoris lusoris added this to the 1.0.0 — First release milestone Sep 4, 2026
lusoris added a commit that referenced this pull request Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was
self-contradictory: a 2026-06-27 note inside the
T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open"
for a correct re-attempt, while its Recently-closed row already
recorded a 2026-08-30 closure — and that closure narrative described
only the integer-path dropped-tap defect and cited a branch name
(`fix/adr-1057-arm-fma-drift`) instead of a commit.

Re-verified against origin/master, with no defective code left to point
at:

- The float-ADM follow-up the row asked for landed in `a6c4dfffb`
  (PR #853): the kernel lives in a dedicated non-contracting TU
  `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with
  `-ffp-contract=off` (`core/src/meson.build`), and
  `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls
  `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar
  `adm_dwt2_s` carries the matching function-scoped guard, so both
  sides of the comparison are non-contracting and the 1-ULP FMA gap
  that motivated the row cannot arise.
- The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was
  fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by
  `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity
  tests for the remaining uncovered kernels.
- `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar
  bit-exactness by bit pattern and is registered in the default `fast`
  suite.

The row stays in "Recently closed" (exactly one row for the id),
rewritten in past tense with real commit shas, and the stale "stays
Open" note is marked superseded so the next session does not
re-investigate. Documentation only — no code or behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was
self-contradictory: a 2026-06-27 note inside the
T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open"
for a correct re-attempt, while its Recently-closed row already
recorded a 2026-08-30 closure — and that closure narrative described
only the integer-path dropped-tap defect and cited a branch name
(`fix/adr-1057-arm-fma-drift`) instead of a commit.

Re-verified against origin/master, with no defective code left to point
at:

- The float-ADM follow-up the row asked for landed in `a6c4dfffb`
  (PR #853): the kernel lives in a dedicated non-contracting TU
  `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with
  `-ffp-contract=off` (`core/src/meson.build`), and
  `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls
  `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar
  `adm_dwt2_s` carries the matching function-scoped guard, so both
  sides of the comparison are non-contracting and the 1-ULP FMA gap
  that motivated the row cannot arise.
- The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was
  fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by
  `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity
  tests for the remaining uncovered kernels.
- `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar
  bit-exactness by bit pattern and is registered in the default `fast`
  suite.

The row stays in "Recently closed" (exactly one row for the id),
rewritten in past tense with real commit shas, and the stale "stays
Open" note is marked superseded so the next session does not
re-investigate. Documentation only — no code or behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was
self-contradictory: a 2026-06-27 note inside the
T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open"
for a correct re-attempt, while its Recently-closed row already
recorded a 2026-08-30 closure — and that closure narrative described
only the integer-path dropped-tap defect and cited a branch name
(`fix/adr-1057-arm-fma-drift`) instead of a commit.

Re-verified against origin/master, with no defective code left to point
at:

- The float-ADM follow-up the row asked for landed in `a6c4dfffb`
  (PR #853): the kernel lives in a dedicated non-contracting TU
  `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with
  `-ffp-contract=off` (`core/src/meson.build`), and
  `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls
  `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar
  `adm_dwt2_s` carries the matching function-scoped guard, so both
  sides of the comparison are non-contracting and the 1-ULP FMA gap
  that motivated the row cannot arise.
- The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was
  fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by
  `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity
  tests for the remaining uncovered kernels.
- `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar
  bit-exactness by bit pattern and is registered in the default `fast`
  suite.

The row stays in "Recently closed" (exactly one row for the id),
rewritten in past tense with real commit shas, and the stale "stays
Open" note is marked superseded so the next session does not
re-investigate. Documentation only — no code or behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 7, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was
self-contradictory: a 2026-06-27 note inside the
T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open"
for a correct re-attempt, while its Recently-closed row already
recorded a 2026-08-30 closure — and that closure narrative described
only the integer-path dropped-tap defect and cited a branch name
(`fix/adr-1057-arm-fma-drift`) instead of a commit.

Re-verified against origin/master, with no defective code left to point
at:

- The float-ADM follow-up the row asked for landed in `a6c4dfffb`
  (PR #853): the kernel lives in a dedicated non-contracting TU
  `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with
  `-ffp-contract=off` (`core/src/meson.build`), and
  `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls
  `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar
  `adm_dwt2_s` carries the matching function-scoped guard, so both
  sides of the comparison are non-contracting and the 1-ULP FMA gap
  that motivated the row cannot arise.
- The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was
  fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by
  `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity
  tests for the remaining uncovered kernels.
- `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar
  bit-exactness by bit pattern and is registered in the default `fast`
  suite.

The row stays in "Recently closed" (exactly one row for the id),
rewritten in past tense with real commit shas, and the stale "stays
Open" note is marked superseded so the next session does not
re-investigate. Documentation only — no code or behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 7, 2026
…#1331)

The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was
self-contradictory: a 2026-06-27 note inside the
T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open"
for a correct re-attempt, while its Recently-closed row already
recorded a 2026-08-30 closure — and that closure narrative described
only the integer-path dropped-tap defect and cited a branch name
(`fix/adr-1057-arm-fma-drift`) instead of a commit.

Re-verified against origin/master, with no defective code left to point
at:

- The float-ADM follow-up the row asked for landed in `a6c4dfffb`
  (PR #853): the kernel lives in a dedicated non-contracting TU
  `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with
  `-ffp-contract=off` (`core/src/meson.build`), and
  `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls
  `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar
  `adm_dwt2_s` carries the matching function-scoped guard, so both
  sides of the comparison are non-contracting and the 1-ULP FMA gap
  that motivated the row cannot arise.
- The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was
  fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by
  `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity
  tests for the remaining uncovered kernels.
- `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar
  bit-exactness by bit pattern and is registered in the default `fast`
  suite.

The row stays in "Recently closed" (exactly one row for the id),
rewritten in past tense with real commit shas, and the stale "stays
Open" note is marked superseded so the next session does not
re-investigate. Documentation only — no code or behaviour change.

Co-authored-by: Lusoris <lusoris@pm.me>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant