Skip to content

fix(arm64): adm_dwt2_8_neon dropped the 4th filter tap at j=0 (ADR-1057 rediagnosed) - #1134

Merged
lusoris merged 1 commit into
masterfrom
fix/adr-1057-arm-fma-drift
Aug 30, 2026
Merged

lusoris merged 1 commit into
masterfrom
fix/adr-1057-arm-fma-drift

Conversation

@lusoris

@lusoris lusoris commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

What

Fixes the aarch64 Netflix golden failure. One character: idx < 3 → idx < 4.

The bug

adm_dwt2_8_neon special-cases the j = 0 output column because its source
indices are mirrored ({1, 0, 1, 2} rather than the regular 2j-1 .. 2j+2
stride). That special case accumulated with:

for (int idx = 0; idx < 3; idx++) {          // ← three
    accum_a += (int32_t)dwt2_db2_coeffs_lo[idx] * s_lo;

The scalar reference adm_dwt2_8 (core/src/feature/integer_adm.c:2606)
multiplies all four taps — filter_lo[0..3] against s0..s3 from
ind_x[0..3][j]. The NEON path therefore dropped ind_x[3][0] and its
coefficient -4240 from the first output column of every DWT2 row, on every
scale
.

ADR-1057's diagnosis was wrong

ADR-1057 attributes this to
float-ADM NEON FMA contraction. It is not an FP problem at all — integer ADM
accumulates in int64. Proof: building the entire tree with
-ffp-contract=off on aarch64 leaves the score at 88.030322, byte-for-byte
unchanged. That is why PR #1060's fp-contract approach could not have worked and
#1063 reverted it.

Reproduced locally — no CI round-trip needed

aarch64-linux-gnu-gcc cross-build run under qemu-aarch64-static, driving the
real pytest harness via a shim at core/build/tools/vmaf:

Configuration integer_adm2 adm3 adm_scale3 akiyo
NEON 1.116698 1.052571 1.13728 88.030322 — 3 fail
-Denable_asm=false 1.116701 1.052573 1.13729 10/10 pass
NEON + -ffp-contract=off — — — 88.030322 unchanged
after this fix — — — 38 passed, 1 skipped, 0 failed

The final row covers vmafexec_test.py + result_test.py on aarch64, including
every akiyo golden assertion.

x86 is unaffected — the diff touches only core/src/feature/arm64/adm_neon.c.

Two gaps this exposes

  1. No NEON-vs-scalar parity test covers the ADM DWT2 kernels on any
    architecture.
    test_integer_adm_simd.c only exercises adm_cm_avx2. A
    three-vs-four tap error shipped because nothing compares the kernels directly.
  2. The ARM golden leg is not a required check, so it stayed red for months
    rather than blocking a merge.

Both are worth closing separately; neither is in scope here.

Deep-dive deliverables

  • Research digest — no digest needed: the isolation table above is the analysis and is reproduced in the commit body.
  • Decision matrix — the table above: FP-contraction ruled out by direct measurement before the integer off-by-one was identified.
  • AGENTS.md invariant note — no rebase-sensitive invariants: fork-local NEON kernel.
  • Reproducer / smoke-test command — see below.
  • CHANGELOG fragment — changelog.d/fixed/adr-1057-adm-neon-dwt2-tap.md
  • Rebase note — no rebase impact: fork-local arm64 SIMD path with no upstream counterpart.

Reproducer / smoke-test command

cat > /tmp/aarch64.ini <<'EOF'
[binaries]
c = 'aarch64-linux-gnu-gcc'
cpp = 'aarch64-linux-gnu-g++'
ar = 'aarch64-linux-gnu-ar'
strip = 'aarch64-linux-gnu-strip'
exe_wrapper = '/tmp/qemu-wrap'
[host_machine]
system = 'linux'
cpu_family = 'aarch64'
cpu = 'aarch64'
endian = 'little'
EOF
printf '#!/bin/sh\nexec qemu-aarch64-static -L /usr/aarch64-linux-gnu "$@"\n' > /tmp/qemu-wrap
chmod +x /tmp/qemu-wrap

meson setup build-arm core --cross-file /tmp/aarch64.ini -Denable_cuda=false -Denable_sycl=false
ninja -C build-arm tools/vmaf

mkdir -p core/build/tools
printf '#!/bin/sh\nexec qemu-aarch64-static -L /usr/aarch64-linux-gnu "$(dirname "$0")/../../../build-arm/tools/vmaf" "$@"\n' > core/build/tools/vmaf
chmod +x core/build/tools/vmaf

PYTHONPATH=$PWD/python python3 -m pytest python/test/vmafexec_test.py \
  -k akiyo_multiply -q          # 10 passed

Bug status hygiene

  • docs/state.md updated — T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 moved to Recently closed with the corrected root cause.

The aarch64 Netflix golden failure (akiyo 88.030322 vs 88.030463, delta
1.41e-04, three failing assertions on Build - Ubuntu ARM clang (CPU)) is an
integer off-by-one, not the FMA-contraction problem ADR-1057 describes.

adm_dwt2_8_neon special-cases the j = 0 output column because its source indices
are mirrored ({1, 0, 1, 2} rather than the regular 2j-1..2j+2 stride). That
special case accumulated with

    for (int idx = 0; idx < 3; idx++)

while the scalar reference adm_dwt2_8 in core/src/feature/integer_adm.c
multiplies all four taps (filter_lo[0..3] against s0..s3 from ind_x[0..3][j]).
The NEON path therefore dropped ind_x[3][0] and its coefficient -4240 from the
first output column of every DWT2 row, on every scale. Fix: idx < 4.

Why ADR-1057's diagnosis was wrong: integer ADM accumulates in int64, so FP
contraction cannot affect it. Building the entire tree with -ffp-contract=off on
aarch64 leaves the score at 88.030322, byte-for-byte unchanged. That is why
PR #1060's fp-contract approach could not have worked and #1063 reverted it.

Reproduced and fixed locally without a CI round-trip, using an
aarch64-linux-gnu-gcc cross-build under qemu-aarch64-static:

  NEON                     integer_adm2=1.116698 adm3=1.052571 scale3=1.13728
  -Denable_asm=false       integer_adm2=1.116701 adm3=1.052573 scale3=1.13729
  NEON + -ffp-contract=off 88.030322  (unchanged - rules out FP contraction)
  after this fix           38 passed, 1 skipped, 0 failed on aarch64
                           (vmafexec_test.py + result_test.py, all akiyo
                           golden assertions included)

x86 is unaffected: the change is confined to core/src/feature/arm64/adm_neon.c.

No NEON-vs-scalar parity test covers the ADM DWT2 kernels on any architecture -
test_integer_adm_simd.c only exercises adm_cm_avx2 - which is how a three-vs-four
tap error shipped. Only the ARM golden leg caught it, and that leg is not a
required check, so it stayed red. Both gaps are worth closing separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@lusoris
lusoris enabled auto-merge (squash) August 30, 2026 11:28
@lusoris
lusoris merged commit a013c14 into master Aug 30, 2026
4 of 61 checks passed
@lusoris
lusoris deleted the fix/adr-1057-arm-fma-drift branch August 30, 2026 11:51
@lusoris lusoris added this to the 1.0.0 — First release milestone Sep 4, 2026
lusoris added a commit that referenced this pull request Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was
self-contradictory: a 2026-06-27 note inside the
T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open"
for a correct re-attempt, while its Recently-closed row already
recorded a 2026-08-30 closure — and that closure narrative described
only the integer-path dropped-tap defect and cited a branch name
(`fix/adr-1057-arm-fma-drift`) instead of a commit.

Re-verified against origin/master, with no defective code left to point
at:

- The float-ADM follow-up the row asked for landed in `a6c4dfffb`
  (PR #853): the kernel lives in a dedicated non-contracting TU
  `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with
  `-ffp-contract=off` (`core/src/meson.build`), and
  `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls
  `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar
  `adm_dwt2_s` carries the matching function-scoped guard, so both
  sides of the comparison are non-contracting and the 1-ULP FMA gap
  that motivated the row cannot arise.
- The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was
  fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by
  `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity
  tests for the remaining uncovered kernels.
- `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar
  bit-exactness by bit pattern and is registered in the default `fast`
  suite.

The row stays in "Recently closed" (exactly one row for the id),
rewritten in past tense with real commit shas, and the stale "stays
Open" note is marked superseded so the next session does not
re-investigate. Documentation only — no code or behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was
self-contradictory: a 2026-06-27 note inside the
T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open"
for a correct re-attempt, while its Recently-closed row already
recorded a 2026-08-30 closure — and that closure narrative described
only the integer-path dropped-tap defect and cited a branch name
(`fix/adr-1057-arm-fma-drift`) instead of a commit.

Re-verified against origin/master, with no defective code left to point
at:

- The float-ADM follow-up the row asked for landed in `a6c4dfffb`
  (PR #853): the kernel lives in a dedicated non-contracting TU
  `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with
  `-ffp-contract=off` (`core/src/meson.build`), and
  `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls
  `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar
  `adm_dwt2_s` carries the matching function-scoped guard, so both
  sides of the comparison are non-contracting and the 1-ULP FMA gap
  that motivated the row cannot arise.
- The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was
  fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by
  `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity
  tests for the remaining uncovered kernels.
- `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar
  bit-exactness by bit pattern and is registered in the default `fast`
  suite.

The row stays in "Recently closed" (exactly one row for the id),
rewritten in past tense with real commit shas, and the stale "stays
Open" note is marked superseded so the next session does not
re-investigate. Documentation only — no code or behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was
self-contradictory: a 2026-06-27 note inside the
T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open"
for a correct re-attempt, while its Recently-closed row already
recorded a 2026-08-30 closure — and that closure narrative described
only the integer-path dropped-tap defect and cited a branch name
(`fix/adr-1057-arm-fma-drift`) instead of a commit.

Re-verified against origin/master, with no defective code left to point
at:

- The float-ADM follow-up the row asked for landed in `a6c4dfffb`
  (PR #853): the kernel lives in a dedicated non-contracting TU
  `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with
  `-ffp-contract=off` (`core/src/meson.build`), and
  `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls
  `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar
  `adm_dwt2_s` carries the matching function-scoped guard, so both
  sides of the comparison are non-contracting and the 1-ULP FMA gap
  that motivated the row cannot arise.
- The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was
  fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by
  `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity
  tests for the remaining uncovered kernels.
- `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar
  bit-exactness by bit pattern and is registered in the default `fast`
  suite.

The row stays in "Recently closed" (exactly one row for the id),
rewritten in past tense with real commit shas, and the stale "stays
Open" note is marked superseded so the next session does not
re-investigate. Documentation only — no code or behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 7, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was
self-contradictory: a 2026-06-27 note inside the
T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open"
for a correct re-attempt, while its Recently-closed row already
recorded a 2026-08-30 closure — and that closure narrative described
only the integer-path dropped-tap defect and cited a branch name
(`fix/adr-1057-arm-fma-drift`) instead of a commit.

Re-verified against origin/master, with no defective code left to point
at:

- The float-ADM follow-up the row asked for landed in `a6c4dfffb`
  (PR #853): the kernel lives in a dedicated non-contracting TU
  `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with
  `-ffp-contract=off` (`core/src/meson.build`), and
  `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls
  `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar
  `adm_dwt2_s` carries the matching function-scoped guard, so both
  sides of the comparison are non-contracting and the 1-ULP FMA gap
  that motivated the row cannot arise.
- The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was
  fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by
  `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity
  tests for the remaining uncovered kernels.
- `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar
  bit-exactness by bit pattern and is registered in the default `fast`
  suite.

The row stays in "Recently closed" (exactly one row for the id),
rewritten in past tense with real commit shas, and the stale "stays
Open" note is marked superseded so the next session does not
re-investigate. Documentation only — no code or behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 7, 2026
…#1331)

The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was
self-contradictory: a 2026-06-27 note inside the
T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open"
for a correct re-attempt, while its Recently-closed row already
recorded a 2026-08-30 closure — and that closure narrative described
only the integer-path dropped-tap defect and cited a branch name
(`fix/adr-1057-arm-fma-drift`) instead of a commit.

Re-verified against origin/master, with no defective code left to point
at:

- The float-ADM follow-up the row asked for landed in `a6c4dfffb`
  (PR #853): the kernel lives in a dedicated non-contracting TU
  `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with
  `-ffp-contract=off` (`core/src/meson.build`), and
  `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls
  `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar
  `adm_dwt2_s` carries the matching function-scoped guard, so both
  sides of the comparison are non-contracting and the 1-ULP FMA gap
  that motivated the row cannot arise.
- The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was
  fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by
  `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity
  tests for the remaining uncovered kernels.
- `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar
  bit-exactness by bit pattern and is registered in the default `fast`
  suite.

The row stays in "Recently closed" (exactly one row for the id),
rewritten in past tense with real commit shas, and the stale "stays
Open" note is marked superseded so the next session does not
re-investigate. Documentation only — no code or behaviour change.

Co-authored-by: Lusoris <lusoris@pm.me>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant