Repository navigation
fix(arm64): make adm_dwt2_8_neon bit-exact with the scalar kernel - #1154
Merged
Merged
Conversation
ADR-1057 was a dropped filter tap in this kernel that reached master and drifted the ARM Netflix golden. It survived every test because NO unit test covered adm_dwt2 on any architecture -- the SIMD suite reaches adm_cm, not adm_dwt2. This adds that test, and it immediately found two more divergences. 1. Stale intermediate columns for widths congruent to 8 mod 16. integer_adm.c dispatches this kernel on `!(w % 8)`, but its vertical pass advances 16 columns per iteration and stops at `w - 15`, with no scalar tail. For a width like 584 or 776 the final 8 columns of tmplo/tmphi were never written in that iteration, so the horizontal pass consumed whatever the previous row had left in the shared buf->tmp_ref scratch. Fixed by adding the tail. 2. Unmirrored final output column, at every width. The horizontal pass walks tmplo/tmphi with plain pointer arithmetic and never consults ind_x, so for the last column it read tmplo[2j + 2] == tmplo[w]. That is one past the lo half -- the first element of tmphi, since tmphi == tmplo + w -- where the scalar mirrors the index back to w - 1. Fixed by recomputing that column with the indices dwt2_src_indices_filt() actually specifies. NEITHER CHANGES A VMAF SCORE on anything tested. I built a synthetic 584-wide clip specifically to trigger (1), and pre-fix, post-fix and the x86 scalar all produce 80.322171: ADM crops the affected border columns before they reach a feature value. The Netflix golden fixtures are 576, 1280 and 1920 wide -- all multiples of 16 -- so (1) could never have surfaced there either. This is a correctness and cross-backend-parity fix, not a scoring fix, and it is worth making because (1) reads scratch entries that were never written in that iteration: the output depends on the previous row's residue. The new test asserts bit-exactness across six geometries covering w % 16 == 0, w % 16 == 8 and odd heights. It reproduced both defects before the fix (24x16: 160 mismatches, 128 of them the stale columns) and reports 0 after. Verified: aarch64 fast suite 98/98 under qemu-aarch64-static, x86 105/105, clang-format and assertion-density clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 30, 2026
5 of 9 tasks
lusoris
added a commit
that referenced
this pull request
Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was self-contradictory: a 2026-06-27 note inside the T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open" for a correct re-attempt, while its Recently-closed row already recorded a 2026-08-30 closure — and that closure narrative described only the integer-path dropped-tap defect and cited a branch name (`fix/adr-1057-arm-fma-drift`) instead of a commit. Re-verified against origin/master, with no defective code left to point at: - The float-ADM follow-up the row asked for landed in `a6c4dfffb` (PR #853): the kernel lives in a dedicated non-contracting TU `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with `-ffp-contract=off` (`core/src/meson.build`), and `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar `adm_dwt2_s` carries the matching function-scoped guard, so both sides of the comparison are non-contracting and the 1-ULP FMA gap that motivated the row cannot arise. - The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity tests for the remaining uncovered kernels. - `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar bit-exactness by bit pattern and is registered in the default `fast` suite. The row stays in "Recently closed" (exactly one row for the id), rewritten in past tense with real commit shas, and the stale "stays Open" note is marked superseded so the next session does not re-investigate. Documentation only — no code or behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was self-contradictory: a 2026-06-27 note inside the T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open" for a correct re-attempt, while its Recently-closed row already recorded a 2026-08-30 closure — and that closure narrative described only the integer-path dropped-tap defect and cited a branch name (`fix/adr-1057-arm-fma-drift`) instead of a commit. Re-verified against origin/master, with no defective code left to point at: - The float-ADM follow-up the row asked for landed in `a6c4dfffb` (PR #853): the kernel lives in a dedicated non-contracting TU `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with `-ffp-contract=off` (`core/src/meson.build`), and `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar `adm_dwt2_s` carries the matching function-scoped guard, so both sides of the comparison are non-contracting and the 1-ULP FMA gap that motivated the row cannot arise. - The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity tests for the remaining uncovered kernels. - `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar bit-exactness by bit pattern and is registered in the default `fast` suite. The row stays in "Recently closed" (exactly one row for the id), rewritten in past tense with real commit shas, and the stale "stays Open" note is marked superseded so the next session does not re-investigate. Documentation only — no code or behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 6, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was self-contradictory: a 2026-06-27 note inside the T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open" for a correct re-attempt, while its Recently-closed row already recorded a 2026-08-30 closure — and that closure narrative described only the integer-path dropped-tap defect and cited a branch name (`fix/adr-1057-arm-fma-drift`) instead of a commit. Re-verified against origin/master, with no defective code left to point at: - The float-ADM follow-up the row asked for landed in `a6c4dfffb` (PR #853): the kernel lives in a dedicated non-contracting TU `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with `-ffp-contract=off` (`core/src/meson.build`), and `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar `adm_dwt2_s` carries the matching function-scoped guard, so both sides of the comparison are non-contracting and the 1-ULP FMA gap that motivated the row cannot arise. - The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity tests for the remaining uncovered kernels. - `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar bit-exactness by bit pattern and is registered in the default `fast` suite. The row stays in "Recently closed" (exactly one row for the id), rewritten in past tense with real commit shas, and the stale "stays Open" note is marked superseded so the next session does not re-investigate. Documentation only — no code or behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 7, 2026
The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was self-contradictory: a 2026-06-27 note inside the T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open" for a correct re-attempt, while its Recently-closed row already recorded a 2026-08-30 closure — and that closure narrative described only the integer-path dropped-tap defect and cited a branch name (`fix/adr-1057-arm-fma-drift`) instead of a commit. Re-verified against origin/master, with no defective code left to point at: - The float-ADM follow-up the row asked for landed in `a6c4dfffb` (PR #853): the kernel lives in a dedicated non-contracting TU `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with `-ffp-contract=off` (`core/src/meson.build`), and `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar `adm_dwt2_s` carries the matching function-scoped guard, so both sides of the comparison are non-contracting and the 1-ULP FMA gap that motivated the row cannot arise. - The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity tests for the remaining uncovered kernels. - `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar bit-exactness by bit pattern and is registered in the default `fast` suite. The row stays in "Recently closed" (exactly one row for the id), rewritten in past tense with real commit shas, and the stale "stays Open" note is marked superseded so the next session does not re-investigate. Documentation only — no code or behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 7, 2026
…#1331) The ledger row for T-NEON-FMA-FLOAT-ADM-DWT2-2026-06-06 was self-contradictory: a 2026-06-27 note inside the T-MASTER-CI-TSAN-ARM-GOLDEN-2026-06-27 row said the bug "stays Open" for a correct re-attempt, while its Recently-closed row already recorded a 2026-08-30 closure — and that closure narrative described only the integer-path dropped-tap defect and cited a branch name (`fix/adr-1057-arm-fma-drift`) instead of a commit. Re-verified against origin/master, with no defective code left to point at: - The float-ADM follow-up the row asked for landed in `a6c4dfffb` (PR #853): the kernel lives in a dedicated non-contracting TU `core/src/feature/arm64/float_adm_dwt2_neon.c`, built with `-ffp-contract=off` (`core/src/meson.build`), and `adm_dwt2_dispatch()` in `core/src/feature/adm.c` calls `float_adm_dwt2_neon()` under `VMAF_ARM_CPU_FLAG_NEON`. The scalar `adm_dwt2_s` carries the matching function-scoped guard, so both sides of the comparison are non-contracting and the 1-ULP FMA gap that motivated the row cannot arise. - The integer `idx < 3` dropped filter tap in `adm_dwt2_8_neon` was fixed by `a013c1410` (PR #1134) and hardened to bit-exactness by `89a8e3258` (PR #1154); `6d61106ed` (PR #1156) added NEON parity tests for the remaining uncovered kernels. - `core/test/test_float_adm_dwt2_neon.c` gates NEON-vs-scalar bit-exactness by bit pattern and is registered in the default `fast` suite. The row stays in "Recently closed" (exactly one row for the id), rewritten in past tense with real commit shas, and the stale "stays Open" note is marked superseded so the next session does not re-investigate. Documentation only — no code or behaviour change. Co-authored-by: Lusoris <lusoris@pm.me> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ADR-1057 was a dropped filter tap in
adm_dwt2_8_neonthat reached master and drifted the ARM Netflix golden. Itsurvived every test because no unit test covered
adm_dwt2on anyarchitecture —
test_integer_adm_simd.creachesadm_cm, notadm_dwt2.This adds that test. It immediately found two more divergences from the scalar
kernel.
1. Stale intermediate columns for widths ≡ 8 (mod 16)
integer_adm.cdispatches this kernel on!(w % 8):but its vertical pass advances 16 columns per iteration and stops at
w - 15, with no scalar tail:For a width such as 584 or 776 the final 8 columns of
tmplo/tmphiwerenever written in that iteration, so the horizontal pass consumed whatever the
previous row had left in the shared
buf->tmp_refscratch.2. Unmirrored final output column, at every width
The horizontal pass walks
tmplo/tmphiwith plain pointer arithmetic andnever consults
ind_x, so for the last column it readtmplo[2j + 2]==tmplo[w]— one past the lo half, i.e.tmphi[0], sincetmphi == tmplo + w—where the scalar mirrors the index back to
w - 1.Score impact: none
I want to be precise about this rather than oversell it. I built a synthetic
584-wide clip specifically to trigger defect 1, and measured:
80.32217180.32217180.322171ADM crops the affected border columns before they reach a feature value. And the
Netflix golden fixtures are 576, 1280 and 1920 wide — all multiples of 16 —
so defect 1 could never have surfaced there either.
This is a correctness and cross-backend-parity fix, not a scoring fix. It is
still worth making: defect 1 reads scratch entries that were never written in
that iteration, so the kernel's output depends on the previous row's residue.
Type
fix— bug fixtest— adds the missing coveragesimd— backend-specificChecklist
make lintclean;clang-formatandassertion-densitypass on both touched files.qemu-aarch64-static, x86 105/105..ccarries the license header.Bug-status hygiene
docs/state.mdupdated —T-ADM-DWT2-NEON-PARITY-2026-08-30under Recently closed.Netflix golden-data gate
assertAlmostEqual(...)score. Verified score-neutral on the 576-wide golden pair (76.667831pre- and post-fix) and on the synthetic 584-wide clip.Deep-dive deliverables
docs/state.mdand the commit message.AGENTS.mdinvariant note — captured indocs/rebase-notes.mdinstead: the invariant is about the relationship between the dispatch granularity (w % 8) and the vector width (16), which is not a package-scoped concern.changelog.d/fixed/adm-dwt2-neon-scalar-parity.md.docs/rebase-notes.md, "fix/adm-dwt2-neon-parity".Reproducer
Show the defects on the pre-fix kernel — revert
adm_neon.cand the test reportsper-geometry mismatch counts:
128 = 4 columns × 8 rows × 4 subbands, exactly the region the missing vertical
tail leaves unwritten at w=24.
Known follow-ups
arm64/kernels.
adm_cmhas coverage viatest_integer_adm_simd.c; the rest ofadm_neon.c(decouple, csf, sum) does not.