Skip to content

refactor(simd): bring the integer motion SIMD kernels to the lint and HISS standard (ADR-1142) - #1868

Merged
lusoris merged 1 commit into
masterfrom
refactor/std-motion-simd
Oct 2, 2026
Merged

lusoris merged 1 commit into
masterfrom
refactor/std-motion-simd

Conversation

@lusoris

@lusoris lusoris commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

First of four PRs for the 16 files that still had no SPDX-License-Identifier line because they carried baselined debt. This one covers the integer motion SIMD kernels: core/src/feature/x86/motion_avx2.c, core/src/feature/x86/motion_avx512.c and core/src/feature/arm64/motion_neon.c are at the lint and HISS standard, carry their SPDX line, and no bit of any score moves.

cpu cuda hip sycl arm64
x86/motion_avx2.c, clang-tidy 7 → 0 7 → 0 7 → 0 7 → 0 n/a
x86/motion_avx512.c, clang-tidy 18 → 0 18 → 0 18 → 0 18 → 0 n/a
arm64/motion_neon.c, clang-tidy n/a n/a n/a n/a 2 → 0
lane total in scripts/ci/tidy-baseline-<lane>.json 310 → 285 662 → 637 664 → 639 766 → 741 558 → 556

HISS: ten rows gone (three in motion_avx2.c, six in motion_avx512.c, one in motion_neon.c, all "function over 60 lines"); .standards-baseline.json 242 → 232, recorded with praetorctl baseline --record; the README count follows. Uncited NOLINTs: 0, and the three files have no NOLINT at all.

SPDX: BSD-2-Clause-Patent on all three. scripts/dev/relicense_fork_files.py --list classifies them upstream-path (the files exist in Netflix/vmaf), so they keep Netflix's terms (ADR-1250).

What changed

Each motion_score_pipeline_{8,16}_* keeps its name and signature and is a row loop:

Stage AVX2 AVX-512
vertical pass of one row; returns whether any y_row value is non-zero y_conv_row_{8,16}_avx2 over y_conv16_8_avx2 / y_conv8_16_avx2 y_conv_row_{8,16}_avx512 over y_conv32_8_avx512 / y_conv16_16_avx512
horizontal pass + abs + SAD of one row x_conv_row_sad_avx2 over x_conv_abs8_avx2, x_conv_abs_scalar, hsum_epi32_avx2 x_conv_row_sad_avx512 over x_conv_abs16_avx512, x_conv_abs_scalar

The three test-only kernels of motion_avx512.c (y_convolution_8_avx512, y_convolution_16_avx512, x_convolution_16_avx512) share filter5_epu32_avx512() and filter5_scalar(), and x_convolution_16_neon is three passes over two helpers. sad_avx512 is untouched.

Every statement that computes moved whole: the widths of the accumulators, the order of the widening additions, the rounding constants, the logical _mm256_srlv_epi64 of the AVX2 16-bit path and the arithmetic _mm512_srav_epi64 of the AVX-512 one are as they were. All arithmetic in these files is integer.

The findings, by check:

Check Count Fixed by
readability-function-size 10 the split
misc-use-internal-linkage 9 each file includes its own header, so the exported functions are checked against their declarations
readability-isolate-declaration 4 one declaration per line
readability-braces-around-statements 4 the edge loop of y_convolution_8_avx512 is one braced helper

Not one bit moves

  • Kernels, old against new in one process. Master's and this PR's objects linked into one harness under different names, compared on generated input (noise, extremes, flat, near-equal pictures; widths 3 to 140 and 576, 1919, 1920, 3840; 8, 10, 12 and 16 bit; the returned SAD and the whole y_row scratch; the destination buffers of the three convolution kernels including their padding): 286 952 comparisons, 0 mismatches, with GCC 16, clang 22 and icx 2026.0. x_convolution_16_neon under qemu-aarch64: 3 920 comparisons, 0 mismatches, with aarch64 GCC and clang.

  • Scores. motion, motion=debug=true, motion_v2, float_motion and the default model at --precision max, this PR against master 513d2a6fc:

    Build Fixtures Dispatch Cases identical Values
    x86-64 GCC Netflix 576x324 at 8, 10, 12, 16 bit; both 1080p checkerboards; BBB 3840x2160, 8 frames scalar, AVX2, AVX-512 105 of 105 5 964
    aarch64 GCC under qemu the same without BBB, 6 frames scalar, NEON 48 of 48 1 050
  • Netflix golden gate. x86-64 GCC: 271 passed, 12 skipped. aarch64 GCC under qemu: 271 passed, 12 skipped (86 minutes; the aarch64 binary was built from this PR's sources on 25fe95d73, one rebase before the pushed head, so master's two later aarch64 changes, refactor(feature): bring the NEON float ADM kernels to the lint and HISS standard (ADR-1142) #1865 and the NEON psnr_hvs fix, are not in that binary). x_convolution_16_neon, the one aarch64 function this PR changes, is called by core/test/test_motion_neon.c only; no extractor reaches it.

  • Unit tests. --suite=fast on the CPU build: 244 of 244 (includes test_motion_avx512_parity, test_motion_v2_simd, test_integer_motion_coverage). Under qemu: test_motion_neon and test_motion_pipeline_neon pass.

  • Speed. The helpers are static inline and the two AVX2 pipelines are the only symbols of their object file, as before. One process, 1920x1080, minimum of 300 calls, old then new, load 56 on the host:

    GCC clang icx
    pipeline_8_avx2 0.962 → 0.948 ms 0.860 → 0.890 0.891 → 0.929
    pipeline_16_avx2 (10 bit) 1.479 → 1.449 1.345 → 1.395 1.480 → 1.475
    pipeline_8_avx512 0.517 → 0.533 0.565 → 0.519 0.592 → 0.624
    pipeline_16_avx512 (10 bit) 0.742 → 0.755 0.792 → 0.755 0.842 → 0.903

    Between 0.92x and 1.07x with no direction; not a timing claim on a host this loaded.

  • GPU twins. No header changes; feature/integer_motion.h is untouched, so the CUDA, HIP and SYCL motion twins compile the same text.

  • scripts/dev/preflight.sh --stage msvcism: pass.

Lane notes

Type

  • feat — new feature
  • fix — bug fix
  • perf — performance improvement
  • refactor — no behavior change
  • docs — documentation only
  • test — test-only
  • build / ci — tooling / infra
  • port — cherry-pick from upstream Netflix/vmaf
  • sycl / cuda / simd — backend-specific

Checklist

  • Commits follow Conventional Commits (the commit-msg hook enforces this).
  • make format && make lint is green locally. (clang-format and the commit hooks; clang-tidy 0 for the three files in every lane that reads them.)
  • Unit tests pass: python3 scripts/ci/run_meson_test.py -- -C build. (--suite=fast on a CPU build: 244 of 244.)
  • If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2. (Bit-identical: old and new kernels compared directly, and scores under every dispatch.)
  • If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap under "Known follow-ups" below. (No arithmetic changes; nothing for a twin to follow.)
  • If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header (see CONTRIBUTING.md). (None added; the SPDX line of the three files added.)
  • If this is a breaking change, the commit message uses ! or BREAKING CHANGE: and the migration path is documented below. (Not breaking.)
  • If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/<NNNN-slug>.md. (No ADR: ADR-1142 is the rule.)

Bug-status hygiene (ADR-0165)

  • docs/state.md updated in this PR. no state delta: refactor to the lint and HISS standard; no row names these files' debt.

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
  • If I believe a golden value must change, I have explained why below AND pinged @lusoris for a CODEOWNERS exception. (None changes.)

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: the findings and the split are described here and in the rebase note.
  • Decision matrix — no alternatives: only-one-way fix. ADR-1142 is the rule; the split follows each pipeline's own stages.
  • AGENTS.md invariant note — core/src/feature/x86/AGENTS.d/motion.md, "Pipelines are row loops over inlined stages".
  • Reproducer / smoke-test command — pasted below under "Reproducer".
  • CHANGELOG fragment — changelog.d/changed/std-motion-simd.md.
  • Rebase note — docs/rebase-notes.md, "Integer motion SIMD kernels split into stages".

Reproducer

meson setup /tmp/tidy-cpu core -Denable_cuda=false -Denable_sycl=false -Db_lto=false
ninja -C /tmp/tidy-cpu
python3 scripts/ci/write-compile-commands.py --build-dir /tmp/tidy-cpu
python3 scripts/ci/tidy-ratchet.py --lane cpu --build-dir /tmp/tidy-cpu \
  --only core/src/feature/x86/motion_avx2.c --only core/src/feature/x86/motion_avx512.c   # 0 warnings
praetorctl baseline --verify      # 232 recorded
for mask in 0 48 56; do           # AVX-512, AVX2, scalar
  /tmp/tidy-cpu/tools/vmaf -r python/test/resource/yuv/src01_hrc00_576x324.yuv \
    -d python/test/resource/yuv/src01_hrc01_576x324.yuv -w 576 -h 324 -p 420 -b 8 \
    --no_prediction --feature motion=debug=true --feature motion_v2 --cpumask $mask \
    --precision max --json -o /dev/stdout -q | sha256sum
done
python3 scripts/ci/run_meson_test.py -- -C /tmp/tidy-cpu --suite=fast   # 244 of 244
make test-netflix-golden          # 271 passed, 12 skipped
make test-netflix-golden-arm64    # 271 passed, 12 skipped

Known follow-ups

  • The other 13 of my 16 files: the extractor sources and their tests, the CLI tool files, and compat/python-vmaf/__init__.py, one PR each.

@github-actions github-actions Bot added the type:refactor Internal refactor label Oct 2, 2026
… HISS standard (ADR-1142) (#1868)

* refactor(simd): bring the integer motion SIMD kernels to the lint and HISS standard (ADR-1142)

core/src/feature/x86/motion_avx2.c, motion_avx512.c and
core/src/feature/arm64/motion_neon.c had 7, 18 and 2 clang-tidy findings
and ten functions over 60 lines, which is why the SPDX backfill (#1739)
could not touch them.

Each motion_score_pipeline_{8,16}_* keeps its name and signature and is a
row loop over inlined stages: the vertical pass of one row
(y_conv_row_{8,16}_*), which reports whether the row is non-zero, and the
horizontal pass (x_conv_row_sad_*), each with one vector block helper and
one scalar helper for the columns the vector loop cannot reach. The three
test-only convolution kernels of motion_avx512.c share two filter
helpers; x_convolution_16_neon is three passes over two helpers. Each
file includes its own header. All arithmetic is integer and every
statement moved whole.

Verified: master's and the new objects in one process, 286 952
comparisons on generated input with GCC, clang and icx, 0 mismatches
(NEON: 3 920 under qemu). motion, motion_v2, float_motion and the default
model at --precision max: 105 of 105 cases identical on x86-64 under
scalar, AVX2 and AVX-512 dispatch, 48 of 48 on aarch64 under scalar and
NEON. Netflix golden gate on x86-64 GCC: 271 passed, 12 skipped. Fast
suite: 244 of 244.

clang-tidy 0 in every lane that reads the files (lane totals: cpu 310 to
285, cuda 662 to 637, hip 664 to 639, sycl 766 to 741, arm64 558 to 556).
HISS baseline 242 to 232. The three files carry
SPDX-License-Identifier: BSD-2-Clause-Patent (ADR-1250: upstream paths).
@lusoris
lusoris force-pushed the refactor/std-motion-simd branch from f5e310f to b801f28 Compare October 2, 2026 17:14
@lusoris
lusoris merged commit b801f28 into master Oct 2, 2026
3 of 78 checks passed
@lusoris
lusoris deleted the refactor/std-motion-simd branch October 2, 2026 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:refactor Internal refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant