Skip to content

fix(cpu): name xmm0 in the AVX-512 warm-up's clobber list so a clang LTO build keeps its caller's value - #1886

Merged
lusoris merged 1 commit into
masterfrom
fix/cpu-avx512-warmup-clobber
Oct 2, 2026
Merged

lusoris merged 1 commit into
masterfrom
fix/cpu-avx512-warmup-clobber

Conversation

@lusoris

@lusoris lusoris commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Summary

vmaf_init_cpu() zeroed a value its caller held in xmm0 when a clang build with link-time optimisation ran on a host with AVX-512. This PR names xmm0 in the clobber list of the AVX-512 warm-up, which fixes the hosted Ubuntu clang failure of test_ciede_device_math (item 4 of T-CI-MASTER-FIRST-FULL-RUN-2026-10-02).

Opens and closes T-CPU-AVX512-WARMUP-CLOBBER-2026-10-02.

What was wrong

On a host with AVX-512, vmaf_init_cpu() runs one 512-bit instruction as inline assembly, so the first frame does not wait for the 512-bit units to power up. The statement declared "zmm0" clobbered. clang drops the clobber of a register the enclosing function's target does not have, and nothing that contains the statement is compiled for AVX-512. GCC maps zmm0 and xmm0 to one hard register and is not affected.

Out of line the statement is harmless, because xmm0 is caller-saved. The default build is link-time optimised (b_lto=true). clang inlined vmaf_init() and vmaf_init_cpu() into the test's check_case(), kept -20 * log10(x) in xmm0 across the statement and added 45 to the zero it left. Assembly of the linked test:

mulsd   .LCPI2_66(%rip), %xmm0      # -20 * log10(...)
vpxord  %zmm0, %zmm0, %zmm0         # the warm-up
addsd   .LCPI2_67(%rip), %xmm0      # + 45

The hosted line is 96x64 8-bit fmt 1: cpu=35.418753523280415 replay=45 delta=9.581e+00. It appears on runners with AVX-512 and not on the others, which is why the same job passed on a27ebd114. ciede and the test are correct; T-CIEDE-CLANG-POWF-BUILTIN-2026-10-02 (ADR-1467) was a different difference of 2e-11.

A short program shows the compiler behaviour alone (log10(x) * -20, the statement, + 45):

Clobber list clang 22.1.8 / 23.1.1 clang with -mavx512f GCC 16.2.1
"zmm0" 45 (wrong) 35.457574905606748 35.457574905606748
"xmm0" 35.457574905606748 35.457574905606748 35.457574905606748
"xmm0", "zmm0" 35.457574905606748 35.457574905606748 35.457574905606748

What changes

  • The statement moves to core/src/x86/avx512_warm_up.h as vmaf_x86_avx512_warm_up(), with the clobber list "xmm0", "zmm0". vmaf_init_cpu() calls it.
  • test_cpu gains test_avx512_warm_up_keeps_xmm0: a product held in xmm0 across the statement. It returns early on a host without AVX-512.
  • test_inline_asm_clobber_contract.py (device-free) requires xmmN next to every ymmN / zmmN clobber of an inline-assembly statement under core/src, with four planted regressions.

Reach

Any clang build with link-time optimisation that inlines vmaf_init_cpu() next to a live value in xmm0, on an AVX-512 host. The vmaf tool of the clang 22.1.8 LTO build returns the same 2256 per-frame values and pooled scores before and after the fix (Netflix 576x324 pair, 15 extractors, two models, --precision max), so no score of that build was wrong. Another compiler version or another caller can inline differently. GCC builds were never affected.

Verification

Ryzen 9 9950X3D (AVX-512). clang 22.1.8 is the hosted job's build from apt.llvm.org (++20260714...ca7933e47d3a), installed in a throwaway container of the dev image (glibc 2.43); the hosted runner is Ubuntu 24.04.

Build Before (master 10d6a0505) After
clang 22.1.8, -O3 -flto (the hosted Ubuntu clang configuration), all suites test_ciede_device_math fails with the hosted line 256 passed, 0 failed, 2 skipped
clang 23.1.1, -O3 -flto, host same failure test_ciede_device_math and test_cpu pass
GCC 16.2.1, fast suite, host passes 247 passed, 0 failed
new test_cpu case with the old list, clang 22.1.8 fails with and without LTO passes
  • clang-tidy cpu lane in the dev container: exit 0 (330 translation units, 70 warnings, the baseline). The new header has no finding.
  • scripts/dev/preflight.sh --stage msvcism: pass. Under MSVC the function is empty, as the statement was skipped there before.
  • Not run: a hosted runner. The job needs an AVX-512 runner to show either result.

Type

  • feat — new feature
  • fix — bug fix
  • perf — performance improvement
  • refactor — no behavior change
  • docs — documentation only
  • test — test-only
  • build / ci — tooling / infra
  • port — cherry-pick from upstream Netflix/vmaf
  • sycl / cuda / simd — backend-specific

Checklist

  • Commits follow Conventional Commits (the commit-msg hook enforces this).
  • make format && make lint is green locally. Not run as one target: clang-format is clean on the touched files and the cpu clang-tidy lane was measured in the dev container (exit 0).
  • Unit tests pass: python3 scripts/ci/run_meson_test.py -- -C build.
  • If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2. Not applicable: no SIMD or GPU path.
  • If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap under "Known follow-ups" below. Not applicable.
  • If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header (see CONTRIBUTING.md).
  • If this is a breaking change, the commit message uses ! or BREAKING CHANGE: and the migration path is documented below. Not a breaking change.
  • If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/<NNNN-slug>.md. No ADR.

Bug-status hygiene (ADR-0165)

  • docs/state.md updated in this PR with a row in the appropriate section (Open / Recently closed / Confirmed not-affected / Deferred), OR no state delta: REASON.

T-CPU-AVX512-WARMUP-CLOBBER-2026-10-02 under "Recently closed"; item 4 of T-CI-MASTER-FIRST-FULL-RUN-2026-10-02 updated (six items fixed, one open).

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
  • If I believe a golden value must change, I have explained why below AND pinged @lusoris for a CODEOWNERS exception.

Cross-backend numerical results

no numeric change: CPU feature detection only; the vmaf tool returns the same 2256 values before and after

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: one statement; the cause, the reproducer and the measurements are in this body and in the docs/state.md row.
  • Decision matrix — no alternatives: only-one-way fix. The statement has to declare the register it writes in a form every compiler honours; removing the warm-up was the other option and would change start-up behaviour for no correctness gain.
  • AGENTS.md invariant note — core/src/AGENTS.d/inline-asm-clobbers.md.
  • Reproducer / smoke-test command — pasted below under "Reproducer".
  • CHANGELOG fragment — changelog.d/fixed/cpu-avx512-warmup-clobber.md.
  • Rebase note — entry added to docs/rebase-notes.md.

Reproducer

# AVX-512 host, clang, default options (b_lto=true)
CC=clang CXX=clang++ meson setup build-clang core -Denable_float=true
ninja -C build-clang test/test_cpu test/test_ciede_device_math
build-clang/test/test_ciede_device_math   # master: cpu=35.418753523280415 replay=45
build-clang/test/test_cpu                 # this branch: 2 tests run, 2 passed
python3 core/test/test_inline_asm_clobber_contract.py

Known follow-ups

no ffmpeg-patches update needed: no public surface changes.

The Ubuntu clang jobs also fail their tox step on a Cython compile error (adm_dwt2_cy.c, incompatible pointer types under clang 22). That is a separate failure and not part of this PR.

Breaking changes / migration

None.

@github-actions github-actions Bot added the type:bug Something isn't working label Oct 2, 2026
Comment thread core/test/test_cpu.c
vmaf_x86_avx512_warm_up();
const double after = scaled + offset;

mu_assert("the AVX-512 warm-up changed a value its caller holds in xmm0", after == expected);
…LTO build keeps its caller's value (#1886)

* fix(cpu): name xmm0 in the AVX-512 warm-up's clobber list so a clang LTO build keeps its caller's value

On a host with AVX-512, vmaf_init_cpu() runs one 512-bit instruction as
inline assembly so the first frame does not wait for the 512-bit units.
The statement declared "zmm0" clobbered. clang drops the clobber of a
register the enclosing function's target does not have, and nothing that
contains the statement is compiled for AVX-512. Out of line that is
harmless (xmm0 is caller-saved); the default build is link-time
optimised, and clang inlined vmaf_init() and vmaf_init_cpu() into
test_ciede_device_math, kept -20 * log10(x) in xmm0 across the statement
and added 45 to the zero it left. That is the hosted `Ubuntu clang`
failure (cpu=35.418753523280415 replay=45), seen only on runners with
AVX-512. GCC maps zmm0 to the same hard register as xmm0 and was never
affected.

The statement moves to x86/avx512_warm_up.h as
vmaf_x86_avx512_warm_up() with the list "xmm0", "zmm0". test_cpu holds a
product in xmm0 across it (fails with the old list under clang 22.1.8,
with and without LTO); test_inline_asm_clobber_contract.py requires xmmN
next to every ymmN / zmmN clobber under core/src.

Measured with the hosted compiler (clang 22.1.8, -O3 -flto, glibc 2.43,
dev container) on a Ryzen 9 9950X3D: all suites 256 passed, 0 failed
(before: test_ciede_device_math failed). The vmaf tool of that build
returns the same 2256 per-frame values before and after. GCC 16.2.1 fast
suite: 247 passed.
@lusoris
lusoris force-pushed the fix/cpu-avx512-warmup-clobber branch from 7995621 to 6bb4cc7 Compare October 2, 2026 23:02
@lusoris
lusoris merged commit 6bb4cc7 into master Oct 2, 2026
3 of 54 checks passed
@lusoris
lusoris deleted the fix/cpu-avx512-warmup-clobber branch October 2, 2026 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants