Repository navigation
fix(ciede): write the squares of ciede.c as products so every compiler computes the same ciede2000 (ADR-1467) - #1862
Merged
Merged
Conversation
lusoris
force-pushed
the
fix/ciede-powf-explicit
branch
from
October 2, 2026 16:35
228e28e to
00028f4
Compare
lusoris
force-pushed
the
fix/ciede-powf-explicit
branch
from
October 2, 2026 16:48
00028f4 to
6510d61
Compare
… HISS standard (ADR-1142) (#1868) * refactor(simd): bring the integer motion SIMD kernels to the lint and HISS standard (ADR-1142) core/src/feature/x86/motion_avx2.c, motion_avx512.c and core/src/feature/arm64/motion_neon.c had 7, 18 and 2 clang-tidy findings and ten functions over 60 lines, which is why the SPDX backfill (#1739) could not touch them. Each motion_score_pipeline_{8,16}_* keeps its name and signature and is a row loop over inlined stages: the vertical pass of one row (y_conv_row_{8,16}_*), which reports whether the row is non-zero, and the horizontal pass (x_conv_row_sad_*), each with one vector block helper and one scalar helper for the columns the vector loop cannot reach. The three test-only convolution kernels of motion_avx512.c share two filter helpers; x_convolution_16_neon is three passes over two helpers. Each file includes its own header. All arithmetic is integer and every statement moved whole. Verified: master's and the new objects in one process, 286 952 comparisons on generated input with GCC, clang and icx, 0 mismatches (NEON: 3 920 under qemu). motion, motion_v2, float_motion and the default model at --precision max: 105 of 105 cases identical on x86-64 under scalar, AVX2 and AVX-512 dispatch, 48 of 48 on aarch64 under scalar and NEON. Netflix golden gate on x86-64 GCC: 271 passed, 12 skipped. Fast suite: 244 of 244. clang-tidy 0 in every lane that reads the files (lane totals: cpu 310 to 285, cuda 662 to 637, hip 664 to 639, sycl 766 to 741, arm64 558 to 556). HISS baseline 242 to 232. The three files carry SPDX-License-Identifier: BSD-2-Clause-Patent (ADR-1250: upstream paths).
…r computes the same ciede2000 (ADR-1467) (#1862) * fix(ciede): write the squares of ciede.c as products so every compiler computes the same ciede2000 (ADR-1467) get_r_sub_t() in ciede.c wrote powf(degrees, 2). GCC emits the call; clang replaces it by degrees * degrees, the correctly rounded square. glibc's powf is not correctly rounded: on 0.12 % of the arguments, where the exact square is a tie, it returns the other neighbouring float. A clang build and a GCC build of the CPU ciede therefore differed on 65 of 180 measured frames, by up to 2.0e-11, on x86-64 and on aarch64. Both ways out were built and measured: keeping the call under clang (-fno-builtin-powf for this file) leaves the GCC build untouched but keeps the value a property of the C library, cannot cover icx (Intel's math library) and costs a clang build three powf calls per pixel; writing the product moves the GCC build and makes the value the same everywhere. The maintainer chose the product, provided the golden gate holds. ciede.c now writes degrees * degrees, and square(x), the double product of a float, for the 13 uses of pow(x, 2) (exact, so no value changes through them). ciede_device.h, the CUDA twin's arithmetic, multiplies as well instead of rounding the fp64 pow() to float. After: x86-64 and aarch64 builds with GCC and with clang return the same ciede2000 on 180 of 180 frames. A GCC build moves on 65 of them, by at most 1.96e-11; clang and icx builds do not move. Netflix golden gate: 271 passed, 12 skipped on x86-64 and aarch64, each with GCC and with clang. No fork snapshot stores ciede. The twins are closer to the CPU: against the GCC build ciede_cuda, ciede_sycl and ciede_hip are within 5.2e-12 (2.0e-11 before), and what is left is glibc's powf(x, 7). ciede_cuda itself moves on 3 of 180 frames, by at most 1.1e-13. LIBM_TWINS stays 1e-9. test_ciede_device_math holds under every compiler now and runs on every architecture. Also recorded, at the coordinator's request: with ADR-1461 (#1829) an icx build compiles svm.cpp, predict.c, model.c and libvmaf.c under the precise floating-point model instead of icx's fast default, and its predicted vmaf moved by up to 7.05e-12 on every frame with a non-zero score, towards the GCC build (frames with identical features but another vmaf: 153 of 163 before, 15 after). No extractor value moved. Row T-ICX-PREDICT-FP-MODEL-SCORE-MOVE-2026-10-02, a paragraph in docs/development/build-flags.md and a changelog fragment; measured by the SYCL lane, cited here. Closes T-CIEDE-CLANG-POWF-BUILTIN-2026-10-02 * docs: regenerate the indexes and the citation map after rebasing
lusoris
force-pushed
the
fix/ciede-powf-explicit
branch
from
October 2, 2026 17:14
6510d61 to
6f46d55
Compare
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A clang build and a GCC build of the CPU
ciedeextractor returned differentciede2000scores on the same machine, on x86-64 and on aarch64. This PR writes the squares ofciede.cas products, so every compiler and every C library computes the same value. ClosesT-CIEDE-CLANG-POWF-BUILTIN-2026-10-02(ADR-1467,docs/adr/1467-ciede-squares-as-products.md).get_r_sub_t()wrotepowf(degrees, 2). GCC calls the C library; clang replaces the call bydegrees * degrees. glibc'spowfis not correctly rounded, so the two differ on 0.12 % of the arguments.degrees * degreesin the source, andsquare(x)for the 13pow(x, 2)of the formula. No compiler flag. Decided by the maintainer: "Write the product, if golden holds".LIBM_TWINSstays1e-9.The first version of this PR took the other option (
-fno-builtin-powfforciede.cunder clang, GCC untouched). It is removed: the policy block, the separate library and its two tests are gone. Both options and their measurements are in the ADR.Which call, measured
Each power form of
ciede.ccompiled alone with the build's flags (-O3 -std=c23 -ffp-contract=off; icx with-fp-model=precise), then the object inspected:ciede.cpowf(x, 2)powfpowf(x, 7)powfpowfpowfpowf(25., 7)pow(x, 2)(13 uses)powpow(x, 7),pow(x, 2.4),pow(x, 1.0 / 3.0)powpowpowpow(25, 7)No exponent 0.5 occurs in the file. clang leaves
pow(x, 0.5),powf(x, 0.5f),pow(x, 3)andpow(10.0, x)as calls and turnspow(2, n)intoldexp, which is exact either way.powf(x, 2)→x * xget_r_sub_t()(0.12 %), the tiespow(x, 2)→(double)x * x,xafloatfloatis exact indouble)Scores on the final head
ciede2000at--precision maxon ten fixtures: the Netflix 576x324 pair at 8, 10, 12 and 16 bits and as 10-bit 4:2:2, Sparks 480x270, both 1920x1080 checkerboard pairs, BBB 1920x1080 (48 frames) and 3840x2160 (16 frames). 180 frames. Before = master5d67b4939, after =513d2a6fcplus this change (no CPU source differs between the two); the x86-64 builds were measured again on the final head,38e8ec0b0plus this change, with the same 180 values. aarch64 throughqemu-aarch6411.1.1.The GCC build against master, exactly (the same frames and values on x86-64 and aarch64):
The clang build against master: 0 of 180 move. icx: 0 of 180 move; it differs from GCC on 50 frames by at most 9.7e-12 afterwards (68 and 1.4e-11 before), through Intel's math library.
The
doublesquares change nothing: a GCC build with only thefloatproduct returns the same 180 values as this PR.Dispatch: host, AVX2-only and scalar agree on 180 of 180 for GCC and for clang.
Golden gate
The pytest command of
make test-netflix-golden, 283 tests collected, on the final head (38e8ec0b0plus this change), and before the rebase on513d2a6fcplus this change with the same counts:No assertion is edited and none moves.
test_ciede_device_mathThe host replay of the CUDA kernel's arithmetic against the CPU extractor, bit for bit. It passes under GCC and under clang, on x86-64 and on aarch64 (qemu), and in the CUDA build. Both sides now write the squares as products, so the result no longer depends on which side a compiler folds. I could not reproduce the hosted
Ubuntu clangfailure with clang 22.1.8 on master here; whatever folds differently there, this PR removes the call it could fold. The test is now registered on every architecture (it was x86-64 only whileciede.cwas contraction-free only there; ADR-1461 changed that).The three GPU twins
ciede_syclandciede_hipalready wrotedegrees * degrees(feature/ciede_ff_math.h).ciede_cudarounded the fp64pow()tofloat(CIEDE_POWF), which is not the product on every tie;ciede_device.hmultiplies now. SYCL and HIP sources change in comments only:ciede_ff_math.handciede_score.hiphave the same comment-stripped sha256 before and after (2b153850bf86ac58,8cdb58288dd219b7).Against the GCC build's
--backend cpu, same 180 frames, each twin on its device under its lock:ciede_cudaciede_syclciede_hipciede_cudaitself moves on 3 of 180 frames (BBB), by at most 1.1e-13.powf(x, 7). Against a GCC build with that one call correctly rounded (an experiment, not in this PR): CUDA identical on 180 of 180, SYCL on 173, HIP on 176; the rest is the precision of an fp32 pair, at most 8.4e-13.LIBM_TWINS["ciede"]stays1e-9: the bound is the size of one straddling pixel on the smallest gated frame (ADR-1426), and this change removes pixels, not their size.cross_backend_parity_gate.py, CPU of the same build): 0 on the Netflix pair and both checkerboards for all three; BBB 1920x1080 5.2e-12 (CUDA, HIP) and 7.2e-12 (SYCL, against its icx CPU).test_cuda_ciede_parity9 of 9,test_hip_ciede_parity11 of 11,test_sycl_ciede_parity11 of 11,test_sycl_ciede_math4 of 4.Fork snapshots
No file under
testdata/storesciede(the score snapshots hold the features ofvmaf_v0.6.1). Nothing to regenerate.Other integer powers in
core/srcpowf(vif_sigma_nsq, 2.0f):vif_tools.c:309, and its copies inx86/vif_statistic_avx2.c,cuda/float_vif_cuda.c,hip/float_vif_hip.c,sycl/sycl_float_vif_math.hpowfvif_sigma_nsq=1.000244140625(a tie, where glibc and the product differ) 384 of 384 valuespow(spatial_frequency / 7, 2):barten_csf_tools.h:147powadm_csf_mode=1: 1344 of 1344 values (adm,float_adm)pow(scores[i] - mean, 2):predict.c:561(bootstrap standard deviation)powvmaf_b_v0.6.3: 3744 of 3744 valuespow(x, 3)(integer ADM),pow(2, n),pow(10.0, x)ldexp, callMeasured on the Netflix 8-bit pair and BBB 1920x1080, 5472 values, none differs. glibc's
pow(x, 2)differs from the product on 41 676 of 50 million generaldoublearguments, so the first three are compiler-dependent in principle; no score difference was found, so no row is opened and nothing is changed for them here.Also in this PR: the icx model-score move of ADR-1461 is written down
Requested by the coordinator after the SYCL lane measured it. It is a record, not a code change.
vmaf_strict_fp_argsproject-wide (fix(build): build every C and C++ file without FP contraction so aarch64 clang and GCC builds agree (ADR-1461) #1829), an icx build compilessvm.cpp,predict.c,model.candlibvmaf.cwith-fp-model=precise -ffp-contract=off; before, with-O3alone (icx's fast model).vmafof an icx build moved on every frame with a non-zero score (160 of 163), by at most 7.05e-12, towards GCC: frames with identical features but a differentvmafwent from 153 of 163 to 15 of 163. Netflix 8-bit frame 42: 83.13509129537665 before, 83.1350912953696 after (the GCC value). GCC did not move.libimf(an icx-built library with glibc'slibmpreloaded returns the GCC values on 107 of 107 frames).docs/state.md(closed rowT-ICX-PREDICT-FP-MODEL-SCORE-MOVE-2026-10-02),docs/development/build-flags.md("Floating-point contraction is off everywhere"),changelog.d/changed/icx-predicted-vmaf-moves-towards-gcc.md.testdata/: none would show it. The sixteenscores_cpu_*/scores_sycl_*reports that hold avmafand the pooled scores of the three benchmark files are stored at six decimals. The only values stored at more are timings and the BRISQUE and NIQE extractor scores, which are not model scores and are compared atplaces=4.sweep/cmp6.txt,cmp6_200.txt,model6.txtand the stored reports undersweep/model/andsweep/preload/(mastersdedae7035and8cf182a85).What changed
core/src/feature/ciede.c:degrees * degrees;square()for the 13pow(x, 2).core/src/feature/cuda/integer_ciede/ciede_device.h:ciede_r_sub_t()multiplies; comments.core/src/feature/ciede_ff_math.h,core/src/feature/hip/integer_ciede/ciede_score.hip: comments only (same comment-stripped hash before and after).core/test/meson.build:test_ciede_device_mathon every architecture.core/test/test_ciede_device_math.c: thepow(x, 2)library fact is no longer needed and is dropped; one message reworded.ciede_ff_math.hand that message say "pi" where they saidM_PI: the MSVC preflight stage (scripts/dev/preflight.sh --stage msvcism) reads comments and strings as uses of the macro.core/test/test_sycl_ciede_exact_contract.py,core/test/test_cuda_ciede_exact_contract.py: the pinned statements.scripts/ci/cross_backend_calibration.py: comment with the new measurements;LIBM_TWINSunchanged.docs/metrics/features.md(CIEDE2000),docs/development/build-flags.md,docs/development/cross-backend-gate.md,core/src/AGENTS.d/ciede-squares-are-products.md, two lines each incore/src/feature/cuda/AGENTS.mdandcore/src/feature/sycl/AGENTS.md,docs/rebase-notes.md,docs/state.md, two changelog fragments.Type
feat— new featurefix— bug fixperf— performance improvementrefactor— no behavior changedocs— documentation onlytest— test-onlybuild/ci— tooling / infraport— cherry-pick from upstream Netflix/vmafsycl/cuda/simd— backend-specificChecklist
make format && make lintis green locally. Not run as one target: the commit hooks (clang-format, black, ruff, markdownlint, the HISS audit with its touched-file rule, generated-docs and ADR-citation checks) are green.python3 scripts/ci/run_meson_test.py -- -C build --suite=fast(244 of 244 with GCC on x86-64, final head)./cross-backend-diffand the worst ULP is ≤ 2. Run as the three device comparisons above (CUDA, SYCL, HIP against the CPU): at most 5.2e-12..c/.cpp/.cu/.h/.hpp, it has the appropriate license header (seeCONTRIBUTING.md).!orBREAKING CHANGE:and the migration path is documented below. Not a breaking change.docs/adr/_index_fragments/<NNNN-slug>.mdand the slug is appended todocs/adr/_index_fragments/_order.txt.Bug-status hygiene (ADR-0165)
docs/state.mdupdated in this PR with a row in the appropriate section (Open / Recently closed / Confirmed not-affected / Deferred), ORno state delta: REASON.T-CIEDE-CLANG-POWF-BUILTIN-2026-10-02moves from Open to Recently closed.T-CUDA-CIEDE-LIBM-RESIDUAL-2026-10-01stays open with the new measurements (it names the SYCL twin too; there is no separate SYCL or HIP residual row, so all three twins' figures are in that row).T-ICX-PREDICT-FP-MODEL-SCORE-MOVE-2026-10-02is new, under Recently closed.Netflix golden-data gate (ADR-0024)
assertAlmostEqual(...)score in the Netflix golden Python tests.Cross-backend numerical results
Deep-dive deliverables (ADR-0108)
docs/state.mdrows.docs/adr/1467-ciede-squares-as-products.md, the two-option table in "Context" and "Alternatives considered".AGENTS.mdinvariant note —core/src/AGENTS.d/ciede-squares-are-products.md(indexed incore/src/AGENTS.md);core/src/feature/cuda/AGENTS.md,core/src/feature/sycl/AGENTS.md.changelog.d/changed/ciede-squares-as-products.mdandchangelog.d/changed/icx-predicted-vmaf-moves-towards-gcc.md.docs/rebase-notes.md, "ADR-1467 —ciede.cwrites its squares as products (2026-10-02)".Reproducer
Known follow-ups
powf(c_bar_prime, 7)is the last float power of the formula and the whole remaining CPU-versus-twin difference. No compiler folds it, so builds agree on it; making it correctly rounded would move every build again. Recorded inT-CUDA-CIEDE-LIBM-RESIDUAL-2026-10-01.powfdoes with a tie.powfand 13powcalls fewer per pixel and takes about a third less time inciede: 563 against 837 ms per 1920x1080 frame (user CPU time, minimum of five interleaved runs on one core of a busy host); clang 517 against 509, unchanged.Breaking changes / migration
None. No public API, CLI or FFmpeg patch surface changes.
ciede2000from an existing GCC build differs from a new one by up to 2.0e-11 on some frames;ciede_cudaoutputs by up to 1.1e-13.