Repository navigation
test(metal): add CPU-vs-Metal parity tests for remaining 4 kernels (round 3) - #447
Merged
Merged
Conversation
Round 3 of Metal kernel test coverage. PR #351 closed the registration audit (all 8 extractors discoverable) and PR #379 added parity tests for the 4 highest-priority kernels (motion_v2, integer_psnr, float_psnr, float_ssim). This commit fills the remaining 4 gaps so every registered Metal extractor now has a real per-kernel CPU-vs-Metal score gate. New tests (all in core/test/, all skip on -ENODEV): - test_metal_integer_motion_parity.c -> motion2_score (1e-4, ADR-0214) - test_metal_float_motion_parity.c -> motion2_score (1e-4, ADR-0214) - test_metal_float_moment_parity.c -> 4 moment keys (1e-4, ADR-0214) - test_metal_float_ms_ssim_parity.c -> ms_ssim (1e-3, ADR-0589) Pattern mirrors PR #379 / test_sycl_motion3_parity.c: synthetic 256x144 YUV420P fixture fed through both the CPU twin and the Metal extractor; assert places=4 (1e-4) parity per ADR-0214, except float_ms_ssim which inherits the 1e-3 SSIM-family bound from ADR-0589. Skip path: when vmaf_metal_state_init returns -ENODEV (Linux, Windows, Intel Mac), each test emits "[skip: no Metal device]" and passes cleanly. Runs the live kernel only on Apple-Family-7+ macOS CI lanes. Wiring: 4 new executable() + test() entries appended inside the existing enable_metal guard in core/test/meson.build (after the round-2 block, or after test_metal_install_header pre-merge), suite ['fast', 'gpu'] for consistency with the existing Metal smoke / round-2 parity tests. ADR-0108 deliverables: - Research digest: no digest needed: trivial test-only addition - Decision matrix: no alternatives: only-one-way fix (synthetic-fixture + skip-on-ENODEV is the established test_sycl_motion3_parity.c + PR #379 precedent) - AGENTS.md invariant note: no rebase-sensitive invariants in this PR; PR #379 already updates core/test/AGENTS.md for the Metal -ENODEV skip rule - Reproducer: see PR description - CHANGELOG fragment: changelog.d/added/metal-kernel-coverage-round3.md - Rebase note: docs/rebase-notes.md "Metal kernel parity tests round 3 (2026-05-31)" State.md updated with T-METAL-KERNEL-PARITY-ROUND3-2026-05-31 row under Recently closed (CLAUDE r13). Refs: PR #294, PR #308, PR #351 (registration audit), PR #379 (round 2) Cross-refs: ADR-0214, ADR-0361, ADR-0421, ADR-0589 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
Author
|
parity sweep audit: clean, no symmetric bug. Verified all 4 new test files use the correct Metal-suffixed registry names:
CPU baselines (motion, float_motion, float_moment, float_ms_ssim) all resolve correctly in the CPU registry. The integer_motion_metal test correctly pairs against CPU 'motion' (integer path in core/src/feature/integer_motion.c). No fix needed. |
This was referenced May 31, 2026
lusoris
marked this pull request as ready for review
May 31, 2026 04:22
lusoris
enabled auto-merge (squash)
May 31, 2026 04:22
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Round 3 of Metal kernel test coverage. With this PR all 8 registered Metal extractors have a real per-kernel CPU-vs-Metal score parity gate:
integer_motion,float_motion,float_moment,float_ms_ssim.New tests (all
core/test/, allsuite : ['fast', 'gpu']):integer_motion_metalmotionVMAF_integer_feature_motion2_scorefloat_motion_metalfloat_motionVMAF_feature_motion2_scorefloat_moment_metalfloat_momentref1st,dis1st,ref2nd,dis2nd)float_ms_ssim_metalfloat_ms_ssimfloat_ms_ssimSkip path. Each test calls
vmaf_metal_state_init. On Linux / Windows / Intel Mac that returns-ENODEV; the test emits[skip: no Metal device]and passes cleanly. Apple-Family-7+ macOS CI lanes run the live kernels and assert the per-kernel tolerance. Pattern mirrors PR #379 andcore/test/test_sycl_motion3_parity.c.Wiring. 4 new
executable()+test()entries appended inside the existingenable_metalguard incore/test/meson.build, immediately after the round-2 block (or aftertest_metal_install_headerpre-PR-#379-merge) — no boundary edits, so upstream syncs cannot conflict here.PR overlap (avoided)
-ENOSYScontracts; this PR exercises the real runtime.vmaf_get_feature_extractor_by_namediscoverability; this PR runs the actual extractor pipeline for 4 more kernels.test_metal_install_header.Per-kernel coverage count (post-merge)
Test plan
cd core && meson setup build-cpu -Denable_cuda=false -Denable_sycl=false -Denable_metal=disabledsucceeds (new tests gated off on non-Metal).pre-commit run --fileson every touched file -> all hooks pass (clang-format, copyright header, semgrep, conventional-commit, etc.).bash scripts/ci/check-copyright.sh core/test/test_metal_*_parity.c-> silent (clean).bash scripts/ci/assertion-density.sh core/test/test_metal_*_parity.c-> PASS (every fork-added function ≥20 lines has ≥1 assert).clang-format --dry-run --Werror core/test/test_metal_*_parity.c-> no diff.meson test -C build --suite=fast test_metal_integer_motion_parity test_metal_float_motion_parity test_metal_float_moment_parity test_metal_float_ms_ssim_parityexercises the live kernels and asserts parity. (Will run automatically when the macOS Metal CI lane picks up this PR — Apple-Family-7+ device required.)ADR-0108 deliverables
test_sycl_motion3_parity.c+ PR test(metal): add CPU-vs-Metal parity tests for 4 kernels #379 precedent; tolerance choices inherit from existing ADR-0214 / ADR-0589).core/test/AGENTS.md"GPU tests must skip gracefully" rule to mentionvmaf_metal_state_initand Intel-Mac-ENODEV; this PR consumes that contract without re-stating it.changelog.d/added/metal-kernel-coverage-round3.md.docs/rebase-notes.md-> "Metal kernel parity tests round 3 (2026-05-31)" entry added; rebase impact NONE (all new file paths are fork-local Metal-only, the meson edit lives inside the fork-addedenable_metalblock).docs/state.mdupdated with theT-METAL-KERNEL-PARITY-ROUND3-2026-05-31row under "Recently closed" (per CLAUDE §12 r13).Cross-references
🤖 Generated with Claude Code