Skip to content

fix(ci): update parity gate motion metric keys to match default emissions - #1676

Merged
lusoris merged 1 commit into
masterfrom
fix/ci-parity-gate-stale-keys
Oct 1, 2026
Merged

lusoris merged 1 commit into
masterfrom
fix/ci-parity-gate-stale-keys

Conversation

@lusoris

@lusoris lusoris commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

scripts/ci/cross_backend_parity_gate.py and cross_backend_vif_diff.py mapped FEATURE_METRICS["motion"] to ("integer_motion", "integer_motion2", "integer_motion3"). Because neither CPU nor CUDA emits integer_motion in default (debug=false) mode, diff_frames() raised KeyError: 'integer_motion', causing a default all-features parity matrix run to abort immediately on its motion cell.

This PR updates FEATURE_METRICS["motion"] to ("integer_motion2", "integer_motion3") matching default emissions. It also updates scripts/ci/test_cross_backend_feature_names.py to test active backends (cuda, sycl, hip) instead of the removed vulkan backend (resolving 3/3 test failures), adds test_motion_cells_read_emitted_keys in core/test/test_parity_gate_metric_names.py, adds unit tests in scripts/ci/test_cross_backend_parity_gate.py, and adds core/test/test_cuda_parity_gate_default_run.py (registered in core/test/meson.build).

Verified on RTX 4090: default all-features run for CPU vs CUDA completes with all 18 features OK in ~8 seconds.

Type

  • feat — new feature
  • fix — bug fix
  • perf — performance improvement
  • refactor — no behavior change
  • docs — documentation only
  • test — test-only
  • build / ci — tooling / infra
  • port — cherry-pick from upstream Netflix/vmaf
  • sycl / cuda / simd — backend-specific

Checklist

  • Commits follow Conventional Commits (the commit-msg hook enforces this).
  • make format && make lint is green locally.
  • Unit tests pass: pytest scripts/ci/test_cross_backend_feature_names.py scripts/ci/test_cross_backend_parity_gate.py passes 71/71, test_parity_gate_metric_names passes, and test_cuda_parity_gate_default_run passes.
  • If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is <= 2. Parity gate tooling only.
  • If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap under "Known follow-ups" below.
  • If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header (see CONTRIBUTING.md).
  • If this is a breaking change, the commit message uses ! or BREAKING CHANGE: and the migration path is documented below.
  • If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/<NNNN-slug>.md and the slug is appended to docs/adr/_index_fragments/_order.txt.

Bug-status hygiene (ADR-0165)

  • docs/state.md updated in this PR: T-CI-PARITY-GATE-STALE-METRIC-KEYS-2026-09-29 moved to Recently closed.

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
  • If I believe a golden value must change, I have explained why below AND pinged @lusoris for a CODEOWNERS exception.

Cross-backend numerical results

Tooling fix for cross_backend_parity_gate.py and cross_backend_vif_diff.py.
Default 18-feature matrix run completes cleanly CPU vs CUDA on RTX 4090:

  • adm: OK (1.000e-06)
  • cambi: OK (0.000e+00)
  • ciede: OK (1.200e-05)
  • float_adm: OK (2.000e-06)
  • float_moment: OK (0.000e+00)
  • float_motion: OK (3.000e-06)
  • float_ms_ssim: OK (1.000e-06)
  • float_ms_ssim_lcs: OK (1.000e-06)
  • float_psnr: OK (0.000e+00)
  • float_ssim: OK (1.000e-06)
  • float_ssim_lcs: OK (1.000e-06)
  • float_vif: OK (2.800e-05)
  • motion: OK (0.000e+00)
  • motion_v2: OK (0.000e+00)
  • psnr: OK (0.000e+00)
  • psnr_hvs: OK (8.300e-05)
  • ssimulacra2: OK (0.000e+00)
  • vif: OK (0.000e+00)

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: metric key correction in test/parity tooling.
  • Decision matrix — no alternatives: aligns gate with actual default emitted metric keys.
  • AGENTS.md invariant note — no AGENTS.md invariant needed: test and tooling change only.
  • Reproducer / smoke-test command — python3 scripts/ci/cross_backend_parity_gate.py --vmaf-binary build/tools/vmaf --reference python/test/resource/yuv/src01_hrc00_576x324.yuv --distorted python/test/resource/yuv/src01_hrc01_576x324.yuv --width 576 --height 324 --backends cpu cuda
  • CHANGELOG fragment — changelog.d/fixed/ci-parity-gate-motion-keys.md.
  • Rebase note — entry added to docs/rebase-notes.md.

@lusoris
lusoris force-pushed the fix/ci-parity-gate-stale-keys branch from 501627a to 680b283 Compare October 1, 2026 10:49
…ions

Update FEATURE_METRICS["motion"] in scripts/ci/cross_backend_parity_gate.py
and scripts/ci/cross_backend_vif_diff.py to ("integer_motion2",
"integer_motion3"). The previous mapping included "integer_motion", which
the CLI only writes in debug mode; consequently, the default 16-feature
parity gate run halted on the motion cell with KeyError: 'integer_motion'.

Update scripts/ci/test_cross_backend_feature_names.py to test active backends
(cuda, sycl, hip) instead of the removed vulkan backend, resolving 3 test
failures. Add test_motion_cells_read_emitted_keys to
core/test/test_parity_gate_metric_names.py, unit tests in
scripts/ci/test_cross_backend_parity_gate.py, and e2e validation in
core/test/test_cuda_parity_gate_default_run.py (registered in
core/test/meson.build).

Verified on RTX 4090: default all-features parity matrix for CPU vs CUDA
runs to completion with all 18 features OK.
@lusoris
lusoris force-pushed the fix/ci-parity-gate-stale-keys branch from 680b283 to 5075f76 Compare October 1, 2026 11:01
@lusoris
lusoris merged commit 27fd5a6 into master Oct 1, 2026
63 of 69 checks passed
@lusoris
lusoris deleted the fix/ci-parity-gate-stale-keys branch October 1, 2026 11:01
@github-actions github-actions Bot added the type:bug Something isn't working label Oct 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant