Skip to content

chore(dev): add rc3-home-gpu-retest.sh and verification harness (ADR-1386) - #1644

Merged
lusoris merged 3 commits into
masterfrom
chore/rc3-home-gpu-retest
Oct 1, 2026
Merged

lusoris merged 3 commits into
masterfrom
chore/rc3-home-gpu-retest

Conversation

@lusoris

@lusoris lusoris commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Add scripts/dev/rc3-home-gpu-retest.sh and helper module scripts/dev/rc3_retest_helpers.py to automate verification and timing measurements for the RC3 handoff on ryzen-4090-arc across CUDA, HIP, and SYCL backends. It executes each open docs/state.md verify-and-time command explicitly by row ID without markdown parsing at runtime, enforces per-device flock serialization (cuda-4090.lock, hip-gfx1036.lock, sycl-a380.lock), logs per row, and produces TSV/markdown summaries with median-of-N frame timing subtraction.

Type

  • feat — new feature
  • fix — bug fix
  • perf — performance improvement
  • refactor — no behavior change
  • docs — documentation only
  • test — test-only
  • build / ci — tooling / infra
  • port — cherry-pick from upstream Netflix/vmaf
  • sycl / cuda / simd — backend-specific

Checklist

  • Commits follow Conventional Commits (the commit-msg hook enforces this).
  • make format && make lint is green locally.
  • Unit tests pass: python3 scripts/ci/run_meson_test.py -- -C build.
  • If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2.
  • If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap under "Known follow-ups" below.
  • If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header (see CONTRIBUTING.md).
  • If this is a breaking change, the commit message uses ! or BREAKING CHANGE: and the migration path is documented below.
  • If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/<NNNN-slug>.md and the slug is appended to docs/adr/_index_fragments/_order.txt — do not edit docs/adr/README.md directly (regenerated by scripts/docs/concat-adr-index.sh; see ADR-0221).

Bug-status hygiene (ADR-0165)

  • docs/state.md updated in this PR with a row in the appropriate section (Open / Recently closed / Confirmed not-affected / Deferred), OR no state delta: test tooling and harness scaffolding for RC3 verification.

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
  • If I believe a golden value must change, I have explained why below AND pinged @lusoris for a CODEOWNERS exception.

Cross-backend numerical results

Baseline verification runs on master (10f27efe2) on ryzen-4090-arc via scripts/dev/rc3-home-gpu-retest.sh:

CUDA Baseline (master 10f27ef)

Row ID Backend Result Details
T-CUDA-PSNR-HVS-HOST-ROUNDTRIP-2026-09-29 cuda FAIL 576x324 max 8.37e-05 (psnr_hvs_y); 3840x2160 max 0.011 (psnr_hvs_y) vs bound 3.34e-3
T-GPU-MOTION-V2-INT64-VERTICAL-2026-09-29 cuda PASS 576x324 max 0 (48/48 identical); 3840x2160 max 0 (22/22 identical)
T-CUDA-MOTION-BLUR-THEN-DIFF-2026-09-29 cuda FAIL 576x324 max 1.26e-05 (integer_motion2), 1/48 identical (waits for PR #1637)
T-CUDA-FLOAT-SSIM-SCALE-GT1-2026-09-29 cuda ERROR 3840x2160 float_ssim CPU fallback warning; exit 234
T-CUDA-SSIMULACRA2-HOST-COMBINE-2026-09-29 cuda PASS 576x324 max 0 (48/48 identical); 3840x2160 max 0 (50/50 identical)
T-CUDA-CAMBI-HOST-RESIDUAL-2026-09-29 cuda PASS 576x324 max 0; 3840x2160 max 0
T-CUDA-SPEED-HOST-RESIDUAL-2026-09-29 cuda ERROR speed_gpu_parity.py exit 234 (waits for PR #1639)

HIP Baseline (master 10f27ef)

Row ID Backend Result Details
T-HIP-PSNR-HVS-HOST-CONVERT-2026-09-29 hip FAIL 576x324 max 8.37e-05; 3840x2160 max 0.011 vs bound 3.34e-3
T-GPU-MOTION-V2-INT64-VERTICAL-2026-09-29 hip PASS 576x324 max 0 (48/48 identical); 3840x2160 max 0 (22/22 identical)
T-HIP-TWIN-PRIVATE-PLANE-UPLOADS-2026-09-29 hip PASS Recorded baseline JSONs; 576x324 hip 5.64 ms, 3840x2160 hip 270.76 ms
T-HIP-MOTION-BLUR-THEN-DIFF-2026-09-29 hip FAIL 576x324 max 1.26e-05 (integer_motion2), 1/48 identical (waits for PR #1636)
T-HIP-FLOAT-SSIM-SCALE-GT1-2026-09-29 hip ERROR 3840x2160 float_ssim CPU fallback warning; exit 234
T-HIP-SSIMULACRA2-HOST-COMBINE-2026-09-29 hip PASS 576x324 max 0 (48/48 identical); 3840x2160 max 0 (50/50 identical)
T-HIP-CAMBI-HOST-RESIDUAL-2026-09-29 hip PASS 576x324 max 0; 3840x2160 max 0
T-HIP-SPEED-HOST-RESIDUAL-2026-09-29 hip FAIL 576x324 max 3.05e-05 (speed_chroma_u); 3840x2160 max 0.000196 (speed_temporal)
T-GPU-ADM-AIM-DEVICE-PASS-MISSING-SYCL-HIP-2026-09-05 hip FAIL missing integer_adm3, integer_aim

SYCL Status

Performance (if perf or feat)

Not applicable (tooling PR).

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: dev tooling and verification harness.
  • Decision matrix — captured in the corresponding ADR's ## Alternatives considered (docs/adr/1386-rc3-home-gpu-retest-kit.md).
  • AGENTS.md invariant note — added to scripts/dev/AGENTS.md.
  • Reproducer / smoke-test command — pasted below under "Reproducer".
  • CHANGELOG fragment — a new file under changelog.d/added/rc3-home-gpu-retest-kit.md.
  • Rebase note — entry added to docs/rebase-notes.md under RC3-RETEST-KIT.

Reproducer

python3 -m pytest scripts/dev/tests/test_rc3_home_gpu_retest.py
bash scripts/dev/rc3-home-gpu-retest.sh --help
bash scripts/dev/rc3-home-gpu-retest.sh --list

Known follow-ups

None. The script will be used to measure PRs #1630, #1636, #1637, and #1639 against the above master baseline.

Breaking changes / migration

None.

@github-actions github-actions Bot added the type:chore Maintenance, no user-visible change label Sep 30, 2026
…1386)

Add scripts/dev/rc3-home-gpu-retest.sh to run verify-and-time commands
that docs/state.md carries for ryzen-4090-arc (RTX 4090 CUDA, Arc A380
SYCL, Zen 5 gfx1036 HIP).

- Explicitly encode each command per row and backend without parsing
  markdown at run time.
- Enforce per-device flock serialization (cuda-4090.lock,
  hip-gfx1036.lock, sycl-a380.lock) with clean lock acquisition and
  release.
- Add scripts/dev/rc3_retest_helpers.py for JSON metrics comparison,
  median-of-N frame timing subtraction, and markdown summary generation.
- Add test suite scripts/dev/tests/test_rc3_home_gpu_retest.py covering
  arguments, dry-run, device locking, and row ID consistency against
  docs/state.md.
- Document usage and baseline comparison workflow in
  docs/development/rc3-home-gpu-retest.md and ADR-1386.
- Register rc3-home-gpu-retest.sh in EXPECTED_RUNNER_PATHS in
  core/test/test_meson_secret_env_sanitization.py.
@lusoris
lusoris force-pushed the chore/rc3-home-gpu-retest branch from f711a33 to 00d2794 Compare October 1, 2026 00:36
@lusoris
lusoris merged commit 6047082 into master Oct 1, 2026
70 of 75 checks passed
@lusoris
lusoris deleted the chore/rc3-home-gpu-retest branch October 1, 2026 00:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:chore Maintenance, no user-visible change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant