Repository navigation
chore(ci): infra/CI bundle batch-1 (#370 coverage override + #396 envtest + #446 SYCL parity round 3) - #528
Merged
Merged
Conversation
…_template.h 10 → 75 Periodic audit of `scripts/ci/coverage-check.sh`'s `PER_FILE_MIN` map (ADR-0114). Three findings: 1. `core/src/dnn/tiny_extractor_template.h` — actual 77.4 %, override 10 % (67.4 pp slack). Original 10 % cap was set when only one extractor (`feature_lpips.c`) instantiated the inline helpers; four extractors now do (`feature_lpips`, `fastdvdnet_pre`, `feature_mobilesal`, `feature_transnet_v2`). Tighten to 75 (2.4 pp slack, mirroring the 1.3-1.7 pp slack ADR-0114 used for the at-cap entries). Locks 65 pp of de-facto regression-coverage. 2. `core/src/dnn/ort_backend.c` — actual 77.8 %, override 78 % (at cap, currently failing on master). Keep at 78 — PR #338 in flight adds `vmaf_ort_output_name_at` unit test, lifting actual to 78.5 % without raising the bar (correct pattern per ADR-0114). 3. `core/src/dnn/dnn_api.c` — actual 78.0 %, override 78 % (at cap). Keep at 78 — structural ceiling rationale per ADR-0114 §Context unchanged. No new override entries required. All other dnn/ files plus opt.c and read_json_model.c clear the global 85 % critical floor (dnn_attach_api.c 92 %, model_loader.c 87 %, onnx_scan.c 93 %, op_allowlist.c 100 %, tensor_io.c 98 %, opt.c 100 %, read_json_model.c 88 %). ADR-0881 also codifies the recurring audit rule (tighten when slack > 5 pp; keep at-cap; remove when actual ≥ global 85 % floor; audit quarterly + before any PER_FILE_MIN edit). Audit procedure documented in the companion research digest. Reproducer: PKG_CONFIG_PATH=/path/to/onnxruntime/lib/pkgconfig \ meson setup core/build-coverage core --buildtype=debug \ -Db_coverage=true -Denable_cuda=false -Denable_sycl=false \ -Denable_float=true -Denable_avx512=true -Denable_dnn=enabled \ -Dc_args=-fprofile-update=atomic -Dcpp_args=-fprofile-update=atomic ninja -C core/build-coverage LD_LIBRARY_PATH=/path/to/onnxruntime/lib \ meson test -C core/build-coverage --num-processes 1 cd core && gcovr --root .. --filter 'src/.*' \ --exclude '.*/test/.*' --exclude '.*/tests/.*' \ --json-summary build-coverage/coverage.json build-coverage bash scripts/ci/coverage-check.sh core/build-coverage/coverage.json 37 85 # Pre-audit: tiny_extractor_template.h reports "min 10%" # Post-audit: same file reports "min 75%" — gate enforces the new floor Closes the ADR-0114 implicit follow-up "re-audit slack overrides on a cadence so they don't rot". Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…uite The cmd/vmafx-operator/internal/controller envtest suite was hard-failing in BeforeSuite with a nil-pointer deref from controlplane.(*APIServer).Stop because the kubebuilder envtest control-plane binaries (etcd + kube-apiserver + kubectl) were not on PATH. PRs #330, #341, and #362 all called this out as a pre-existing failure they could not address inline. Three-pronged fix: 1. Makefile gains a `setup-envtest` target that installs `sigs.k8s.io/controller-runtime/tools/setup-envtest@latest` and downloads the v1.31 control-plane bundle. A companion `setup-envtest-env` target prints the eval-friendly export line so developers can do `eval $(make -s setup-envtest-env)`. 2. .github/workflows/go-ci.yml installs setup-envtest and exports `KUBEBUILDER_ASSETS` via `$GITHUB_ENV` before `go test ./...`, so the operator suite runs for real in CI instead of skipping. 3. cmd/vmafx-operator/internal/controller/suite_test.go gains a top-of-`TestControllers` `t.Skip()` guard when `KUBEBUILDER_ASSETS` is unset, plus a nil-`testEnv` bailout in `AfterSuite` so the suite never panics on a fresh checkout where the assets are missing (defense in depth). Local verification: - `unset KUBEBUILDER_ASSETS && go test ./cmd/vmafx-operator/...` -> SKIP with an actionable message pointing at `make setup-envtest`. - `eval $(make -s setup-envtest-env) && go test -v ./cmd/vmafx-operator/...` -> 3/3 specs pass in ~5 s. AGENTS.md gains a new invariant #6 documenting the skip-safe envtest pattern; rebase-notes.md, state.md, and a changelog fragment under changelog.d/fixed/ are updated per ADR-0108 + CLAUDE.md r10/r13. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds five new CPU-vs-SYCL parity tests under core/test/, extending the SYCL parity coverage beyond rounds 1 (PR #351: integer psnr + vif) and 2 (PR #376: integer adm / ciede / ssim / ms_ssim / motion_v2). New parity tests at ADR-0214 places=4 (1e-4) tolerance: | Kernel | New test | Headline score | |---------------------|---------------------------------------|-------------------------------------------| | float_psnr_sycl | test_sycl_float_psnr_parity.c | float_psnr | | float_adm_sycl | test_sycl_float_adm_parity.c | VMAF_feature_adm2_score | | float_vif_sycl | test_sycl_float_vif_parity.c | VMAF_feature_vif_scale0_score | | float_motion_sycl | test_sycl_float_motion_parity.c | VMAF_feature_motion2_score (idx 1) | | psnr_hvs_sycl | test_sycl_psnr_hvs_parity.c | psnr_hvs | Each mirrors the round-1 / round-2 pattern: 256x144 synthetic YUV420P fixture, CPU + SYCL feature extractor, parity assertion within ADR-0214 places=4, skip-on-no-device via "[skip: no SYCL device]" printf. Coverage trajectory: 50% (rounds 1+2) -> 78% (this PR). Round-4 backlog: float_moment, speed_chroma, speed_temporal, ssimulacra2 (need scaffold extensions for per-extractor config dicts and ref_pic_90 fill before the same gate can be added). Container compile-check evidence: docker exec vmaf-dev-mcp bash -lc ' source /opt/intel/oneapi/setvars.sh --force >/dev/null 2>&1 && \ cd /tmp/wt-sycl-r3 && \ CC=icx CXX=icpx meson setup build-sycl-r3 core \ -Denable_sycl=true -Denable_avx512=false -Db_lto=false && \ ninja -C build-sycl-r3 \ test/test_sycl_float_psnr_parity \ test/test_sycl_float_adm_parity \ test/test_sycl_float_vif_parity \ test/test_sycl_float_motion_parity \ test/test_sycl_psnr_hvs_parity' All five executables compile + link clean with -Wall -Wextra under icx/icpx. Registration sub-tests pass on this host; parity sub-tests hit a pre-existing level_zero device-passthrough issue (same as PR #376, not introduced by this PR). Refs ADR-0946, ADR-0214, ADR-0868, ADR-0884.
…7) (#465) Closes the SYCL kernel-coverage backlog enumerated in ADR-0946 by adding four new CPU vs. SYCL parity tests under core/test/. | Kernel | New test | Tolerance | |---------------------|---------------------------------------|------------------------| | float_moment_sycl | test_sycl_float_moment_parity.c | 1e-4 (ADR-0214 default)| | speed_chroma_sycl | test_sycl_speed_chroma_parity.c | 1e-4 (ADR-0214 default)| | speed_temporal_sycl | test_sycl_speed_temporal_parity.c | 1e-4 (ADR-0214 default)| | ssimulacra2_sycl | test_sycl_ssimulacra2_parity.c | 5e-3 (FEATURE_TOLERANCE)| Each test mirrors the round-3 scaffold: 256x144 synthetic YUV420P fixture, public vmaf_use_feature API with NULL options dict (defaults match between CPU and SYCL for all four kernels), parity assertion via fabs(cpu - sycl) <= TOL, skip-on-no-device via "[skip: no SYCL device]" printf. The SSIMULACRA2 fixture fills all three planes (the pipeline consumes YUV -> linear-RGB -> XYB and chroma matters for the headline score). The speed_temporal fixture submits two frames (TEMPORAL flag means frame 0 emits 0.0) and asserts at index 1. Discovery during round-4 implementation: speed_chroma_sycl.cpp (752 LOC) and speed_temporal_sycl.cpp (705 LOC) source files exist on disk but are NOT wired into sycl_feature_sources in core/src/meson.build and their extractor symbols are NOT declared/registered in core/src/feature/feature_extractor.c. Both files appear complete (no TODO/FIXME/-ENOSYS/stub markers) but ship as dormant scaffold. Wiring them in is out of scope for a kernel-coverage PR (it changes the production extractor surface). The two SpEED parity tests are added in dormant form with a "[skip: <name> not built into libvmaf]" guard that auto-activates as a real parity gate the day a follow-up PR wires the TUs into the build + registry. Container compile-check evidence (CC=icx CXX=icpx, enable_sycl=true, Intel Arc A380 visible): - All four test executables link against libvmaf.a successfully. - speed_chroma/temporal: pass (skip — dormant scaffold). - float_moment / ssimulacra2: register-existence sub-test passes; parity sub-test hits the same pre-existing level_zero device-passthrough SIGSEGV that PR #446's round-3 tests hit on this dev container (not introduced by this PR). On a host with proper Intel-GPU passthrough or with no SYCL device, the tests pass / skip cleanly. Coverage trajectory: - Round 0 (pre-rounds): 2/18 (11%) - Round 1 (#351): +2 = 4/18 (22%) - Round 2 (#376): +5 = 9/18 (50%) - Round 3 (#446): +5 = 14/18 (78%) - Round 4 (this PR): +4 = 18/18 (100% of round-3 backlog; 16 active + 2 dormant SpEED scaffolds) Refs ADR-0957, ADR-0214, ADR-0867, ADR-0884, ADR-0946. Co-authored-by: Lusoris <lusoris@pm.me> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lusoris
force-pushed
the
chore/infra-ci-bundle-370-396-446
branch
from
June 2, 2026 14:35
443b04c to
715772c
Compare
lusoris
marked this pull request as ready for review
June 2, 2026 14:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bundles three DRAFT source PRs (#370, #396, #446) into a single PR per the 2026-06-01 triage directive. Each source PR is closed as superseded.
tiny_extractor_template.hper-file coverage override 10 → 75 (ADR-0881); codify quarterly audit rule; addcoverage-check.shrow toscripts/ci/AGENTS.md.cmd/vmafx-operator/internal/controller— installsetup-envtestingo-ci.yml, exportKUBEBUILDER_ASSETS, add skip-safe guard insuite_test.go.Deep-dive deliverables (ADR-0108)
docs/research/files; operator envtest is trivial CI plumbing)scripts/ci/AGENTS.mdgainscoverage-check.shrow (ADR-0881);cmd/vmafx-operator/AGENTS.mdgains invariant chore(deps): Update dependency openai to >=2.38.0 #7 (skip-safe envtest);core/src/feature/sycl/AGENTS.mdgains per-kernel parity-test invariant table (ADR-0946 + ADR-0957)bash scripts/ci/coverage-check.sh core/build-coverage/coverage.json 37 85; envtest:make setup-envtest && eval $(make -s setup-envtest-env) && go test -v ./cmd/vmafx-operator/internal/controller/...; SYCL parity:docker exec vmaf-dev-mcp meson test -C /workspace/build-sycl core/test/test_sycl_float_psnr_parity core/test/test_sycl_float_adm_parity core/test/test_sycl_float_vif_parity core/test/test_sycl_float_motion_parity core/test/test_sycl_psnr_hvs_parity core/test/test_sycl_float_moment_parity core/test/test_sycl_speed_chroma_parity core/test/test_sycl_speed_temporal_parity core/test/test_sycl_ssimulacra2_paritychangelog.d/changed/bundle-infra-ci-batch1.mdBug-status hygiene (ADR-0165)
Netflix golden-data gate
assertAlmostEqual(...)score touched.Applied source PRs
scripts/ci/coverage-check.sh(took 75, the ADR-0881 value which dominates ADR-0922's +5pp ratchet),docs/adr/_index_fragments/_order.txt,docs/rebase-notes.md,scripts/ci/AGENTS.md.github/workflows/go-ci.yml(union: gosec step + envtest step),cmd/vmafx-operator/AGENTS.md(union: both invariant 6 and 7),suite_test.go(union: bothlog/slogand"os"imports),docs/rebase-notes.md,docs/state.mdcore/test/meson.build(union: all test blocks from rounds 1-4),docs/adr/README.md,docs/adr/_index_fragments/_order.txt,docs/rebase-notes.mdDO NOT mark ready — active train head is #517.
🤖 Generated with Claude Code