Repository navigation
Conversation
…ADR-1231) Container base images were pinned by hand at each point of use and drifted. `Dockerfile.controller` and `Dockerfile.operator` still built on Debian 12 and shipped on distroless-debian12 after the rest of the tree moved to Debian 13, and both quoted a golang digest in their header comments that did not match their own FROM line. Two CUDA pins sat on Ubuntu 24.04 beside a sibling stage on 26.04. Four further pins were invisible to any FROM scan because they lived in `COPY --from=<image>` -- the Go toolchain in dev/Containerfile and the CUDA, ROCm and oneAPI runtime libraries in Dockerfile.node. Those were the most stale images in the repository, which is the argument both for gating `COPY --from` and for converting them to named stages. build-config.env is now the single source of truth. No Dockerfile names a base directly: each takes it as an ARG whose default mirrors the config, so a plain `docker build` still works with no wrapper and CI can override any base with --build-arg. `scripts/ci/check-base-image-single-source.sh` fails on drift, on a pin that is not digest-pinned, and on a version knob that disagrees with its pin; `make base-images-sync` rewrites the mirrors. Same authority-plus-drift- check shape as check-default-model-single-source.sh. Removes four Debian 12 pins and two Ubuntu 24.04 CUDA pins. `vmafx-controller` gains an explicit `USER 65532:65532`: its old cc-debian12 pin lacked the `:nonroot` suffix, so it ran as root. Both its listen ports are above 1024. ROCm and oneAPI keep Ubuntu 24.04 under explicit, self-closing exemptions. Each is a major SDK migration needing matching source changes, not a pin swap: ROCm 7.2.4 -> 10.0.0 is PR #1386, and oneAPI 2025 -> 2026.1 is blocked on a libsycl soname bump (.so.8 -> .so.9), a glibc gap between Intel's Ubuntu images and Debian 13, and the NEO GPU driver that Intel's runtime image carries and Debian does not package. All measured; see the research digest. Also fixes a latent bug in dev/Containerfile: `ENV PYTHONPATH=/build/vmaf/python:${PYTHONPATH}` expanded to a trailing colon because PYTHONPATH was never set, putting the current directory on sys.path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ovate The first commit unified container base images. That was half the problem: the same versions live in CI workflows, and they had drifted there independently. ROCm was 7.2.4 in build.yml and 7.2.3 in libvmaf-build-matrix.yml, so CI validated a ROCm the published images never shipped. The Level Zero loader existed at FOUR versions simultaneously -- v1.18.5, v1.28.0 twice, v1.29.0 and 1.32.0 -- and a skew there does not fail a build. It surfaces at runtime as "No device of requested type available", which reads like a driver or permissions problem and is not. build-config.env now carries the toolchain versions too (ROCM_APT_VERSION, LEVEL_ZERO_VERSION, CUDA_APT_PACKAGE, PYTHON_CI_VERSION). Workflow `run:` steps source it directly; scripts/ci/load-build-config.sh exports every knob into $GITHUB_ENV for steps that need a value earlier than that. The Windows SYCL leg runs under cmd and cannot source it, so it mirrors the value by hand and the gate keeps that mirror honest. scripts/ci/check-workflow-versions.py enforces it. It is a separate script because the Level Zero clone puts `--branch vX.Y.Z` and the repository URL on different lines, so a line-oriented grep silently sees only the first line -- the first version of this check passed against a deliberately planted literal. Renovate is reconfigured to match. Its `dockerfile` manager understands `ARG X=image` + `FROM $X`, so left alone it would bump the Dockerfile mirrors and leave build-config.env behind -- failing this change's own gate on Renovate's PRs. One custom manager now matches both the config line and the ARG form across all nine wired files (41 pins), so a bump lands every copy in one PR, and a packageRule disables the built-in manager on exactly those files. docker/dev/*.Dockerfile deliberately keeps the built-in manager. The gate also classifies config keys by the SHAPE of their value rather than by a list of names, so adding a version knob cannot accidentally subject it to the digest-pinning rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ADR-1225 landed ROCm 10.0.0 on the ubuntu-24.04 variant, which left the tree with the last release images on a distro everything else had moved off. AMD publishes rocm/dev-ubuntu-26.04:10.0.0-full too, but this is not a pin swap to make casually: Dockerfile.node and Dockerfile.production-gpu COPY these libraries out of the vendor image into a Debian 13 runtime, so a libc mismatch does not fail the build -- it fails the load, on the user's machine, with a "cannot open shared object file" that points at the wrong thing. Measured before moving: /opt/rocm/core-10.0/lib layout identical in both variants max GLIBC symbol in the closure 2.28 in both (AMD builds to an old floor) Debian 13 runtime provides 2.41 So the 26.04 variant is a drop-in. This is the OPPOSITE of the oneAPI case, where Intel's Ubuntu 26.04 image requires glibc 2.43 and genuinely cannot be copied onto Debian 13 -- the question has to be asked per vendor rather than answered once for all of them. With ROCm current, the gate's Ubuntu-24.04 exemption list is down to the two oneAPI entries, which disappear with the 2026.1 migration. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
build-config.env was added to stop version drift, but the oneAPI knob was
deliberately left out with a comment saying the 2026.1 migration was "the
immediate follow-up to this file". That follow-up never happened, so three
workflows kept spelling `intel-oneapi-compiler-dpcpp-cpp-2025.3` inline and the
CI toolchain sat two majors behind the compiler installed on the workstation.
All three already sourced build-config.env for LEVEL_ZERO_VERSION, so the
plumbing existed; only the version was missing from it. They now read
${ONEAPI_APT_PACKAGE}, and ONEAPI_VERSION is 2026.1.
The config also gains ONEAPI_RUNTIME_APT_PACKAGES, which names
intel-oneapi-umf explicitly: intel-oneapi-runtime-dpcpp-cpp does not depend on
it, so libumf.so.1 goes missing and the adapter fails to load -- surfacing as
"No device of requested type available" rather than as a link error. That was
measured while preparing the migration, and is the kind of fact the single
source exists to keep.
The Windows leg stays at 2025.3.0.372 on purpose. Its offline-installer URL
carries an opaque per-build GUID that cannot be derived from a version number,
so bumping it needs a looked-up URL rather than a string substitution; it is
recorded as ONEAPI_WINDOWS_VERSION so the lag is visible in the same file
instead of buried in a workflow, and tracked as
T-ONEAPI-WINDOWS-CI-LAG-2026-09-07.
Verified: load-build-config.sh resolves ONEAPI_APT_PACKAGE to
intel-oneapi-compiler-dpcpp-cpp-2026.1, no literal 2025.3 oneAPI package name
remains in .github/workflows/, and both check-workflow-versions.py and
check-base-image-single-source.sh pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…m-image Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ffmpeg): replay percentile guard after the existing mappings * docs(adr): define stable-release FFmpeg patch maintenance * fix(ffmpeg): automate stable-release patch refresh and required replay * fix(go): confine predictor cards and require Go validation * docs(adr): preserve work during agent-state cleanup * fix(dev): preserve work during agent-state cleanup * fix(ci): recognize root build config in dependency PRs * fix(hooks): preserve dispatch across worktree removal * fix(ci): reject unpinned container image bypasses * fix(docs): refresh ADR metadata and enforce source freshness * fix(ci): align shared config consumers and combined documentation * fix(deps): restore Renovate custom manager file matching * fix(docs): require push toolchain and repair guide links * test(ffmpeg): type patch receipts and safety fixtures for push checks * test(ci): type impact and Go workflow contracts * docs: repair topic links and identify unavailable evidence * fix(build): consume the shared Level Zero version * docs: refresh FFmpeg guide and correct PSNR evidence * fix(dev): make container shell failures explicit * fix(hooks): retain documentation checks on large diffs --------- Co-authored-by: Lusoris <lusoris@pm.me>
This was referenced Sep 8, 2026
Contributor
Author
|
Superseded by the pre-rc.1 fixing train in #1425. Verified by comparing this branch's tree against the train for every path its commits touch: zero differences. Its own commits are on the train already, and everything else it carried was the base-image chain that #1396's squash put on Closing is safe; nothing here is lost. If you would rather keep it open until #1425 merges, that works too — it just has nothing left to contribute. |
Contributor
Author
|
Superseded by the pre-rc.1 fixing train in #1425. Closing because the ADR Collision Guard is a required check on #1425 and fails while two open PRs claim the same ADR numbers. Verified before closing:
The branch is untouched, so this is reversible. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Windows CUDA configuration failed with
Unknown variable cl_pathwhenvswherewas unavailable or returned no compiler, even ifcl.exewas onPATH. Assign the fallback path once and use it for both NVCC and MSVC header discovery. A Meson configure regression exercises the shipped discovery block and runs in the POSIXfastsuite.Also correct native-versus-Python ONNX Runtime ownership documentation: the native CPU archive does not provide CUDA or ROCm execution providers. Runtime pins are unchanged.
This is a held follow-up to #1396. The preceding hygiene PR #1420 was merged into that parent by the old local train, not into master. The parent and this follow-up remain drafts pending full integration acceptance; stacked PRs must not be auto-promoted or merged by the train.
Validation
--suite=fastregistration executes the fixture successfully (1/1 registered test, four internal cases).make lint,make test, fresh image, golden-data and hosted acceptance remain pending. No merge-readiness claim is made from focused checks.Bug-status hygiene
docs/state.mdrecords the Windows fallback bug and the ORT ownership correction.Deep-dive deliverables
AGENTS.mdinvariant note —core/src/AGENTS.mdrecords the shared compiler-path contract.changelog.d/fixed/windows-cuda-cl-path-fallback.mdandchangelog.d/fixed/ort-runtime-owner-comments.md.docs/rebase-notes.mdrecords both contracts.Reproducer
Native compilation and hardware scoring remain separate validation steps.
Verified exact-source validation update — 2026-09-08
This supersedes the earlier statement that
make testremained pending. Current PR headcc022245079f1b7da4005a39bcefc759e2d6ef3d(treec2d83fb0798f4989874ef550e60de31f69a832a8) passed the configured CPUmake testcommand: exit 0, 134/134 Meson tests passed, with ASM and default LTO enabled. The source was mounted read-only in an isolated offline container; the installed application binary was not used.Full
make -k lintexited 2 because clang-tidy, black, shellcheck, npx, and gosec were missing. cppcheck was also absent, but lint-c stopped before reaching it. Ruff's configured scope and generated-document consistency checks passed; that does not make full lint green. Exact commands, logs, configuration, hashes, and limitations are retained in.workingdir2/evidence/2026-09-08-rc1-full-gates-cc022245/README.md. The commands select verified Meson/Ninja paths and suppress only their offline bootstrap attempts; source compilation, tests, and analyzer failures are preserved.The Makefile's separate Python golden-data and sanitizer targets were not run. CUDA, SYCL, HIP, Metal, DNN, MCP, and docs generation were disabled. This GCC 15 CPU result does not establish backend, native Windows/macOS, canonical GCC 14 CI, or RC1 acceptance. Full lint and the remaining integration gates are still required; this PR remains draft and held.
Ancestry is verified: cc02224 → a49d54f → current draft parent #1396 at
851d53dc8112e103e5f55544c09b3dc78f02f7da. Thus the CPU receipt applies to this current combined source without a new rebase. The original #1420 merge event produced834f84063a4640e72b0916a9a5b3a566b6810fecon that parent branch only; the legacy train subsequently rebased it onto master78c9d2bfat 18:17:41 +0200, before suspension. The hygiene stack was not merged into master.Related held work: #1422 contains the queue/backpressure repair; #1423 contains guarded train control and its paused migration helper. #1416 configuration convergence remained pending push at this update (remote head
d71e4336). Runtime installation belongs to the coordinating operator; this documentation update changes no holds, draft states, or processes.Canonical CPU golden follow-up — 2026-09-08, exact cc02224
The unchanged original
QualityRunnerTest::test_run_vmaf_runnerandQualityRunnerTest::test_run_vmaf_runner_checkerboardnodes both passed(2/2). Together they cover all three canonical pairs: normal video and
both checkerboard shifts, plus self-reference and feature-score checks.
No original assertion or tolerance changed. Only executable/resource
paths were bound to the exact retained CPU binary and input data; the same
cc02224 source was mounted read-only with a fresh output workspace.
The bounded CPU container run used NumPy distribution/module 2.5.3 and
the hash-verified official PyWavelets 1.10.0 distribution, installed only
in a private read-only overlay. The PyWavelets wheel's packaged module
reports 1.8.0, despite its distribution METADATA reporting 1.10.0. The
two runtime floors are therefore resolved at distribution metadata level;
PyWavelets runtime version identity remains inconsistent. This limitation
is recorded without changing the package, source or golden assertions.
The central local receipt
is retained on the workstation at
.workingdir2/evidence/2026-09-08-rc1-canonical-golden-cc022245/floor-overlay/README.md(not a hosted GitHub artifact). It includes official package metadata,
wheel/overlay/source/binary/data hashes, exact command and versions, the
initial version-identity preflight refusal, final passing JUnit/logs, and
unchanged-input verification. The initial passing run with existing
dependencies is retained separately in the parent evidence directory.
This supersedes only the earlier pending canonical three-pair CPU golden
status. The complete five-module
make test-netflix-goldentarget, full lint,sanitisers, other backends/platforms and hosted integration acceptance remain
separate and unverified by this run. This PR remains draft and held; no RC1
readiness or merge approval is claimed.