Repository navigation
fix: repair RC1 hooks, patch maintenance and repository hygiene - #1420
Merged
lusoris merged 20 commits intoSep 8, 2026
Merged
Conversation
lusoris
marked this pull request as ready for review
September 8, 2026 16:16
lusoris
added a commit
that referenced
this pull request
Sep 8, 2026
* fix(ffmpeg): replay percentile guard after the existing mappings * docs(adr): define stable-release FFmpeg patch maintenance * fix(ffmpeg): automate stable-release patch refresh and required replay * fix(go): confine predictor cards and require Go validation * docs(adr): preserve work during agent-state cleanup * fix(dev): preserve work during agent-state cleanup * fix(ci): recognize root build config in dependency PRs * fix(hooks): preserve dispatch across worktree removal * fix(ci): reject unpinned container image bypasses * fix(docs): refresh ADR metadata and enforce source freshness * fix(ci): align shared config consumers and combined documentation * fix(deps): restore Renovate custom manager file matching * fix(docs): require push toolchain and repair guide links * test(ffmpeg): type patch receipts and safety fixtures for push checks * test(ci): type impact and Go workflow contracts * docs: repair topic links and identify unavailable evidence * fix(build): consume the shared Level Zero version * docs: refresh FFmpeg guide and correct PSNR evidence * fix(dev): make container shell failures explicit * fix(hooks): retain documentation checks on large diffs --------- Co-authored-by: Lusoris <lusoris@pm.me>
This was referenced Sep 8, 2026
lusoris
added a commit
that referenced
this pull request
Sep 15, 2026
…ADR-1231) (#1396) * build(docker): define every container base image in one config file (ADR-1231) Container base images were pinned by hand at each point of use and drifted. `Dockerfile.controller` and `Dockerfile.operator` still built on Debian 12 and shipped on distroless-debian12 after the rest of the tree moved to Debian 13, and both quoted a golang digest in their header comments that did not match their own FROM line. Two CUDA pins sat on Ubuntu 24.04 beside a sibling stage on 26.04. Four further pins were invisible to any FROM scan because they lived in `COPY --from=<image>` -- the Go toolchain in dev/Containerfile and the CUDA, ROCm and oneAPI runtime libraries in Dockerfile.node. Those were the most stale images in the repository, which is the argument both for gating `COPY --from` and for converting them to named stages. build-config.env is now the single source of truth. No Dockerfile names a base directly: each takes it as an ARG whose default mirrors the config, so a plain `docker build` still works with no wrapper and CI can override any base with --build-arg. `scripts/ci/check-base-image-single-source.sh` fails on drift, on a pin that is not digest-pinned, and on a version knob that disagrees with its pin; `make base-images-sync` rewrites the mirrors. Same authority-plus-drift- check shape as check-default-model-single-source.sh. Removes four Debian 12 pins and two Ubuntu 24.04 CUDA pins. `vmafx-controller` gains an explicit `USER 65532:65532`: its old cc-debian12 pin lacked the `:nonroot` suffix, so it ran as root. Both its listen ports are above 1024. ROCm and oneAPI keep Ubuntu 24.04 under explicit, self-closing exemptions. Each is a major SDK migration needing matching source changes, not a pin swap: ROCm 7.2.4 -> 10.0.0 is PR #1386, and oneAPI 2025 -> 2026.1 is blocked on a libsycl soname bump (.so.8 -> .so.9), a glibc gap between Intel's Ubuntu images and Debian 13, and the NEO GPU driver that Intel's runtime image carries and Debian does not package. All measured; see the research digest. Also fixes a latent bug in dev/Containerfile: `ENV PYTHONPATH=/build/vmaf/python:${PYTHONPATH}` expanded to a trailing colon because PYTHONPATH was never set, putting the current directory on sys.path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * build(config): extend the single source to toolchain versions and Renovate The first commit unified container base images. That was half the problem: the same versions live in CI workflows, and they had drifted there independently. ROCm was 7.2.4 in build.yml and 7.2.3 in libvmaf-build-matrix.yml, so CI validated a ROCm the published images never shipped. The Level Zero loader existed at FOUR versions simultaneously -- v1.18.5, v1.28.0 twice, v1.29.0 and 1.32.0 -- and a skew there does not fail a build. It surfaces at runtime as "No device of requested type available", which reads like a driver or permissions problem and is not. build-config.env now carries the toolchain versions too (ROCM_APT_VERSION, LEVEL_ZERO_VERSION, CUDA_APT_PACKAGE, PYTHON_CI_VERSION). Workflow `run:` steps source it directly; scripts/ci/load-build-config.sh exports every knob into $GITHUB_ENV for steps that need a value earlier than that. The Windows SYCL leg runs under cmd and cannot source it, so it mirrors the value by hand and the gate keeps that mirror honest. scripts/ci/check-workflow-versions.py enforces it. It is a separate script because the Level Zero clone puts `--branch vX.Y.Z` and the repository URL on different lines, so a line-oriented grep silently sees only the first line -- the first version of this check passed against a deliberately planted literal. Renovate is reconfigured to match. Its `dockerfile` manager understands `ARG X=image` + `FROM $X`, so left alone it would bump the Dockerfile mirrors and leave build-config.env behind -- failing this change's own gate on Renovate's PRs. One custom manager now matches both the config line and the ARG form across all nine wired files (41 pins), so a bump lands every copy in one PR, and a packageRule disables the built-in manager on exactly those files. docker/dev/*.Dockerfile deliberately keeps the built-in manager. The gate also classifies config keys by the SHAPE of their value rather than by a list of names, so adding a version knob cannot accidentally subject it to the digest-pinning rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * build(rocm): move the ROCm images to the Ubuntu 26.04 variant of 10.0.0 ADR-1225 landed ROCm 10.0.0 on the ubuntu-24.04 variant, which left the tree with the last release images on a distro everything else had moved off. AMD publishes rocm/dev-ubuntu-26.04:10.0.0-full too, but this is not a pin swap to make casually: Dockerfile.node and Dockerfile.production-gpu COPY these libraries out of the vendor image into a Debian 13 runtime, so a libc mismatch does not fail the build -- it fails the load, on the user's machine, with a "cannot open shared object file" that points at the wrong thing. Measured before moving: /opt/rocm/core-10.0/lib layout identical in both variants max GLIBC symbol in the closure 2.28 in both (AMD builds to an old floor) Debian 13 runtime provides 2.41 So the 26.04 variant is a drop-in. This is the OPPOSITE of the oneAPI case, where Intel's Ubuntu 26.04 image requires glibc 2.43 and genuinely cannot be copied onto Debian 13 -- the question has to be asked per vendor rather than answered once for all of them. With ROCm current, the gate's Ubuntu-24.04 exemption list is down to the two oneAPI entries, which disappear with the 2026.1 migration. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * build(ci): single-source the oneAPI version and move Linux to 2026.1 build-config.env was added to stop version drift, but the oneAPI knob was deliberately left out with a comment saying the 2026.1 migration was "the immediate follow-up to this file". That follow-up never happened, so three workflows kept spelling `intel-oneapi-compiler-dpcpp-cpp-2025.3` inline and the CI toolchain sat two majors behind the compiler installed on the workstation. All three already sourced build-config.env for LEVEL_ZERO_VERSION, so the plumbing existed; only the version was missing from it. They now read ${ONEAPI_APT_PACKAGE}, and ONEAPI_VERSION is 2026.1. The config also gains ONEAPI_RUNTIME_APT_PACKAGES, which names intel-oneapi-umf explicitly: intel-oneapi-runtime-dpcpp-cpp does not depend on it, so libumf.so.1 goes missing and the adapter fails to load -- surfacing as "No device of requested type available" rather than as a link error. That was measured while preparing the migration, and is the kind of fact the single source exists to keep. The Windows leg stays at 2025.3.0.372 on purpose. Its offline-installer URL carries an opaque per-build GUID that cannot be derived from a version number, so bumping it needs a looked-up URL rather than a string substitution; it is recorded as ONEAPI_WINDOWS_VERSION so the lag is visible in the same file instead of buried in a workflow, and tracked as T-ONEAPI-WINDOWS-CI-LAG-2026-09-07. Verified: load-build-config.sh resolves ONEAPI_APT_PACKAGE to intel-oneapi-compiler-dpcpp-cpp-2026.1, no literal 2025.3 oneAPI package name remains in .github/workflows/, and both check-workflow-versions.py and check-base-image-single-source.sh pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(rocm): restore valid Ubuntu 24.04 ROCm 10 image pin from ADR-1225 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(ci): strip tag component from repository name in install-rocm-from-image Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(ci): format check-workflow-versions.py with black Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: repair RC1 hooks, patch maintenance and repository hygiene (#1420) * fix(ffmpeg): replay percentile guard after the existing mappings * docs(adr): define stable-release FFmpeg patch maintenance * fix(ffmpeg): automate stable-release patch refresh and required replay * fix(go): confine predictor cards and require Go validation * docs(adr): preserve work during agent-state cleanup * fix(dev): preserve work during agent-state cleanup * fix(ci): recognize root build config in dependency PRs * fix(hooks): preserve dispatch across worktree removal * fix(ci): reject unpinned container image bypasses * fix(docs): refresh ADR metadata and enforce source freshness * fix(ci): align shared config consumers and combined documentation * fix(deps): restore Renovate custom manager file matching * fix(docs): require push toolchain and repair guide links * test(ffmpeg): type patch receipts and safety fixtures for push checks * test(ci): type impact and Go workflow contracts * docs: repair topic links and identify unavailable evidence * fix(build): consume the shared Level Zero version * docs: refresh FFmpeg guide and correct PSNR evidence * fix(dev): make container shell failures explicit * fix(hooks): retain documentation checks on large diffs --------- Co-authored-by: Lusoris <lusoris@pm.me> * fix(docker): un-ignore build-config.env, which build*/ was silently excluding The Dev Container build failed on this PR with failed to compute cache key: "/build-config.env": not found even though `build-config.env` sits at the repository root, is tracked, and no `.dockerignore` pattern names it. Docker cleans a trailing slash off an ignore pattern, so `build*/` is matched as `build*`, which also matches the new root-level *file* this PR introduces, and the file never enters the build context. `COPY build-config.env` then cannot find it. Reproduced from first principles rather than inferred: a two-line `.dockerignore` containing only `build*/` and `build-*/`, plus a Dockerfile whose sole instruction is `COPY build-config.env`, fails with that exact message on Docker 29.8.0. Adding `!build-config.env` makes it succeed, and a follow-up build confirms `build-cpu/` is still excluded, so the directory patterns keep working. This is the last thing standing between the single-source config and a green container build — which matters beyond this PR, because master's `Docker Image Build` has been red since 2026-09-08 on the FFmpeg patch series that this branch also repairs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(docker): drop NVIDIA's malformed CUDA apt list before installing `Docker Image Build` fails at the first `apt-get install` with E: Encountered a section with no Package: header E: Problem with MergeList /var/lib/apt/lists/developer.download.nvidia.com_ compute_cuda_repos_ubuntu2604_x86%5f64_Packages.lz4 NVIDIA's ubuntu2604 CUDA index is currently malformed. Verified as upstream and not ours by running the pinned digest directly — `docker run nvidia/cuda:13.3.1-devel-ubuntu26.04@sha256:8cf42b8d… apt-get update` reproduces it with no repository content involved — and the digest is byte-identical on master and on this branch, so the base image did not change here. None of the packages installed in that step come from the CUDA repo, and the toolkit is baked into the image rather than installed from it: after removing the list, `nvcc --version` still reports `cuda_13.3.r13.3`, and the install of build-essential, ccache, ninja-build, nasm, python3 and pkg-config succeeds. The whole first stage now builds locally to a clean export. This is the same remedy `.github/workflows/go-ci.yml` already applies to the Microsoft and Azure sources on hosted runners, for exactly this class of third-party-index breakage. Second of the two container failures on this branch. The first was the FFmpeg patch series: patch 0018 stopped applying at `vf_libvmaf.c:940` once percentile pooling landed twice, which is what has kept master's `Docker Image Build` and `FFmpeg SYCL` red since 2026-09-08, and which the 0018 refresh already carried here repairs — `ffmpeg_patch_stack.py --check` replays all 18 patches onto n9.0.1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Lusoris <lusoris@pm.me> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2 of 7 tasks
lusoris
added a commit
that referenced
this pull request
Oct 1, 2026
…23e8f2 (#1761) * docs: record the fork's check of the upstream defects verified on 6ec23e8f2 Fifteen defects reproduced on Netflix master were run against the fork: three reproduced and are fixed (#1305, #1420 as a hang, #1613), two are documented (#910, #755 and #1180), ten are not affected. The dated section in known-upstream-bugs.md and the Confirmed not-affected rows of the state ledger carry the evidence; Netflix 8e7a1ac4e (revert of #1476) needs nothing from the fork. * docs: regenerate the indexes and the citation map after rebasing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FFmpeg patch 0018 duplicated mappings already added by patch 0005, so the maintained series stopped at patch 18. Local pre-push was a dangling link into a removed worktree; generated documentation metadata and Renovate file matching had also drifted. This PR repairs those paths and adds regression checks that exercise the actual Git, hook, workflow and configuration behavior.
This change was stacked on #1396 (
build/base-image-single-source, baseea1158432c3dcfa88c3b8425602d8928c2136db8) and was merged into that parent branch by local merge automation at834f84063a4640e72b0916a9a5b3a566b6810fec. It has not landed on master. The parent is now held in draft with auto-merge disabled pending full local/hosted acceptance. The release PR #1213 is unchanged.Current parent CI independently confirms the patch-18 replay defect: Dev Container Build and Docker Image Build stop while applying that patch. Those are parent-head failures; the full images have not yet been rebuilt from this PR.
Most changed documentation files are generated ADR tag/navigation output: 391 paths under
docs/adr/. Intentional scaffolds and Netflix golden assertions are preserved.Validation
Passed locally on the combined branch:
d54e173bc7ef0cd0ae80792fecfe2f0ee36c381b. Percentile mapper compiled both with and without the feature macro.Separate container-shell probes reproduce the old missing-cache environment and exercise import/collection failures and output validation. They are not a native image build.
Pending: full local
make lint+make test, Netflix golden/sanitizer acceptance, fresh source-verified dev image and hosted required checks. The shared dev container has unknown source provenance and an ongoing device probe; it was not restarted or rebuilt. These focused checks do not establish RC1 readiness; the parent must remain held until full acceptance passes.Deep-dive deliverables
docs/research/1238-go-security-required-gate.md,docs/research/1239-agent-cleanup-preservation.md,docs/research/ffmpeg-release-patch-lifecycle.md,docs/research/renovate-file-pattern-delimiters.md, and hook/documentation research in this diff.AGENTS.mdinvariant note — root, scripts/CI, dev and package notes updated with the maintained contracts.changelog.d/fixed/with rendered output refreshed.docs/rebase-notes.mdcover FFmpeg, required Go validation, hook ownership, cleanup safety, metadata and configuration consumers.Bug-status hygiene
docs/state.mdrecords the fixes and the unresolved original PSNR symptom.assertAlmostEqualscore was changed.make lintandmake testare green locally.Reproducer