Repository navigation
feat(hip): migrate every ROCm consumer to 10.0.0 from digest-pinned images (ADR-1225) - #1386
Merged
Merged
Conversation
lusoris
force-pushed
the
feat/rocm-10-migration
branch
2 times, most recently
from
September 7, 2026 12:16
20bdf88 to
a334256
Compare
…mages
AMD froze the repo.radeon.com apt channel at 7.2.4 when ROCm moved to the
"TheRock" build/release system at 7.14: apt/7.14 and apt/10.0.0 both 404,
the manylinux channel stops at rocm-rel-7.2.4, and the TheRock wheel index
carries only 7.14.0 alphas. The fork was therefore stuck on a ROCm-legacy
toolchain heading into 1.0.0, with no version-string edit available to get
off it.
Take ROCm from the digest-pinned rocm/dev-ubuntu-24.04:10.0.0-full image
instead. dev/Containerfile, docker/Dockerfile.production-gpu and
docker/Dockerfile.node copy a pruned /opt/rocm out of it; the two CI HIP
lanes use the new scripts/ci/install-rocm-from-image.sh, which streams the
image's /opt/rocm out of the registry layer by layer. A docker pull is not
viable on a runner that already carries CUDA and oneAPI -- 8.2 GB
compressed, 29 GB extracted -- so nothing is ever stored whole: peak disk
is the 5.5 GB result.
Three things ROCm 10 changed that this had to handle:
- /opt/rocm/{bin,lib,include,llvm,...} are now /etc/alternatives symlinks
into /opt/rocm/core-10.0/. Copying or extracting /opt/rocm alone lands
dangling links, so both the rocm-src stage and the extractor repoint
them at core-10.0/<name> relative to /opt/rocm.
- libamdhip64.so's link closure widened to 19 libraries (234 MB) wired by
$ORIGIN-relative RPATHs. The node-rocm runtime stage's flat copy of
libamdhip64.so* + libhsa-runtime64.so* no longer loads; it now ships
the whole closure with its layout intact, verified at 0 unresolved
libraries in a scratch stage carrying only the copied files.
- gfx1036 is natively supported, so HSA_OVERRIDE_GFX_VERSION=10.3.0 is
dropped from dev/docker-compose.yml. Keeping it would alias the agent
to gfx1030 while meson compiles gfx1036 code objects.
The prune list (19 GB -> 5.5 GB) is gated by a hipcc smoke compile in the
same stage: an earlier pass removed librocprofiler-register with a tidy
librocprof* glob and every HIP binary then failed at load.
Renovate follows the image digest now rather than scraping the frozen apt
directory listing.
Verified on AMD gfx1036 under Linux 7.2.3: HIP suite 19 Ok / 0 Fail, and an
end-to-end HIP score of 45.315104, bit-identical to the CPU reference.
Refs: ADR-1225
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The badge scraped `ARG ROCM_VER=` out of dev/Containerfile, which this branch removes along with the apt install. Read the version out of the digest-pinned rocm/dev-ubuntu-24.04:<version>-full reference instead, so it keeps reporting a live value rather than going blank. Verified by running the badge's own regex against the new Containerfile: matches `rocm/dev-ubuntu-24.04:10.0.0-full`, renders `10.0.0`. Refs: ADR-1225 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
force-pushed
the
feat/rocm-10-migration
branch
from
September 7, 2026 15:41
a334256 to
f482628
Compare
12 of 18 tasks
lusoris
marked this pull request as ready for review
September 7, 2026 15:43
lusoris
enabled auto-merge (squash)
September 7, 2026 15:43
The docker-publish source-binding test hardcoded the full job names, including the vendor SDK generation: `build-rocm7`, `build-oneapi2025`. Renaming a job as part of an SDK bump therefore failed the Release Script Contract with `AssertionError: missing job build-rocm7` — a failure about the rename, not about anything the test actually checks. This branch's `build-rocm7` -> `build-rocm10` tripped it, and the pending oneAPI 2026 bump would have tripped it again the same way. Match the stable prefix instead and resolve the real job name from the workflow. The assertion that matters is unchanged: exactly one such build job exists and is bound to the published tag. An ambiguous prefix is now an explicit error rather than a silent first-match, and a genuinely missing job still fails (`missing job matching build-rocm*`). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 7, 2026
…ADR-1231) Container base images were pinned by hand at each point of use and drifted. `Dockerfile.controller` and `Dockerfile.operator` still built on Debian 12 and shipped on distroless-debian12 after the rest of the tree moved to Debian 13, and both quoted a golang digest in their header comments that did not match their own FROM line. Two CUDA pins sat on Ubuntu 24.04 beside a sibling stage on 26.04. Four further pins were invisible to any FROM scan because they lived in `COPY --from=<image>` -- the Go toolchain in dev/Containerfile and the CUDA, ROCm and oneAPI runtime libraries in Dockerfile.node. Those were the most stale images in the repository, which is the argument both for gating `COPY --from` and for converting them to named stages. build-config.env is now the single source of truth. No Dockerfile names a base directly: each takes it as an ARG whose default mirrors the config, so a plain `docker build` still works with no wrapper and CI can override any base with --build-arg. `scripts/ci/check-base-image-single-source.sh` fails on drift, on a pin that is not digest-pinned, and on a version knob that disagrees with its pin; `make base-images-sync` rewrites the mirrors. Same authority-plus-drift- check shape as check-default-model-single-source.sh. Removes four Debian 12 pins and two Ubuntu 24.04 CUDA pins. `vmafx-controller` gains an explicit `USER 65532:65532`: its old cc-debian12 pin lacked the `:nonroot` suffix, so it ran as root. Both its listen ports are above 1024. ROCm and oneAPI keep Ubuntu 24.04 under explicit, self-closing exemptions. Each is a major SDK migration needing matching source changes, not a pin swap: ROCm 7.2.4 -> 10.0.0 is PR #1386, and oneAPI 2025 -> 2026.1 is blocked on a libsycl soname bump (.so.8 -> .so.9), a glibc gap between Intel's Ubuntu images and Debian 13, and the NEO GPU driver that Intel's runtime image carries and Debian does not package. All measured; see the research digest. Also fixes a latent bug in dev/Containerfile: `ENV PYTHONPATH=/build/vmaf/python:${PYTHONPATH}` expanded to a trailing colon because PYTHONPATH was never set, putting the current directory on sys.path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 8, 2026
…ADR-1231) Container base images were pinned by hand at each point of use and drifted. `Dockerfile.controller` and `Dockerfile.operator` still built on Debian 12 and shipped on distroless-debian12 after the rest of the tree moved to Debian 13, and both quoted a golang digest in their header comments that did not match their own FROM line. Two CUDA pins sat on Ubuntu 24.04 beside a sibling stage on 26.04. Four further pins were invisible to any FROM scan because they lived in `COPY --from=<image>` -- the Go toolchain in dev/Containerfile and the CUDA, ROCm and oneAPI runtime libraries in Dockerfile.node. Those were the most stale images in the repository, which is the argument both for gating `COPY --from` and for converting them to named stages. build-config.env is now the single source of truth. No Dockerfile names a base directly: each takes it as an ARG whose default mirrors the config, so a plain `docker build` still works with no wrapper and CI can override any base with --build-arg. `scripts/ci/check-base-image-single-source.sh` fails on drift, on a pin that is not digest-pinned, and on a version knob that disagrees with its pin; `make base-images-sync` rewrites the mirrors. Same authority-plus-drift- check shape as check-default-model-single-source.sh. Removes four Debian 12 pins and two Ubuntu 24.04 CUDA pins. `vmafx-controller` gains an explicit `USER 65532:65532`: its old cc-debian12 pin lacked the `:nonroot` suffix, so it ran as root. Both its listen ports are above 1024. ROCm and oneAPI keep Ubuntu 24.04 under explicit, self-closing exemptions. Each is a major SDK migration needing matching source changes, not a pin swap: ROCm 7.2.4 -> 10.0.0 is PR #1386, and oneAPI 2025 -> 2026.1 is blocked on a libsycl soname bump (.so.8 -> .so.9), a glibc gap between Intel's Ubuntu images and Debian 13, and the NEO GPU driver that Intel's runtime image carries and Debian does not package. All measured; see the research digest. Also fixes a latent bug in dev/Containerfile: `ENV PYTHONPATH=/build/vmaf/python:${PYTHONPATH}` expanded to a trailing colon because PYTHONPATH was never set, putting the current directory on sys.path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 8, 2026
…ADR-1231) Container base images were pinned by hand at each point of use and drifted. `Dockerfile.controller` and `Dockerfile.operator` still built on Debian 12 and shipped on distroless-debian12 after the rest of the tree moved to Debian 13, and both quoted a golang digest in their header comments that did not match their own FROM line. Two CUDA pins sat on Ubuntu 24.04 beside a sibling stage on 26.04. Four further pins were invisible to any FROM scan because they lived in `COPY --from=<image>` -- the Go toolchain in dev/Containerfile and the CUDA, ROCm and oneAPI runtime libraries in Dockerfile.node. Those were the most stale images in the repository, which is the argument both for gating `COPY --from` and for converting them to named stages. build-config.env is now the single source of truth. No Dockerfile names a base directly: each takes it as an ARG whose default mirrors the config, so a plain `docker build` still works with no wrapper and CI can override any base with --build-arg. `scripts/ci/check-base-image-single-source.sh` fails on drift, on a pin that is not digest-pinned, and on a version knob that disagrees with its pin; `make base-images-sync` rewrites the mirrors. Same authority-plus-drift- check shape as check-default-model-single-source.sh. Removes four Debian 12 pins and two Ubuntu 24.04 CUDA pins. `vmafx-controller` gains an explicit `USER 65532:65532`: its old cc-debian12 pin lacked the `:nonroot` suffix, so it ran as root. Both its listen ports are above 1024. ROCm and oneAPI keep Ubuntu 24.04 under explicit, self-closing exemptions. Each is a major SDK migration needing matching source changes, not a pin swap: ROCm 7.2.4 -> 10.0.0 is PR #1386, and oneAPI 2025 -> 2026.1 is blocked on a libsycl soname bump (.so.8 -> .so.9), a glibc gap between Intel's Ubuntu images and Debian 13, and the NEO GPU driver that Intel's runtime image carries and Debian does not package. All measured; see the research digest. Also fixes a latent bug in dev/Containerfile: `ENV PYTHONPATH=/build/vmaf/python:${PYTHONPATH}` expanded to a trailing colon because PYTHONPATH was never set, putting the current directory on sys.path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 8, 2026
…ADR-1231) Container base images were pinned by hand at each point of use and drifted. `Dockerfile.controller` and `Dockerfile.operator` still built on Debian 12 and shipped on distroless-debian12 after the rest of the tree moved to Debian 13, and both quoted a golang digest in their header comments that did not match their own FROM line. Two CUDA pins sat on Ubuntu 24.04 beside a sibling stage on 26.04. Four further pins were invisible to any FROM scan because they lived in `COPY --from=<image>` -- the Go toolchain in dev/Containerfile and the CUDA, ROCm and oneAPI runtime libraries in Dockerfile.node. Those were the most stale images in the repository, which is the argument both for gating `COPY --from` and for converting them to named stages. build-config.env is now the single source of truth. No Dockerfile names a base directly: each takes it as an ARG whose default mirrors the config, so a plain `docker build` still works with no wrapper and CI can override any base with --build-arg. `scripts/ci/check-base-image-single-source.sh` fails on drift, on a pin that is not digest-pinned, and on a version knob that disagrees with its pin; `make base-images-sync` rewrites the mirrors. Same authority-plus-drift- check shape as check-default-model-single-source.sh. Removes four Debian 12 pins and two Ubuntu 24.04 CUDA pins. `vmafx-controller` gains an explicit `USER 65532:65532`: its old cc-debian12 pin lacked the `:nonroot` suffix, so it ran as root. Both its listen ports are above 1024. ROCm and oneAPI keep Ubuntu 24.04 under explicit, self-closing exemptions. Each is a major SDK migration needing matching source changes, not a pin swap: ROCm 7.2.4 -> 10.0.0 is PR #1386, and oneAPI 2025 -> 2026.1 is blocked on a libsycl soname bump (.so.8 -> .so.9), a glibc gap between Intel's Ubuntu images and Debian 13, and the NEO GPU driver that Intel's runtime image carries and Debian does not package. All measured; see the research digest. Also fixes a latent bug in dev/Containerfile: `ENV PYTHONPATH=/build/vmaf/python:${PYTHONPATH}` expanded to a trailing colon because PYTHONPATH was never set, putting the current directory on sys.path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 8, 2026
…ADR-1231) Container base images were pinned by hand at each point of use and drifted. `Dockerfile.controller` and `Dockerfile.operator` still built on Debian 12 and shipped on distroless-debian12 after the rest of the tree moved to Debian 13, and both quoted a golang digest in their header comments that did not match their own FROM line. Two CUDA pins sat on Ubuntu 24.04 beside a sibling stage on 26.04. Four further pins were invisible to any FROM scan because they lived in `COPY --from=<image>` -- the Go toolchain in dev/Containerfile and the CUDA, ROCm and oneAPI runtime libraries in Dockerfile.node. Those were the most stale images in the repository, which is the argument both for gating `COPY --from` and for converting them to named stages. build-config.env is now the single source of truth. No Dockerfile names a base directly: each takes it as an ARG whose default mirrors the config, so a plain `docker build` still works with no wrapper and CI can override any base with --build-arg. `scripts/ci/check-base-image-single-source.sh` fails on drift, on a pin that is not digest-pinned, and on a version knob that disagrees with its pin; `make base-images-sync` rewrites the mirrors. Same authority-plus-drift- check shape as check-default-model-single-source.sh. Removes four Debian 12 pins and two Ubuntu 24.04 CUDA pins. `vmafx-controller` gains an explicit `USER 65532:65532`: its old cc-debian12 pin lacked the `:nonroot` suffix, so it ran as root. Both its listen ports are above 1024. ROCm and oneAPI keep Ubuntu 24.04 under explicit, self-closing exemptions. Each is a major SDK migration needing matching source changes, not a pin swap: ROCm 7.2.4 -> 10.0.0 is PR #1386, and oneAPI 2025 -> 2026.1 is blocked on a libsycl soname bump (.so.8 -> .so.9), a glibc gap between Intel's Ubuntu images and Debian 13, and the NEO GPU driver that Intel's runtime image carries and Debian does not package. All measured; see the research digest. Also fixes a latent bug in dev/Containerfile: `ENV PYTHONPATH=/build/vmaf/python:${PYTHONPATH}` expanded to a trailing colon because PYTHONPATH was never set, putting the current directory on sys.path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 8, 2026
…ADR-1231) Container base images were pinned by hand at each point of use and drifted. `Dockerfile.controller` and `Dockerfile.operator` still built on Debian 12 and shipped on distroless-debian12 after the rest of the tree moved to Debian 13, and both quoted a golang digest in their header comments that did not match their own FROM line. Two CUDA pins sat on Ubuntu 24.04 beside a sibling stage on 26.04. Four further pins were invisible to any FROM scan because they lived in `COPY --from=<image>` -- the Go toolchain in dev/Containerfile and the CUDA, ROCm and oneAPI runtime libraries in Dockerfile.node. Those were the most stale images in the repository, which is the argument both for gating `COPY --from` and for converting them to named stages. build-config.env is now the single source of truth. No Dockerfile names a base directly: each takes it as an ARG whose default mirrors the config, so a plain `docker build` still works with no wrapper and CI can override any base with --build-arg. `scripts/ci/check-base-image-single-source.sh` fails on drift, on a pin that is not digest-pinned, and on a version knob that disagrees with its pin; `make base-images-sync` rewrites the mirrors. Same authority-plus-drift- check shape as check-default-model-single-source.sh. Removes four Debian 12 pins and two Ubuntu 24.04 CUDA pins. `vmafx-controller` gains an explicit `USER 65532:65532`: its old cc-debian12 pin lacked the `:nonroot` suffix, so it ran as root. Both its listen ports are above 1024. ROCm and oneAPI keep Ubuntu 24.04 under explicit, self-closing exemptions. Each is a major SDK migration needing matching source changes, not a pin swap: ROCm 7.2.4 -> 10.0.0 is PR #1386, and oneAPI 2025 -> 2026.1 is blocked on a libsycl soname bump (.so.8 -> .so.9), a glibc gap between Intel's Ubuntu images and Debian 13, and the NEO GPU driver that Intel's runtime image carries and Debian does not package. All measured; see the research digest. Also fixes a latent bug in dev/Containerfile: `ENV PYTHONPATH=/build/vmaf/python:${PYTHONPATH}` expanded to a trailing colon because PYTHONPATH was never set, putting the current directory on sys.path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 8, 2026
…ADR-1231) Container base images were pinned by hand at each point of use and drifted. `Dockerfile.controller` and `Dockerfile.operator` still built on Debian 12 and shipped on distroless-debian12 after the rest of the tree moved to Debian 13, and both quoted a golang digest in their header comments that did not match their own FROM line. Two CUDA pins sat on Ubuntu 24.04 beside a sibling stage on 26.04. Four further pins were invisible to any FROM scan because they lived in `COPY --from=<image>` -- the Go toolchain in dev/Containerfile and the CUDA, ROCm and oneAPI runtime libraries in Dockerfile.node. Those were the most stale images in the repository, which is the argument both for gating `COPY --from` and for converting them to named stages. build-config.env is now the single source of truth. No Dockerfile names a base directly: each takes it as an ARG whose default mirrors the config, so a plain `docker build` still works with no wrapper and CI can override any base with --build-arg. `scripts/ci/check-base-image-single-source.sh` fails on drift, on a pin that is not digest-pinned, and on a version knob that disagrees with its pin; `make base-images-sync` rewrites the mirrors. Same authority-plus-drift- check shape as check-default-model-single-source.sh. Removes four Debian 12 pins and two Ubuntu 24.04 CUDA pins. `vmafx-controller` gains an explicit `USER 65532:65532`: its old cc-debian12 pin lacked the `:nonroot` suffix, so it ran as root. Both its listen ports are above 1024. ROCm and oneAPI keep Ubuntu 24.04 under explicit, self-closing exemptions. Each is a major SDK migration needing matching source changes, not a pin swap: ROCm 7.2.4 -> 10.0.0 is PR #1386, and oneAPI 2025 -> 2026.1 is blocked on a libsycl soname bump (.so.8 -> .so.9), a glibc gap between Intel's Ubuntu images and Debian 13, and the NEO GPU driver that Intel's runtime image carries and Debian does not package. All measured; see the research digest. Also fixes a latent bug in dev/Containerfile: `ENV PYTHONPATH=/build/vmaf/python:${PYTHONPATH}` expanded to a trailing colon because PYTHONPATH was never set, putting the current directory on sys.path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 8, 2026
…ADR-1231) Container base images were pinned by hand at each point of use and drifted. `Dockerfile.controller` and `Dockerfile.operator` still built on Debian 12 and shipped on distroless-debian12 after the rest of the tree moved to Debian 13, and both quoted a golang digest in their header comments that did not match their own FROM line. Two CUDA pins sat on Ubuntu 24.04 beside a sibling stage on 26.04. Four further pins were invisible to any FROM scan because they lived in `COPY --from=<image>` -- the Go toolchain in dev/Containerfile and the CUDA, ROCm and oneAPI runtime libraries in Dockerfile.node. Those were the most stale images in the repository, which is the argument both for gating `COPY --from` and for converting them to named stages. build-config.env is now the single source of truth. No Dockerfile names a base directly: each takes it as an ARG whose default mirrors the config, so a plain `docker build` still works with no wrapper and CI can override any base with --build-arg. `scripts/ci/check-base-image-single-source.sh` fails on drift, on a pin that is not digest-pinned, and on a version knob that disagrees with its pin; `make base-images-sync` rewrites the mirrors. Same authority-plus-drift- check shape as check-default-model-single-source.sh. Removes four Debian 12 pins and two Ubuntu 24.04 CUDA pins. `vmafx-controller` gains an explicit `USER 65532:65532`: its old cc-debian12 pin lacked the `:nonroot` suffix, so it ran as root. Both its listen ports are above 1024. ROCm and oneAPI keep Ubuntu 24.04 under explicit, self-closing exemptions. Each is a major SDK migration needing matching source changes, not a pin swap: ROCm 7.2.4 -> 10.0.0 is PR #1386, and oneAPI 2025 -> 2026.1 is blocked on a libsycl soname bump (.so.8 -> .so.9), a glibc gap between Intel's Ubuntu images and Debian 13, and the NEO GPU driver that Intel's runtime image carries and Debian does not package. All measured; see the research digest. Also fixes a latent bug in dev/Containerfile: `ENV PYTHONPATH=/build/vmaf/python:${PYTHONPATH}` expanded to a trailing colon because PYTHONPATH was never set, putting the current directory on sys.path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
Sep 15, 2026
…ADR-1231) (#1396) * build(docker): define every container base image in one config file (ADR-1231) Container base images were pinned by hand at each point of use and drifted. `Dockerfile.controller` and `Dockerfile.operator` still built on Debian 12 and shipped on distroless-debian12 after the rest of the tree moved to Debian 13, and both quoted a golang digest in their header comments that did not match their own FROM line. Two CUDA pins sat on Ubuntu 24.04 beside a sibling stage on 26.04. Four further pins were invisible to any FROM scan because they lived in `COPY --from=<image>` -- the Go toolchain in dev/Containerfile and the CUDA, ROCm and oneAPI runtime libraries in Dockerfile.node. Those were the most stale images in the repository, which is the argument both for gating `COPY --from` and for converting them to named stages. build-config.env is now the single source of truth. No Dockerfile names a base directly: each takes it as an ARG whose default mirrors the config, so a plain `docker build` still works with no wrapper and CI can override any base with --build-arg. `scripts/ci/check-base-image-single-source.sh` fails on drift, on a pin that is not digest-pinned, and on a version knob that disagrees with its pin; `make base-images-sync` rewrites the mirrors. Same authority-plus-drift- check shape as check-default-model-single-source.sh. Removes four Debian 12 pins and two Ubuntu 24.04 CUDA pins. `vmafx-controller` gains an explicit `USER 65532:65532`: its old cc-debian12 pin lacked the `:nonroot` suffix, so it ran as root. Both its listen ports are above 1024. ROCm and oneAPI keep Ubuntu 24.04 under explicit, self-closing exemptions. Each is a major SDK migration needing matching source changes, not a pin swap: ROCm 7.2.4 -> 10.0.0 is PR #1386, and oneAPI 2025 -> 2026.1 is blocked on a libsycl soname bump (.so.8 -> .so.9), a glibc gap between Intel's Ubuntu images and Debian 13, and the NEO GPU driver that Intel's runtime image carries and Debian does not package. All measured; see the research digest. Also fixes a latent bug in dev/Containerfile: `ENV PYTHONPATH=/build/vmaf/python:${PYTHONPATH}` expanded to a trailing colon because PYTHONPATH was never set, putting the current directory on sys.path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * build(config): extend the single source to toolchain versions and Renovate The first commit unified container base images. That was half the problem: the same versions live in CI workflows, and they had drifted there independently. ROCm was 7.2.4 in build.yml and 7.2.3 in libvmaf-build-matrix.yml, so CI validated a ROCm the published images never shipped. The Level Zero loader existed at FOUR versions simultaneously -- v1.18.5, v1.28.0 twice, v1.29.0 and 1.32.0 -- and a skew there does not fail a build. It surfaces at runtime as "No device of requested type available", which reads like a driver or permissions problem and is not. build-config.env now carries the toolchain versions too (ROCM_APT_VERSION, LEVEL_ZERO_VERSION, CUDA_APT_PACKAGE, PYTHON_CI_VERSION). Workflow `run:` steps source it directly; scripts/ci/load-build-config.sh exports every knob into $GITHUB_ENV for steps that need a value earlier than that. The Windows SYCL leg runs under cmd and cannot source it, so it mirrors the value by hand and the gate keeps that mirror honest. scripts/ci/check-workflow-versions.py enforces it. It is a separate script because the Level Zero clone puts `--branch vX.Y.Z` and the repository URL on different lines, so a line-oriented grep silently sees only the first line -- the first version of this check passed against a deliberately planted literal. Renovate is reconfigured to match. Its `dockerfile` manager understands `ARG X=image` + `FROM $X`, so left alone it would bump the Dockerfile mirrors and leave build-config.env behind -- failing this change's own gate on Renovate's PRs. One custom manager now matches both the config line and the ARG form across all nine wired files (41 pins), so a bump lands every copy in one PR, and a packageRule disables the built-in manager on exactly those files. docker/dev/*.Dockerfile deliberately keeps the built-in manager. The gate also classifies config keys by the SHAPE of their value rather than by a list of names, so adding a version knob cannot accidentally subject it to the digest-pinning rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * build(rocm): move the ROCm images to the Ubuntu 26.04 variant of 10.0.0 ADR-1225 landed ROCm 10.0.0 on the ubuntu-24.04 variant, which left the tree with the last release images on a distro everything else had moved off. AMD publishes rocm/dev-ubuntu-26.04:10.0.0-full too, but this is not a pin swap to make casually: Dockerfile.node and Dockerfile.production-gpu COPY these libraries out of the vendor image into a Debian 13 runtime, so a libc mismatch does not fail the build -- it fails the load, on the user's machine, with a "cannot open shared object file" that points at the wrong thing. Measured before moving: /opt/rocm/core-10.0/lib layout identical in both variants max GLIBC symbol in the closure 2.28 in both (AMD builds to an old floor) Debian 13 runtime provides 2.41 So the 26.04 variant is a drop-in. This is the OPPOSITE of the oneAPI case, where Intel's Ubuntu 26.04 image requires glibc 2.43 and genuinely cannot be copied onto Debian 13 -- the question has to be asked per vendor rather than answered once for all of them. With ROCm current, the gate's Ubuntu-24.04 exemption list is down to the two oneAPI entries, which disappear with the 2026.1 migration. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * build(ci): single-source the oneAPI version and move Linux to 2026.1 build-config.env was added to stop version drift, but the oneAPI knob was deliberately left out with a comment saying the 2026.1 migration was "the immediate follow-up to this file". That follow-up never happened, so three workflows kept spelling `intel-oneapi-compiler-dpcpp-cpp-2025.3` inline and the CI toolchain sat two majors behind the compiler installed on the workstation. All three already sourced build-config.env for LEVEL_ZERO_VERSION, so the plumbing existed; only the version was missing from it. They now read ${ONEAPI_APT_PACKAGE}, and ONEAPI_VERSION is 2026.1. The config also gains ONEAPI_RUNTIME_APT_PACKAGES, which names intel-oneapi-umf explicitly: intel-oneapi-runtime-dpcpp-cpp does not depend on it, so libumf.so.1 goes missing and the adapter fails to load -- surfacing as "No device of requested type available" rather than as a link error. That was measured while preparing the migration, and is the kind of fact the single source exists to keep. The Windows leg stays at 2025.3.0.372 on purpose. Its offline-installer URL carries an opaque per-build GUID that cannot be derived from a version number, so bumping it needs a looked-up URL rather than a string substitution; it is recorded as ONEAPI_WINDOWS_VERSION so the lag is visible in the same file instead of buried in a workflow, and tracked as T-ONEAPI-WINDOWS-CI-LAG-2026-09-07. Verified: load-build-config.sh resolves ONEAPI_APT_PACKAGE to intel-oneapi-compiler-dpcpp-cpp-2026.1, no literal 2025.3 oneAPI package name remains in .github/workflows/, and both check-workflow-versions.py and check-base-image-single-source.sh pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(rocm): restore valid Ubuntu 24.04 ROCm 10 image pin from ADR-1225 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(ci): strip tag component from repository name in install-rocm-from-image Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(ci): format check-workflow-versions.py with black Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: repair RC1 hooks, patch maintenance and repository hygiene (#1420) * fix(ffmpeg): replay percentile guard after the existing mappings * docs(adr): define stable-release FFmpeg patch maintenance * fix(ffmpeg): automate stable-release patch refresh and required replay * fix(go): confine predictor cards and require Go validation * docs(adr): preserve work during agent-state cleanup * fix(dev): preserve work during agent-state cleanup * fix(ci): recognize root build config in dependency PRs * fix(hooks): preserve dispatch across worktree removal * fix(ci): reject unpinned container image bypasses * fix(docs): refresh ADR metadata and enforce source freshness * fix(ci): align shared config consumers and combined documentation * fix(deps): restore Renovate custom manager file matching * fix(docs): require push toolchain and repair guide links * test(ffmpeg): type patch receipts and safety fixtures for push checks * test(ci): type impact and Go workflow contracts * docs: repair topic links and identify unavailable evidence * fix(build): consume the shared Level Zero version * docs: refresh FFmpeg guide and correct PSNR evidence * fix(dev): make container shell failures explicit * fix(hooks): retain documentation checks on large diffs --------- Co-authored-by: Lusoris <lusoris@pm.me> * fix(docker): un-ignore build-config.env, which build*/ was silently excluding The Dev Container build failed on this PR with failed to compute cache key: "/build-config.env": not found even though `build-config.env` sits at the repository root, is tracked, and no `.dockerignore` pattern names it. Docker cleans a trailing slash off an ignore pattern, so `build*/` is matched as `build*`, which also matches the new root-level *file* this PR introduces, and the file never enters the build context. `COPY build-config.env` then cannot find it. Reproduced from first principles rather than inferred: a two-line `.dockerignore` containing only `build*/` and `build-*/`, plus a Dockerfile whose sole instruction is `COPY build-config.env`, fails with that exact message on Docker 29.8.0. Adding `!build-config.env` makes it succeed, and a follow-up build confirms `build-cpu/` is still excluded, so the directory patterns keep working. This is the last thing standing between the single-source config and a green container build — which matters beyond this PR, because master's `Docker Image Build` has been red since 2026-09-08 on the FFmpeg patch series that this branch also repairs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(docker): drop NVIDIA's malformed CUDA apt list before installing `Docker Image Build` fails at the first `apt-get install` with E: Encountered a section with no Package: header E: Problem with MergeList /var/lib/apt/lists/developer.download.nvidia.com_ compute_cuda_repos_ubuntu2604_x86%5f64_Packages.lz4 NVIDIA's ubuntu2604 CUDA index is currently malformed. Verified as upstream and not ours by running the pinned digest directly — `docker run nvidia/cuda:13.3.1-devel-ubuntu26.04@sha256:8cf42b8d… apt-get update` reproduces it with no repository content involved — and the digest is byte-identical on master and on this branch, so the base image did not change here. None of the packages installed in that step come from the CUDA repo, and the toolkit is baked into the image rather than installed from it: after removing the list, `nvcc --version` still reports `cuda_13.3.r13.3`, and the install of build-essential, ccache, ninja-build, nasm, python3 and pkg-config succeeds. The whole first stage now builds locally to a clean export. This is the same remedy `.github/workflows/go-ci.yml` already applies to the Microsoft and Azure sources on hosted runners, for exactly this class of third-party-index breakage. Second of the two container failures on this branch. The first was the FFmpeg patch series: patch 0018 stopped applying at `vf_libvmaf.c:940` once percentile pooling landed twice, which is what has kept master's `Docker Image Build` and `FFmpeg SYCL` red since 2026-09-08, and which the 0018 refresh already carried here repairs — `ffmpeg_patch_stack.py --check` replays all 18 patches onto n9.0.1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Lusoris <lusoris@pm.me> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AMD froze the
repo.radeon.com/rocm/apt/channel at 7.2.4 when ROCm moved tothe "TheRock" build/release system at 7.14 —
apt/7.14andapt/10.0.0both 404,the manylinux channel stops at
rocm-rel-7.2.4, and the TheRock wheel index carriesonly 7.14.0 alphas. The fork was heading into 1.0.0 on a ROCm-legacy toolchain with
no version-string edit available to get off it.
This takes ROCm from the digest-pinned
rocm/dev-ubuntu-24.04:10.0.0-fullimageinstead, across every consumer at once: the dev container, both published GPU
images, and both CI HIP lanes.
Three things ROCm 10 changed that this had to handle — each found by a check
that failed, not by reading release notes:
/opt/rocm/{bin,lib,include,llvm,…}are/etc/alternativessymlinks into/opt/rocm/core-10.0/. Copying or extracting/opt/rocmalone lands a directoryof dangling links; both the
rocm-srcstage and the extractor repoint them atcore-10.0/<name>relative to/opt/rocm.libamdhip64.so's link closure widened to 19 libraries / 234 MB(
libLLVM,libclang-cpp,libamd_comgr,librocm_kpack,librocprofiler-register, therocm_sysdepsbundle), wired by$ORIGIN-relative RPATHs. Thenode-rocmstage's flat two-library copy nolonger loads; it now ships the whole closure with its layout intact.
gfx1036is natively supported, soHSA_OVERRIDE_GFX_VERSION=10.3.0is droppedfrom
dev/docker-compose.yml— keeping it would alias the agent togfx1030while meson compiles
gfx1036code objects.docker pullis not viable in CI (8.2 GB compressed / 29 GB extracted, on a runnerthat already carries CUDA and oneAPI), so
scripts/ci/install-rocm-from-image.shstreams each layer blob from the registry straight into
tar, extracting only/opt/rocmminus the math libraries libvmaf never links. Peak disk is the 5.5 GBresult, not the 29 GB image.
The prune list (19 GB → 5.5 GB) is gated by a hipcc smoke compile in the same
stage. That gate earned its keep: an earlier pass removed
librocprofiler-registerwith a tidy-looking
librocprof*glob, and every HIP binary then failed at loadwith
cannot open shared object file.Verification
Measured on the dev host (Linux 7.2.3-1-cachyos, AMD
gfx1036), not inferred:Container stage, built and run:
Reproducer / smoke-test command
Deep-dive deliverables (ADR-0108)
docs/research/1225-rocm-10-distribution-and-layout.md: the channel survey, the alternatives layout, the link closure, the prune measurements, and why CI streams instead of pulling.## Alternatives considered(six options, including the deferred GHCR mirror).AGENTS.mdinvariant note —dev/AGENTS.md(never restore the apt path; never prunelibrocprofiler-register;HSA_OVERRIDE_GFX_VERSIONstays out) and the rootAGENTS.mdcontainer-pin section.changelog.d/changed/1225-rocm-10-therock-migration.md,CHANGELOG.mdregenerated.docs/rebase-notes.md§feat/rocm-10-migration (ADR-1225), five invariants.Docs (rule 10)
docs/backends/hip/overview.md— required ROCm version, how to get it, and thegfx1036story.docs/development/dev-mcp.md— the container pin table and the GPU env-var table.docs/research/1225-rocm-10-distribution-and-layout.md— new digest.State (rule 13)
no state delta: dependency migration — no bug opened, closed, or ruled not-affecting.
ffmpeg-patches (rule 14)
no patch impact: no C-API surface, CLI flag,
meson_options.txtentry or publicheader changes; the meson edit is comment-only.
🤖 Generated with Claude Code