Skip to content

ci: build native release artifacts inside the canonical dev container and verify container-build provenance (ADR-1178) - #1305

Merged
lusoris merged 9 commits into
masterfrom
ci/release-artifacts-built-in-dev-container
Sep 6, 2026
Merged

lusoris merged 9 commits into
masterfrom
ci/release-artifacts-built-in-dev-container

Conversation

@lusoris

@lusoris lusoris commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Reworks native release artifact compilation (build-artifacts in .github/workflows/supply-chain.yml) to execute on the Intel Arc A380 containerised self-hosted runner (labels: [self-hosted, linux, x64, sycl-arc], provisioned by ADR-1177 / PR #1304) using the workstation's local canonical dev container environment (vmaf-sycl-arc-runner:local, derived FROM vmaf-dev-mcp:local) without any registry pull per ADR-1178 (Phase 4b.9). This completely bypasses the 29.5 GB uncompressed layer pull blocker on standard GitHub-hosted ubuntu-latest runners (14 GB SSD limit). The runner container itself is the build environment (contains GCC-13/15, Clang-19, Meson, Ninja, Python 3.14, oneAPI, CUDA 13.3, ROCm 7.2.4, and /etc/vmafx-dev-container; no Docker socket needed). Protected by a canonical tag-release guard (github.repository == 'VMAFx/vmafx' && startsWith(github.ref, 'refs/tags/v')), serialized concurrency (concurrency: group: release-artifacts-build), and generous timeout-minutes: 90. Staged release artifacts are stamped with container-build provenance via scripts/ci/check-container-build.sh --stamp, which now accepts both vmaf-dev-mcp and vmaf-sycl-arc-runner while strictly rejecting bare ubuntu-latest (exit 1). Downstream verify-native-artifacts on ubuntu-latest runs --verify, scripts/release/verify-native-release-artifacts.sh fails closed if the stamp is missing/empty/symlinked, and attach-to-release requires the provenance file alongside Cosign signatures. .github/workflows/dev-container-publish.yml is retained as optional provenance on GHCR, decoupled from release artifact compilation. Configures .github/actionlint.yaml with custom self-hosted labels.

Verification: actionlint on both workflows exit 0; scripts/ci/tests/test-check-container-build.sh 31 passed, 0 failed; scripts/release/tests/test-verify-native-release-artifacts.sh 10 passed, 0 failed; scripts/release/tests/test-publication-environment-binding.sh all PASS; pre-commit on every touched file all hooks passed; concat-changelog-fragments.sh --check exit 0; concat-adr-index.sh --check exit 0.

Type

  • ci — build native release artifacts on self-hosted Arc A380 canonical runner (ADR-1178)

Checklist

  • Commits follow Conventional Commits.
  • make format && make lint is green locally (pre-commit on every touched file).
  • Unit tests: bash scripts/ci/tests/test-check-container-build.sh (31 passed), bash scripts/release/tests/test-verify-native-release-artifacts.sh (10 passed), bash scripts/release/tests/test-publication-environment-binding.sh (all PASS), actionlint on both workflows (exit 0).
  • Docs in the same PR: docs/development/publishing.md, docs/development/release.md, docs/adr/1178-dev-container-image-publish.md, docs/research/1178-dev-container-image-publish.md.
  • SIMD/GPU: n/a (CPU-only release artifact, no kernel change); twins: n/a; new C sources: none; breaking change: none; ADR: docs/adr/1178-dev-container-image-publish.md (+ index fragment, _order.txt, regenerated docs/adr/README.md).

Bug-status hygiene (ADR-0165)

  • docs/state.md — T-PUBLISH-NATIVE-RELEASE-NOT-CONTAINERISED-2026-09-03 moved from Open to Recently closed (branch ci/release-artifacts-built-in-dev-container).

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.

Deep-dive deliverables (ADR-0108)

  • Research digest — docs/research/1178-dev-container-image-publish.md
  • Decision matrix — docs/adr/1178-dev-container-image-publish.md (Alternatives: (a) self-hosted Arc A380 runner with local image [chosen], (b) GHCR dev container pull [lost to 29.5GB blocker], (c) slim release-only stage, (d) on-the-fly build)
  • AGENTS.md invariant note — no rebase-sensitive invariants: fork-only CI workflows and release scripts, nothing upstream Netflix/vmaf touches.
  • Reproducer / smoke-test command — below.
  • CHANGELOG fragment — changelog.d/changed/release-artifacts-built-in-dev-container.md
  • Rebase note — docs/rebase-notes.md entry (ci/release-artifacts-built-in-dev-container, no rebase impact: fork-only CI)

Reproducer

actionlint .github/workflows/dev-container-publish.yml .github/workflows/supply-chain.yml
# exit 0, no output
bash scripts/ci/tests/test-check-container-build.sh | tail -1
# test-check-container-build: 31 passed, 0 failed
bash scripts/release/tests/test-verify-native-release-artifacts.sh | tail -1
# === Results: 10 passed, 0 failed ===
bash scripts/release/tests/test-publication-environment-binding.sh | tail -1
# PASS: container signatures require the exact published tag identity
bash scripts/release/concat-changelog-fragments.sh --check
# exit 0
bash scripts/docs/concat-adr-index.sh --check
# exit 0

@lusoris lusoris added this to the 1.0.0 — First release milestone Sep 5, 2026
@lusoris
lusoris force-pushed the ci/release-artifacts-built-in-dev-container branch 4 times, most recently from 9ce1828 to 774b100 Compare September 6, 2026 02:59
lusoris and others added 9 commits September 6, 2026 08:36
…nd record the runner-disk blocker (ADR-1178)

The build-artifacts job pinned ghcr.io/vmafx/vmafx-dev-mcp to
sha256:ff297e6d…, which is the local image ID of the workstation's
vmaf-dev-mcp:local build, not a GHCR manifest digest; nothing is
published under that name yet, so every release run would have failed
at container pull. Reference the :master tag instead and let the
renovate.json pinDigests rule pin :master@sha256:<digest> after the
first dev-container-publish.yml run.

Replace the research digest's estimated image sizes with measured
layer sums (libvmaf-build ≈ 29.5 GB uncompressed) and record that the
standard GitHub-hosted public-repo runner lists 14 GB of SSD, which a
job-level container: pull cannot be trimmed around; the ADR's
Consequences now carry that as an unresolved blocker with option (c)
as the fallback. Also removes a stray blank line that split the
docs/state.md Recently-closed table.
…l runner (ADR-1178)

Rework build-artifacts in supply-chain.yml to run on the Arc A380
containerised self-hosted runner ([self-hosted, linux, x64, sycl-arc],
provisioned by ADR-1177 / PR #1304) using the local canonical dev
container environment (vmaf-sycl-arc-runner:local, built FROM
vmaf-dev-mcp:local) without registry pull. Update check-container-build.sh
detector to accept canonical runner image title, configure actionlint
runner labels, and update ADR-1178, research digest, and publishing docs.
Each dropped row restates one origin/master already carries; master is the
authoritative record. Verified with scripts/ci/check-state-md-rows.sh.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the ci/release-artifacts-built-in-dev-container branch from 774b100 to 020284d Compare September 6, 2026 06:39
@lusoris
lusoris marked this pull request as ready for review September 6, 2026 08:10
@lusoris
lusoris merged commit 31507ae into master Sep 6, 2026
116 of 117 checks passed
@lusoris
lusoris deleted the ci/release-artifacts-built-in-dev-container branch September 6, 2026 08:36
@lusoris lusoris added the type:ci CI and infrastructure label Sep 7, 2026
lusoris added a commit that referenced this pull request Oct 1, 2026
…23e8f2 (#1761)

* docs: record the fork's check of the upstream defects verified on 6ec23e8f2

Fifteen defects reproduced on Netflix master were run against the fork:
three reproduced and are fixed (#1305, #1420 as a hang, #1613), two are
documented (#910, #755 and #1180), ten are not affected. The dated section
in known-upstream-bugs.md and the Confirmed not-affected rows of the state
ledger carry the evidence; Netflix 8e7a1ac4e (revert of #1476) needs
nothing from the fork.

* docs: regenerate the indexes and the citation map after rebasing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:ci CI and infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant