Skip to content

ci(dev): make the container say which source it was built from (ADR-1195) - #1337

Merged
lusoris merged 3 commits into
masterfrom
ci/container-source-guard
Sep 6, 2026
Merged

lusoris merged 3 commits into
masterfrom
ci/container-source-guard

Conversation

@lusoris

@lusoris lusoris commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Summary

CLAUDE.md rule 15 says to rebuild the dev container when "its image predates the last master sync". Time cannot answer that question. A build run against a checkout that is behind master produces an image newer than every commit in the repository and missing exactly the work it was rebuilt for.

That happened today, on the path this campaign depends on. The container was rebuilt specifically to pick up the GPU default-model fixes (#1307, #1312, #1324) so the epic #1246 GPU smoke could run against them. The build succeeded and the image was the newest thing on disk. It contained none of the three — the build context was 28 commits behind origin/master — and it was caught only because a test file added by one of those PRs was missing. Had the smoke run instead, it would have reported green numbers for code that was not in the image, and those numbers would have been cited as a retrain gate.

ADR-1102's marker answers "did a container build this?". It cannot answer "which code was in that container?". This adds the second answer.

Type

  • build / ci — tooling / infra

Checklist

  • Commits follow Conventional Commits (the commit-msg hook enforces this).
  • make format && make lint is green locally. — pre-commit run --files clean on all 15 touched files; shellcheck clean on both new scripts; actionlint clean on rule-enforcement.yml.
  • Unit tests pass — bash scripts/ci/tests/test-check-container-source.sh: 8/8 assertions, output pasted under "Reproducer".
  • If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2. — no SIMD or GPU code path is touched.
  • If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap. — no feature extractor is touched.
  • If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header. — no C/C++ source is added; the new files are two shell scripts and a test.
  • If this is a breaking change, the commit message uses ! or BREAKING CHANGE:. — not breaking. docker compose build keeps working; it simply records source_rev=unknown, which the checker reports as unverifiable rather than as a pass.
  • If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/<NNNN-slug>.md and the slug is appended to _order.txt. — number allocated with scripts/adr/next-free.sh --claim; README.md regenerated via scripts/docs/concat-adr-index.sh --write and verified with --check.

Bug-status hygiene (ADR-0165)

  • docs/state.md updated — T-DEV-CONTAINER-STALE-SOURCE-UNDETECTABLE-2026-09-06 added to Recently closed with the reproduction, the reason timestamps cannot detect it, and the verification command.

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: trivial. The failure was observed directly rather than researched; the analysis is in ADR-1195's Context.
  • Decision matrix — ADR-1195 ## Alternatives considered weighs five options, including the timestamp comparison that rule 15's current wording implies — which is rejected because it returns "current" for precisely this failure.
  • AGENTS.md invariant note — no rebase-sensitive invariants: dev/, scripts/dev/ and scripts/ci/tests/ are fork-only with no upstream counterpart. The one invariant a rebase could break (the marker must stay in the final stage) is recorded in docs/rebase-notes.md, where this repo keeps container-level ones.
  • Reproducer / smoke-test command — pasted below.
  • CHANGELOG fragment — changelog.d/added/adr-1195-container-source-guard.md; CHANGELOG.md regenerated with --write and verified with --check.
  • Rebase note — docs/rebase-notes.md, section ci/container-source-guard — record the container's source revision (2026-09-06).

Reproducer

bash scripts/ci/tests/test-check-container-source.sh

Actual output on this workstation:

ok   a checkout level with the ref may build (rc=0)
ok   a checkout behind the ref is refused (rc=1)
ok   a checkout ahead of the ref may build (rc=0)
ok   uncommitted changes warn but do not block (rc=0)
ok   an unresolvable ref is rc=2 (rc=2)
ok   outside a git repo is rc=2 (rc=2)
ok   an unknown argument is rejected (rc=2)
ok   an image without the marker is rc=2 (rc=2)
ok   docker-backed cases ran
all check-container-source cases passed (8 assertions)

And against the live checkout:

bash scripts/dev/check-container-source.sh --pre-build
# check-container-source: OK — HEAD cd52f2670 matches origin/master; safe to build.

Known follow-ups

  • Existing images predate the marker. Any container built before this merges has no /etc/vmafx-dev-source and reports rc=2 (cannot verify) rather than rc=0. That is the intended reading — it is not a regression, and one rebuild through dev/scripts/container-build.sh clears it.
  • source_rev=unknown remains reachable for anyone invoking docker compose build directly, because compose cannot run git. The checker reports it honestly rather than guessing; the wrapper is the documented path.

@lusoris lusoris added this to the 1.0.0 — First release milestone Sep 5, 2026
lusoris added a commit that referenced this pull request Sep 5, 2026
…ouch gate)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Sep 6, 2026
…ouch gate)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the ci/container-source-guard branch from 7e9936f to a0df85d Compare September 6, 2026 00:00
lusoris added a commit that referenced this pull request Sep 6, 2026
…ouch gate)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the ci/container-source-guard branch from a0df85d to 0f8ab06 Compare September 6, 2026 03:03
lusoris and others added 3 commits September 6, 2026 05:34
…195)

CLAUDE.md rule 15 says to rebuild the container when "its image predates the
last master sync". That is stated in terms of time, and time cannot answer the
question. A build run against a checkout that is behind master produces an
image newer than every commit in the repository and missing exactly the work
it was rebuilt for.

That happened today. The container was rebuilt specifically to pick up the GPU
default-model fixes (#1307, #1312, #1324) so the epic #1246 GPU smoke could run
against them. The build succeeded, the image was the newest thing on disk, and
it contained none of the three: the build context was 28 commits behind
origin/master. It was caught only because a test file added by one of those PRs
was missing. Had the smoke run instead, it would have reported green numbers
for code that was not in the image, and those numbers would have been cited as
a retrain gate.

ADR-1102's marker answers "did a container build this?". It cannot answer
"which code was in that container?", and that second question is the one that
was wrong.

dev/Containerfile now records /etc/vmafx-dev-source (source_rev, source_ref,
source_repo) from a VMAFX_SOURCE_REV build argument supplied by
dev/docker-compose.yml. It is written in the LAST stage, deliberately away from
the ADR-1102 marker in the first: the first stage is reused by every rebuild,
so a marker there would report the revision of whichever build first populated
the layer cache -- authoritative and stale, which is worse than absent.

scripts/dev/check-container-source.sh answers it in both directions.
--pre-build refuses a checkout that is behind the reference and lists the
commits under baked-in paths the image would be missing; --image reads the
marker out of an existing image and reports current, stale (naming what is
missing), or unverifiable. A build that never received the argument records
`unknown`, and `unknown` is reported as "cannot verify", not as a pass: an
image that cannot say what it holds is not evidence.

dev/scripts/container-build.sh makes the correct path the easy one -- check,
build with the verified revision, re-verify the result. --allow-behind exists
for local experiments and says so loudly.

Verified by scripts/ci/tests/test-check-container-source.sh: 8 assertions,
hermetic apart from one Docker case that skips when no daemon is reachable.
The stale-context case reproduces today's shape; ahead-of-master is
deliberately allowed, since a feature branch legitimately leads master.

no digest needed: trivial

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ouch gate)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each dropped row restates one origin/master already carries; master is the
authoritative record. Verified with scripts/ci/check-state-md-rows.sh.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the ci/container-source-guard branch from 0f8ab06 to 194d43b Compare September 6, 2026 03:34
@lusoris
lusoris marked this pull request as ready for review September 6, 2026 03:34
@lusoris
lusoris merged commit 9cec679 into master Sep 6, 2026
116 of 118 checks passed
@lusoris
lusoris deleted the ci/container-source-guard branch September 6, 2026 04:03
@lusoris lusoris added the type:ci CI and infrastructure label Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:ci CI and infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant