Repository navigation
fix(container): fall back to the GitHub mirror for nv-codec-headers (ADR-1200) - #1348
Merged
Merged
Conversation
6 of 17 tasks
lusoris
force-pushed
the
fix/container-nv-codec-mirror-fallback
branch
from
September 6, 2026 12:54
db37050 to
9062ed3
Compare
lusoris
added a commit
that referenced
this pull request
Sep 6, 2026
… run (#1349) The Containerfile's own layout comment has always described compat/ as "pip-installed via the python/ shim", and the Netflix golden YUV fetch exists, in its own words, "so that `pytest python/test/` passes inside the container without any host-side manual provisioning". Nothing ever performed that install, so the command died at import with ModuleNotFoundError: No module named 'slugify', and the fixture fetch served no reachable purpose. Found while running the epic #1246 GPU smoke, which had to fall back to the CLI because the suite could not be collected. Installs python/ (package name vmaf, re-exporting compat/python-vmaf per ADR-0700) into /opt/vmaf-venv, first among the editable packages so the ai/ and vmaf-tune pins resolve last and win on shared dependencies. Adds a build-time assertion that the harness imports and the golden suite collects, because the failure mode is silent: the build stayed green for a long time while the command those fixtures exist for could not run at all. --collect-only keeps it to an import plus collection check; the suite itself is a CI gate, not a container build step. The interpreter is spelled /opt/vmaf-venv/bin/python explicitly because ENV PATH is set below this layer, so a bare python3 here would be the system interpreter. VERIFIED by a full dev-mcp build: #72 harness import OK: /build/vmaf/compat/python-vmaf/__init__.py #72 62 tests collected in 0.89s #73 naming to docker.io/library/vmaf-harness-test:local That build needed the nv-codec-headers mirror fallback from #1348 applied locally, because code.ffmpeg.org has been unreachable all day; the fallback was NOT committed here and is not part of this change. Once #1348 lands this branch builds on its own. no docs needed: dev/Containerfile is build infrastructure; the user-visible effect is that a documented command now works, and docs/development/dev-mcp.md already tells operators to run the suite in the container no state delta: closes no tracked bug id -- the gap was between the Containerfile's stated intent and its contents, not a defect in shipped code Co-authored-by: Lusoris <lusoris@pm.me> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
lusoris
marked this pull request as ready for review
September 6, 2026 14:55
lusoris
force-pushed
the
fix/container-nv-codec-mirror-fallback
branch
from
September 6, 2026 14:55
9062ed3 to
b9cd0a1
Compare
…ADR-1200) dev/Containerfile fetched nv-codec-headers from a single host, code.ffmpeg.org. On 2026-09-06 that host was unreachable for over six hours -- curl returning HTTP 000 after a 30 s timeout on every attempt -- and the layer stalled with no symptom beyond an apparent hang. The dev container is the canonical build and measurement environment under CLAUDE.md rule 15 and, since ADR-1102, the environment published artifacts come from. It could not be built at all for that window. The stated reason for the single source does not apply to the pin in use. The comment said "GitHub mirror lags so use code.ffmpeg.org", which is true of an unreleased commit but not of a TAG: NV_CODEC_HEADERS_REF is n13.1.15.0, that tag is published on both hosts, and the GitHub tarball carries the cuStreamCreateWithPriority declaration in include/ffnvcodec/dynlink_loader.h that the pin exists for -- checked, not assumed. Upstream stays first; GitHub runs only when it fails outright. Whichever archive arrives is then asserted rather than trusted: the build requires include/ffnvcodec/dynlink_cuda.h to exist and greps dynlink_loader.h for cuStreamCreateWithPriority before make install, because a fallback that silently installs the wrong headers would be worse than the outage it replaces. The two archives also unpack to different top-level directory names, so the build cds into whatever was extracted instead of the hard-coded `nv-codec-headers`. Verified end to end while code.ffmpeg.org was still down: the layer fell through to GitHub, installed the headers, and the next meson step reported `Has header "ffnvcodec/dynlink_cuda.h" : YES`. The build then proceeded into the CUDA targets that had been unreachable all day. The primary gets a short leash (--retry 1 --connect-timeout 10 --max-time 60). With a working fallback behind it, minutes on a dead host are pure latency: measured, the first draft's --retry 3 --max-time 120 took 487 s to fall through and succeed, the tightened values take 123 s, and the healthy path is a single 0.5 s fetch either way. no docs needed: dev/Containerfile is build infrastructure with no user-discoverable CLI, API or output surface; the operator-visible rationale is in ADR-1200 and docs/rebase-notes.md no state delta: an upstream host outage, not a tracked defect in this codebase Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lusoris
force-pushed
the
fix/container-nv-codec-mirror-fallback
branch
from
September 6, 2026 14:58
b9cd0a1 to
dc88b2f
Compare
3 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dev/Containerfilefetched nv-codec-headers from a single host,code.ffmpeg.org. That host has been unreachable for over six hours today —curlreturning HTTP 000 after a 30-second timeout on every attempt — and the layer simply stalled. Under CLAUDE.md rule 15 and ADR-1102 the dev container is the canonical build and measurement environment, so for that whole window it could not be built at all, with no symptom beyond an apparent hang.The stated reason for the single source does not apply to the pin in use. The comment read "GitHub mirror lags so use code.ffmpeg.org" — true of an unreleased commit, false of a tag.
NV_CODEC_HEADERS_REFisn13.1.15.0, that tag is published on both hosts, and the GitHub tarball carries thecuStreamCreateWithPrioritydeclaration ininclude/ffnvcodec/dynlink_loader.hthat the pin exists for. Checked, not assumed.What changes
include/ffnvcodec/dynlink_cuda.hand grepsdynlink_loader.hforcuStreamCreateWithPrioritybeforemake install. A fallback that silently installs the wrong headers would be worse than the outage it replaces.nv-codec-headersvsnv-codec-headers-<tag>), so the buildcds into whatever was extracted rather than the hard-coded name that would have broken on the mirror.--retry 3 --max-time 120took 487 s to fall through and succeed;--retry 1 --max-time 60takes 123 s. The healthy path is one 0.5 s fetch either way.Type
build/ci— tooling / infraChecklist
make format && make lintgreen locally —pre-commit run --filesclean on all eight touched files.meson test -C build. — no library code is touched; the change is a container build layer plus documentation..c/.cpp/.cu/.h— none added.docs/adr/_index_fragments/with the slug in_order.txt;README.mdregenerated with--writeand verified with--check.Bug-status hygiene (ADR-0165)
docs/state.mdupdated — no state delta: this is an upstream host outage, not a tracked defect in this codebase.Netflix golden-data gate (ADR-0024)
assertAlmostEqual(...)score in the Netflix golden Python tests.Deep-dive deliverables (ADR-0108)
## Alternatives consideredweighs five options, including switching to GitHub outright and vendoring the 86 KB of headers.AGENTS.mdinvariant note — recorded indocs/rebase-notes.md, where this repo keeps container-level invariants: do not collapse the fallback back to onecurl, and keep both the content assertion and thefind-basedcd.changelog.d/fixed/nv-codec-headers-mirror-fallback.md.fix/container-nv-codec-mirror-fallback — second source for nv-codec-headers (2026-09-06).Reproducer
The outage itself, still live at the time of writing:
The fallback logic exactly as committed:
Observed: 123 s total, 86247 bytes, and the tag's tree contains
cuStreamCreateWithPriority.End to end, with the primary still down:
docker build --file dev/Containerfile --target libvmaf-build -t nvmirror-test .Observed:
nv-codec-headers: code.ffmpeg.org unreachable, falling back to the GitHub mirror, thenmake: Leaving directory '/tmp/nv-codec-headers-n13.1.15.0',#41 DONE, and the next meson step reportingHas header "ffnvcodec/dynlink_cuda.h" : YES. The build then proceeded into the CUDA targets that had been unreachable all day.Known follow-ups
dev/Containerfileremain single-sourced — oneAPI, ROCm, ONNX Runtime, SVT-AV1, vvenc, AMF, FFmpeg. This PR does not claim to have audited them; it fixes the one that actually failed. A build cache or internal proxy would address the class, and is the right shape at a larger scale.