Skip to content

fix(tooling): harden repository automation boundaries - #1509

Merged
lusoris merged 6 commits into
masterfrom
fix/nolint-citation-closeout-gpu
Sep 23, 2026
Merged

fix(tooling): harden repository automation boundaries#1509
lusoris merged 6 commits into
masterfrom
fix/nolint-citation-closeout-gpu

Conversation

@lusoris

@lusoris lusoris commented Sep 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Hardens two repository automation trust boundaries:

  • Intel NEO matched-set downloads now keep credentials on the API host, bound
    metadata/output, resolve assets fail-closed, download atomically with retries,
    validate checksums and Debian archives, and delete corrupt output.
  • Python automation now launches child processes through one runtime-enforced
    boundary with executable allowlists, argument/environment/output limits,
    deadlines, process-group cleanup, canonical package identity, and typed text
    and binary results. The migration removes 63 raw launches from 38 modules and
    clears all mypy findings in the 44-file changed Python scope.

Type

  • fix — bug fix
  • perf — performance improvement
  • refactor — bounded automation migration
  • docs — operator and design documentation
  • test — regression coverage
  • build / ci — tooling / infra

Checklist

  • Commits follow Conventional Commits.
  • make format && make lint — changed files passed the full local hook
    profile; whole-tree historical debt remains tracked separately.
  • meson test -C build — no libvmaf/native runtime code changed.
  • SIMD/GPU checks — no metric or backend implementation changed.
  • New native-file license header — no native file added.
  • Breaking change — none.

Bug-status hygiene

  • no state delta: these fixes harden build/repository automation and do not
    change a tracked product bug state.

Netflix golden-data gate

  • No Netflix assertAlmostEqual(...) score was modified.

Deep-dive deliverables

  • Research digests —
    docs/research/2070-intel-neo-fetch-fail-closed.md and
    docs/research/2071-bounded-process-execution.md.
  • Decision matrix — ADR-1270 records the process-boundary alternatives;
    the NEO change is a one-way fail-closed security correction.
  • AGENTS.md invariant notes — scripts/AGENTS.md and
    scripts/lib/AGENTS.md preserve canonical process imports/package identity;
    the existing ADR-1145 NEO invariant remains applicable.
  • Reproducer / smoke-test commands — below.
  • CHANGELOG fragments —
    changelog.d/fixed/intel-neo-fetch-fail-closed.md and
    changelog.d/security/bounded-process-execution.md.
  • Rebase notes — both load-bearing boundaries are recorded in
    docs/rebase-notes.md.

Reproducers

python3 -m pytest -q dev/scripts/test_fetch_intel_neo.py
.venv/bin/python -m unittest scripts.lib.test_safe_subprocess
.venv/bin/python scripts/git-hooks/test-pre-push-mypy.py

Validation

  • NEO resolver: 14 hermetic tests passed.
  • Migrated automation: 263 tests passed, 1 skipped under system Python; the
    skipped real-mypy case passed separately in the repository venv.
  • Safe-process helper: 10 tests passed.
  • Pre-push mypy regression: 19 tests passed.
  • Changed Python scope: 0 mypy findings across 44 files; Ruff and Black clean.
  • Full pre-commit and pre-push profiles passed, including strict MkDocs,
    semgrep, governance/flavor audits, vulnerability scan, and cumulative FFmpeg
    patch replay.

Known follow-ups

This PR does not claim whole-tree cleanup. The audit still reports 1,399 active
HISS violations, and the broader assertion/tidy/NULL and application-package
process-execution queues remain explicit follow-up work.

Merge with master, 2026-09-23

This branch sat as a draft while master's zero-warning pass (#1518) landed, and
the two split the same oversized functions independently. The merge commit
resolves 44 conflict hunks across 15 files. Master's spelling wins wherever the
two are equivalent; where master changed behaviour, master's behaviour is kept:

  • agent-eligibility-precheck.py keeps master's fail-closed verdicts — a
    failed gh search, or no gh on PATH, blocks dispatch — and catches
    safe_subprocess.CommandFailed instead of subprocess.CalledProcessError,
    because backlog_tracker no longer calls subprocess directly. Its test file
    is master's fail-closed one, not this branch's fail-soft one.
  • hw_encoder_corpus.py keeps master's per-quality failure accounting and
    non-zero exit, and gains this branch's resolved-executable boundary.
  • lint-configured.py keeps master's corrected Cppcheck POSIX model and
    --warnings-as-errors=*; the new model-generator call is routed through the
    bounded runner.
  • test_git_fixture_isolation.py keeps master's pelorus-sync fixture case.
  • ffmpeg_patch_stack.py keeps master's finer split with this branch's
    CommandTimedOut.
  • cross_backend_vif_diff.py, cross_backend_parity_gate.py,
    scorecard_gate.py and fetch-intel-neo.py take this branch's rewrite whole:
    master changed nothing in them beyond the same size split.

One gap is deliberately left open: scripts/ci/write_cppcheck_posix_model.py
arrives with master and still calls subprocess directly. Converting it is a
follow-up so this merge changes nothing master just gated on.

Generated files (CHANGELOG.md, the ADR index, docs/adr/by-tag/, the mkdocs
ADR nav block) were regenerated with their scripts rather than hand-resolved.

@github-actions github-actions Bot added the type:bug Something isn't working label Sep 20, 2026
@lusoris lusoris changed the title fix(dev): harden Intel NEO release downloads fix(tooling): harden repository automation boundaries Sep 20, 2026
@lusoris
lusoris marked this pull request as ready for review September 22, 2026 23:39
@lusoris
lusoris force-pushed the fix/nolint-citation-closeout-gpu branch from 3c5efc4 to 37a3b9a Compare September 23, 2026 04:47
@lusoris lusoris mentioned this pull request Sep 23, 2026
Open
6 tasks
@lusoris
test(tooling): reconcile the subprocess tests with master's newer cases
e6c5263 
Master's zero-warning sweep (#1518) reworked files this branch also
rewrote. Neither side is wholly right, so each is resolved on its own
terms rather than by picking a side.

test_lint_configured.py — master's version is kept. It carries fixtures
and a test this branch never had (the model-fixture install, the
CPPCHECK_FILESDIR stub, and clang-tidy warning promotion), and taking the
branch's file would have dropped them. The branch's contribution, routing
the fixture's own command runs through the bounded wrapper, is ported onto
it instead, and the Make sandbox now stages scripts/lib/safe_subprocess.py
so the copied driver can import it.

test_cppcheck_posix_model.py — `import subprocess` comes back. This branch
moved the test's own runs onto the wrapper and dropped the import with
them, but master's new case mocks write_cppcheck_posix_model.py's
subprocess.run and builds a CompletedProcess for it. That module is
outside this branch's adoption set, so the mock target is still subprocess.

test_agent_eligibility_precheck.py — the tracker raises CommandFailed now
rather than CalledProcessError, so the mock's side_effect follows. The
contract it feeds is unchanged: a gh query that could not run still blocks
dispatch. The branch's end-to-end CLI harness is kept and re-pointed at
that same contract; it previously asserted the opposite, which would have
turned two fail-closed dispatch guards into fail-open ones.

hw_encoder_corpus.py — the usage example moves off .workingdir2/, retired
while this branch was open, onto .corpus/ as master spells it.
@lusoris
lusoris force-pushed the fix/nolint-citation-closeout-gpu branch from 37a3b9a to e6c5263 Compare September 23, 2026 04:59
@lusoris
fix(tooling): size three subprocess bounds to the work they actually …
62f2a59 
…wrap

Three CI legs went red on this branch's own bounds rather than on anything
the bounds were meant to catch. A bound that fails legitimate work is not a
safety property, it is an outage, so each is sized against a measurement or
against the job's own ceiling and says which.

check-win64-stack-alignment.py: the whole-library `objdump -d` inherited the
wrapper's 4 MiB default and Windows MinGW64 died on
`CommandOutputLimitExceeded: command exceeded 4194304 captured bytes`.
Measured on three local x86-64 builds: a 2.2-2.7 MiB libvmaf.so disassembles
to 7.5-8.0 MiB in about 90 ms, already twice the default, and the artifact
this check runs against carries more of the matrix. Raised to 256 MiB, about
30x the measured size -- still a real ceiling on a runaway objdump. The 60 s
timeout is untouched; the measured run is three orders of magnitude inside it.

tidy-ratchet.py: `measurement/output failed: command timed out after 600s:
clang-tidy`. Before the wrapper this call had no timeout at all and the Tidy
Ratchet job completed inside its own 45-minute budget, so no legitimate
translation unit can be slower than that job allows, and 600 s was under a
quarter of it. Raised to 1800 s, which still fails a hung clang-tidy well
before the job ceiling and cannot be the cause of a failure the unbounded
version would not also have had. Labelled in the source as a ceiling, not a
measurement -- the slowest TU has not been timed.

githooks/tests/test_install.py: `git worktree remove` failed with "contains
modified or untracked files". The installer now imports
scripts.lib.safe_subprocess, so running it inside a worktree leaves
scripts/lib/__pycache__/ there; the fixture repository's .gitignore carried
only `.claude/` and `.workingdir`, where the real one has `__pycache__/` and
`*.py[cod]` at lines 23 and 13. The fixture was less faithful than the
repository it stands in for. Mirroring those two lines fixes it without
--force, which would have hidden the side effect instead. 7 tests pass.
@lusoris
lusoris merged commit 4a1e466 into master Sep 23, 2026
82 checks passed
@lusoris
lusoris deleted the fix/nolint-citation-closeout-gpu branch September 23, 2026 07:28
@lusoris lusoris mentioned this pull request Oct 4, 2026
Merged
5 of 9 tasks
lusoris added a commit that referenced this pull request Oct 4, 2026
@lusoris
fix(dev): exit non-zero again when a hardware-corpus quality point fa…
687b794 
…ils (#1982)

* fix(dev): exit non-zero again when a hardware-corpus quality point fails

scripts/dev/hw_encoder_corpus.py exited 0 when an encode, decode or
score failed or a quality point produced no canonical-6 row, so a failed
run looked like a complete corpus. #1518 added the non-zero exit,
main(argv) and its test; #1509, merged after it from an older base,
rewrote main() for executable resolution and dropped both. No CI job ran
the test, which failed every case.

main(argv) now returns 1 when any point fails and keeps the rows of the
points that succeeded. encode_hw is split into helpers (HISS-04 baseline
445 to 444) and builds the same commands. The test stubs ffmpeg and the
vmaf binary as executables and gains a one-failed-point and a
non-executable-binary control.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

type:bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@lusoris