feat(supply-chain): verify lockfile hashes against upstream registries

[Jack Batzner (jackbatzner)]

Description

Adds a new CI check that verifies every added/changed entry in a lockfile has an integrity hash matching what the upstream registry actually publishes today. This catches lockfile-poisoning attacks where an attacker pins a known-good version number but swaps the bytes — a different attack class from G1 / proposal #1 (install-script bypass), which is about new entries with hostile lifecycle scripts rather than tampered hashes on otherwise-known entries.

Supports package-lock.json (npm v7+ packages dict, SRI hashes), Cargo.lock (crates.io sparse index, hex cksum), and requirements*.txt with --hash=sha256: pins. Out of scope (noted as future work): yarn.lock, pnpm-lock.yaml, NuGet packages.lock.json.

Pure stdlib (urllib, json, hashlib, base64, tomllib). No package installation, no shell-out beyond git show/git diff --name-only/git ls-files. Local-only run with --paths … is supported for offline testing; in CI it diffs lockfiles against the PR base ref via git show <base>:<path>, the same pattern scripts/check_install_scripts.py already uses.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update
• Maintenance (dependency updates, CI/CD, refactoring) — new CI job
• Security fix — supply-chain hardening, catches lockfile-poisoning class

Package(s) Affected

• agent-os-kernel
• agent-mesh
• agent-runtime
• agent-sre
• agent-governance
• docs / root — .github/workflows/supply-chain-check.yml extended with one new job; new files under scripts/

Checklist

• My code follows the project style guidelines (ruff check --select E,F,W --ignore E501 clean on both new files)
• I have added tests that prove my fix/feature works (81 tests in scripts/tests/test_check_lockfile_integrity.py, 250 scripts-wide all green)
• All new and existing tests pass (pytest scripts/tests/ -x -q → 250 passed)
• I have updated documentation as needed — script docstring covers usage; threat model documented inline
• I have signed the Microsoft CLA

Attribution & Prior Art

• This contribution does not contain code copied or derived from other projects without attribution
• Any external projects that inspired this design are credited in code comments or documentation
• If this PR implements functionality similar to an existing open-source project, I have listed it below

Prior art / related projects:

• npm ssri semantics for SRI parsing (W3C SRI spec, MIT) — no code copied, design referenced inline.
• Cargo sparse-index format (https://index.crates.io/<aa>/<bb>/<name>) — public crates.io spec; URL construction follows the documented bucket scheme.
• Diff-base-against-HEAD pattern matches the existing scripts/check_install_scripts.py in this repo.

AI Assistance

• I can explain every meaningful change in this PR: what it does, why, and what tradeoffs were considered
• I have run tests and verification appropriate for this change
• No part of this PR was autonomously submitted by an AI agent without my review
• I have not used AI to generate review comments on others' PRs

GitHub Copilot CLI assisted with parser scaffolding and an iterative red-team review loop (see "Adversarial review history" below). All hash logic, registry interactions, threat-model decisions, and final code were reviewed line-by-line.

IP, Patents, and Licensing

• This contribution does not implement patent-pending or patent-encumbered techniques
• This contribution does not require an NDA or licensing agreement to understand or use
• Any AI tools used have terms compatible with the MIT License

Related Issues

Part of the supply-chain hardening series (proposals #1–6); this is proposal #5 of 6. Other proposals are being implemented in parallel and do not overlap with this scope.

---

What runs in CI

New job in .github/workflows/supply-chain-check.yml:

Property	Value
Trigger	on: pull_request
Permissions	contents: read (top-level), no per-job writes
Interpolation	env-vars only, no ${{ }} script interpolation
Network	HTTPS only to fixed hosts: registry.npmjs.org, index.crates.io, pypi.org. No redirects followed.
Resource caps	--max-deps 2000 (DoS exit 2); 5 MB per registry response; 16 MB per lockfile; 64 MB per git stdout; 15 s request timeout

Sample PR experiences

Clean PR (no lockfile churn): lockfile-integrity ✓ — checked 0, ok

Legitimate axios 1.7.0 → 1.7.9 bump: lockfile-integrity ✓ — checked 1, ok

Tampered hash on otherwise-known version:

ERROR axios@1.7.9 (package-lock.json::node_modules/axios)
  integrity sha512-AAAA…== does not match registry sha512-r+JM…==

Threat model summary

Threat	Mitigation
Lockfile-poisoning (hash swap on known version)	Core check — any mismatch is an error finding
npm same-algorithm "alternatives" RCE (sha512-EVIL sha512-LEGIT)	Parser emits npm-suspicious sentinel; verify always errors
pip URL/VCS smuggling, include directives, short-attached -iURL form	pip-suspicious sentinel covers all three forms
SSRF via redirect to internal host	Custom _NoRedirect opener refuses any 3xx
SSRF via attacker-controlled scheme/hostname	Hostnames hard-pinned, scheme https only
Argv injection via base ref (--upload-pack=…, @{u}, .., leading -/.)	Strict ref grammar ^[A-Za-z0-9_/][A-Za-z0-9_./-]*$ + .. reject + length cap
Log injection via package names	_safe() strips control chars + tight name regex per ecosystem
Resource exhaustion — huge lockfile	16 MB cap, oversize files warn-and-skip
Resource exhaustion — huge git blob in history	Popen + 64 MB stdout cap; overflow kills subprocess
Resource exhaustion — many entries	--max-deps 2000 with exit 2 on overflow
Untrusted YAML / pickle / eval	None used; pure stdlib JSON + tomllib

Adversarial review history

This script went through five rounds of self-attack before being opened for review. Each round found real defects:

• R1: 3 fixes — pip URL/VCS smuggling (pip-suspicious sentinel), urllib redirect SSRF (_NoRedirect), weak ref regex
• R2: 3 fixes — npm multi-algo SRI parser/comparator asymmetry, pip include/VCS directive bypass, leading -/./.. in refs
• R3: 2 fixes — npm same-algorithm alternatives RCE, pip short-attached-arg form (-iURL, -rfoo, -efoo, -ffoo, -cfoo)
• R4: 2 fixes — unbounded fh.read() lockfile DoS, unbounded subprocess.run git stdout DoS
• R5: clean — Cargo path-traversal, TOML dup-keys, npm node_modules/ log injection, pip --hash whitespace tricks all already mitigated

Every finding above has a regression test in scripts/tests/test_check_lockfile_integrity.py. A fresh maintainer eye is still worthwhile — self-attack reaches diminishing returns.

[github-actions]

PR Review Summary

Check	Status	Details
🔍 Code Review	⚠️ Missing	No current-run comment
🛡️ Security Scan	⚠️ Missing	No current-run comment
🔄 Breaking Changes	⚠️ Missing	No current-run comment
📝 Docs Sync	⚠️ Missing	No current-run comment
🧪 Test Coverage	⚠️ Missing	No current-run comment

Verdict: ⚠️ AI review incomplete; ready for human review

AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims.

[Imran Siddique (imran-siddique)]

The CI failure is generated-file drift: policy-engine-ci.yml was edited directly to downgrade actions/checkout from v6.0.3 to v4.2.2, but this repo generates that file from a manifest. test_committed_yaml_matches_manifest catches the mismatch. Fix: apply the checkout pin change through the workflow generator and re-run --write, or revert the direct policy-engine-ci.yml edit if it was an accidental carry-over from another branch. Baraa Attabbaa (@AbuOmar) for maintainer review once CI is green.

[Jack Batzner (jackbatzner)]

Thanks Imran Siddique (@imran-siddique) — your diagnosis was right. The policy-engine-ci.yml drift was an unrelated fix that landed on main after I'd opened this PR. I merged origin/main into the branch in 08473ade, which pulled in the regenerated workflow, and Check generated workflows / inline-script-tests now both pass.

Verified locally: python scripts/ci/generate_workflows.py --check is clean against the merged tree.

Current CI on 08473ade: only Policy: Awaiting maintainer review is red, which is the expected always-fail gate until maintainer review. PR is mergeable: MERGEABLE. Tagging Baraa Attabbaa (@AbuOmar) per your note. Ready when you are.

[Imran Siddique (imran-siddique)]

Lockfile integrity check: comprehensive threat model, 5 rounds of adversarial self-review, 250 tests. Generated-workflow drift was fixed in the latest commit. Approving.