Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@
<!-- CWL-ENTRY -->
> **Agents: read the master context FIRST.** Before any work, read [`docs/CWL-MASTER-CONTEXT.md`](docs/CWL-MASTER-CONTEXT.md) (mission · naruon-as-platform + inter-component UML · cross-cutting disciplines · conventions · roadmap · current state), the live **GitHub Project #1** <https://github.com/orgs/ContextualWisdomLab/projects/1> (work/roadmap source of truth), the full spec **ContextualWisdomLab/naruon#974**, and operate the Project per [`docs/agent-github-project-protocol.md`](docs/agent-github-project-protocol.md). The repo/Project — not any private agent memory — is the source of truth.

Materialize accepts only exact SHA-256 pins or a bounded relative `-r` include (no `.`/`..`); a lone `--require-hashes` directive is not trust evidence. See [`docs/doctoring/hourly-nvidia-nim-autofix.md`](docs/doctoring/hourly-nvidia-nim-autofix.md).
Materialize syntax may recognize a bounded relative `-r` include (no `.`/`..`); flat publication admits only standalone exact SHA-256 pins. A lone `--require-hashes` directive is not trust evidence. See [`docs/doctoring/trusted-uv-flat-include-isolation.md`](docs/doctoring/trusted-uv-flat-include-isolation.md).
Conflict-scope roots fail closed when the immediate parent directory is a symbolic link.
5 changes: 3 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Semantic Versioning where the repository publishes a release.

### Fixed

- Materialized base Python locks only when every package line is an exact SHA-256 pin or a bounded relative `-r`/`--requirement` include. A lone `--require-hashes` directive, a dotted include such as `./lock.txt`, or `-r other-hashes.txt` no longer enters the trusted build context.
- Reject relative `-r` and `--requirement` directives from generated flat base-lock publication; only standalone exact SHA-256 package closures are renamed into the trusted Docker context until a complete immutable include graph can be reconstructed and rewritten. Discovery still uses the repository-relative `_is_candidate_lock_path` predicate, and the pinned download client remains `cwl-trusted-uv-materializer/1`.
- Refused a conflict-scope repository root whose immediate parent is a symbolic link, so a swapped parent cannot redirect the canonical worktree after the last-component check (CWE-367).
- Bounded the Strix quality self-test's deterministic timeout fixtures to 3-second process and 5-second fake-sleep budgets so exact-head policy evidence completes inside the existing job limit without changing production Strix scanner timeouts, providers, credentials, or review semantics.
- Allowed commas and ASCII parentheses in the bounded Strix changed-file path policy so legal tracked Packrat fixtures can receive exact-head security analysis, while rejecting raw `..` components before normalization and keeping controls, backslashes, whitespace ambiguity, and shell punctuation fail-closed.
Expand Down Expand Up @@ -60,4 +60,5 @@ Semantic Versioning where the repository publishes a release.
- Added DiskSage operational documentation for the hourly RCA loop, bounded retry cadence, permission model, standalone and MSA reuse, verification, rollback, and APA 7 references.
- Added fast-mlsirm operational documentation for the hourly RCA loop, psychometric scientific gates, Rust ownership, bounded retry cadence, credential isolation, modular reuse, rollback, and APA 7 references.
- Documented the ordinary and conflict repair write-scope parity, ignored-path and symlink inventory, Git-control-file denial, hook suppression, explicit push destination, RED/GREEN evidence, operator response, and local-versus-protected evidence boundary.
- Documented the review-authentication boundary that excludes autonomous writer control-plane paths from review-derived file authority, its test-first Strix security evidence, exact-head coverage contract, and rollback prohibition.
- Documented the review-authentication boundary that excludes autonomous writer control-plane paths from review-derived file authority, its test-first Strix security evidence, exact-head coverage contract, and rollback prohibition.
- Documented why pip-relative include syntax cannot cross the generated flat-lock publication boundary and the exact-head regression evidence required before the policy can change.
65 changes: 65 additions & 0 deletions docs/doctoring/trusted-uv-flat-include-isolation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Flat requirements include isolation

## Status

Accepted on 2026-08-16 for the trusted base Python lock materializer.

## Buyer-facing failure

The central coverage lane publishes every accepted base lock under a generated
flat name such as `requirements-000.txt`. A source file containing
`-r requirements-other.txt` is valid pip syntax, but the include is resolved
relative to the generated output location. Copying only the referring file can
therefore produce an unusable or incorrectly bound dependency closure before a
consumer repository's tests start.

## Decision

The syntax-oriented `_is_hash_pinned` predicate may continue to recognize a
strictly bounded relative include for compatibility diagnostics. The publication
boundary is stricter: `_is_flat_materializable_lock` accepts only a non-empty set
of exact package pins carrying complete SHA-256 hashes. `base_hash_locks` first
uses `_is_candidate_lock_path` so conventional names and direct `.txt` children
of a `requirements/` directory remain eligible, then applies the standalone
publication policy before assigning generated names.

The downloader identity stays the pinned `cwl-trusted-uv-materializer/1` client
already contracted by ContextualWisdomLab/.github#939. This change does not
introduce a second User-Agent.

Relative `-r` and `--requirement` directives are rejected from flat publication
until the materializer can reconstruct the complete immutable include graph,
preserve source-directory identity, rewrite every edge to generated names, and
prove the rewritten closure before Docker receives it.

## Verification contract

The regression suite must prove all of the following on the exact PR head:

1. empty and directive-only files are rejected;
2. both relative include spellings are rejected at the publication boundary;
3. an exact SHA-256 package pin is accepted;
4. exact-base discovery excludes a referring file while retaining an
independently complete included lock;
5. a standalone hash-pinned `requirements/ci.txt` and
`service/requirements/package.txt` remain published; and
6. production statement and branch coverage, public docstrings, compilation,
redirect, output-path, workspace, downloader, and portable-runner contracts
remain at their existing gates.

## Security and operability consequences

This is a fail-closed availability and supply-chain correction. It does not
expand network access, accepted URLs, proxy behavior, redirects, package syntax,
output authority, or repository write scope. A repository using nested
requirements files receives no central lock candidate rather than a silently
relocated include. Its next action is to provide one standalone hash-locked
closure or adopt a later graph-aware materializer.

## References

The pip developers. (2026). *Requirements file format*. Python Packaging
Authority. https://pip.pypa.io/en/stable/reference/requirements-file-format/

The pip developers. (2026). *Secure installs*. Python Packaging Authority.
https://pip.pypa.io/en/stable/topics/secure-installs/
37 changes: 30 additions & 7 deletions scripts/ci/materialize_base_python_requirements.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
"https://releases.astral.sh/github/uv/releases/download/0.12.1/"
"uv-x86_64-unknown-linux-gnu.tar.gz"
)
TRUSTED_UV_DOWNLOAD_USER_AGENT = "cwl-trusted-uv-materializer/1"
TRUSTED_UV_ARCHIVE_SHA256 = (
"90b2f223fb69d19db49e117da601f64978593417988530aa733d456141b4bcbb"
)
Expand Down Expand Up @@ -87,7 +88,6 @@ def _is_candidate_lock_name(name: str) -> bool:
)



def _is_candidate_lock_path(path: pathlib.PurePosixPath) -> bool:
"""Return whether one safe tracked path can name a pip requirements lock.

Expand Down Expand Up @@ -180,6 +180,25 @@ def _is_hash_pinned(content: bytes) -> bool:
or _is_bounded_requirement_include(line)
for line in requirement_lines
)


def _is_flat_materializable_lock(content: bytes) -> bool:
"""Return whether content is one standalone exact SHA-256 lock.

The materializer renames every selected source to a generated flat file.
Relative ``-r`` and ``--requirement`` directives are therefore not portable:
their target is resolved relative to the generated output rather than the
source file. Only independent package pins can cross this publication
boundary until a complete immutable include graph is reconstructed and
rewritten.
"""
lines = _requirement_lines(content)
requirement_lines = [line for line in lines if line != "--require-hashes"]
return bool(requirement_lines) and all(
_is_fully_hash_pinned_requirement(line) for line in requirement_lines
)


def _is_fully_hash_pinned_requirement(line: str) -> bool:
"""Return whether one uv-export line is an exact package pin with SHA-256 hashes."""
fields = re.split(r"\s+(?=--hash=)", line)
Expand Down Expand Up @@ -222,12 +241,16 @@ def _download_trusted_uv_archive() -> bytes:
"""Download the fixed uv release archive through one HTTPS trust boundary."""
_install_trusted_uv_url_opener()
try:
# Keep the audited URL literal at the network sink so static analysis can
# prove that neither user data nor repository content selects a scheme,
# host, path, query, fragment, method, or request header.
with urllib.request.urlopen( # nosemgrep: python.lang.security.audit.dynamic-urllib-use-detected.dynamic-urllib-use-detected # nosec B310
# Keep the audited URL literal and static request header in this trusted
# function so neither user data nor repository content selects the
# scheme, host, path, query, fragment, method, or request header.
request = urllib.request.Request(
"https://releases.astral.sh/github/uv/releases/download/0.12.1/"
"uv-x86_64-unknown-linux-gnu.tar.gz",
headers={"User-Agent": TRUSTED_UV_DOWNLOAD_USER_AGENT},
)
with urllib.request.urlopen( # nosemgrep: python.lang.security.audit.dynamic-urllib-use-detected.dynamic-urllib-use-detected # nosec B310
request,
timeout=TRUSTED_UV_DOWNLOAD_TIMEOUT_SECONDS,
) as response:
final_url = urllib.parse.urlparse(response.geturl())
Expand Down Expand Up @@ -512,9 +535,9 @@ def base_hash_locks(repo_root: pathlib.Path, base_sha: str) -> list[tuple[str, b
regular_paths = {path for path, _candidate in regular_blobs}
locks: list[tuple[str, bytes]] = []
for path, candidate in regular_blobs:
if _is_candidate_lock_name(candidate.name):
if _is_candidate_lock_path(candidate):
content = _git(repo_root, "show", f"{base_sha}:{path}")
if _is_hash_pinned(content):
if _is_flat_materializable_lock(content):
locks.append((path, content))
elif candidate.name == "uv.lock":
if _uv_pyproject_path(path) not in regular_paths:
Expand Down
45 changes: 30 additions & 15 deletions tests/test_trusted_uv_download_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,34 +53,49 @@ def _urlopen_calls() -> list[ast.Call]:
]


def test_urlopen_receives_one_literal_https_release_url() -> None:
"""Static analysis can prove repository or user data never selects the URL."""
def test_urlopen_receives_one_static_release_request() -> None:
"""Static analysis can prove repository data never selects the request."""
calls = _urlopen_calls()

assert len(calls) == 1
assert len(calls[0].args) == 1
url_argument = calls[0].args[0]
assert isinstance(url_argument, ast.Constant)
assert isinstance(url_argument.value, str)
assert url_argument.value == _EXPECTED_URL


def test_literal_network_sink_matches_the_documented_release_constant() -> None:
"""The scanner-friendly sink literal cannot drift from the release identity."""
assert _assigned_literal("TRUSTED_UV_ARCHIVE_URL") == _EXPECTED_URL

request_argument = calls[0].args[0]
assert isinstance(request_argument, ast.Name)
assert request_argument.id == "request"

def test_downloader_never_constructs_a_dynamic_request_object() -> None:
"""The audited downloader cannot hide a dynamic URL inside ``Request``."""
request_calls = [
node
for node in ast.walk(_download_function())
if isinstance(node, ast.Call)
and isinstance(node.func, ast.Attribute)
and node.func.attr == "Request"
]
assert len(request_calls) == 1
assert len(request_calls[0].args) == 1
url_argument = request_calls[0].args[0]
assert isinstance(url_argument, ast.Constant)
assert isinstance(url_argument.value, str)
assert url_argument.value == _EXPECTED_URL

headers = next(
keyword.value
for keyword in request_calls[0].keywords
if keyword.arg == "headers"
)
assert isinstance(headers, ast.Dict)
assert len(headers.keys) == 1
assert isinstance(headers.keys[0], ast.Constant)
assert headers.keys[0].value == "User-Agent"
assert isinstance(headers.values[0], ast.Name)
assert headers.values[0].id == "TRUSTED_UV_DOWNLOAD_USER_AGENT"
assert _assigned_literal("TRUSTED_UV_DOWNLOAD_USER_AGENT") == (
"cwl-trusted-uv-materializer/1"
)

assert request_calls == []

def test_literal_network_sink_matches_the_documented_release_constant() -> None:
"""The scanner-friendly sink literal cannot drift from the release identity."""
assert _assigned_literal("TRUSTED_UV_ARCHIVE_URL") == _EXPECTED_URL


def test_literal_urlopen_sink_has_one_scoped_semgrep_suppression() -> None:
Expand Down
90 changes: 89 additions & 1 deletion tests/test_uv_redirect_boundary.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
"""Behavioral contracts for the trusted uv download redirect boundary."""
"""Behavioral contracts for trusted uv download and flat-lock boundaries."""

from __future__ import annotations

import urllib.request
from collections.abc import Iterator
from pathlib import Path

import pytest

Expand Down Expand Up @@ -65,3 +66,90 @@ def fake_install_opener(opener: object) -> None:
assert isinstance(handlers[0], urllib.request.ProxyHandler)
assert handlers[0].proxies == {}
assert isinstance(handlers[1], materializer._RejectTrustedUvRedirects)


@pytest.mark.parametrize(
("content", "expected"),
[
(b"", False),
(b"--require-hashes\n", False),
(b"-r requirements-other.txt\n", False),
(b"--requirement requirements-other.txt\n", False),
(b"demo==1 --hash=sha256:" + (b"a" * 64) + b"\n", True),
],
)
def test_flat_lock_policy_requires_a_standalone_exact_hash_closure(
content: bytes,
expected: bool,
) -> None:
"""Generated flat lock names cannot preserve source-relative includes."""
assert materializer._is_flat_materializable_lock(content) is expected


def test_base_lock_discovery_excludes_relative_include_files(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""A relative include never crosses from the exact base into flat output."""
tree = (
b"100644 blob "
+ (b"0" * 40)
+ b"\trequirements-other.txt\0"
+ b"100644 blob "
+ (b"1" * 40)
+ b"\trequirements.txt\0"
)
pinned = b"demo==1 --hash=sha256:" + (b"a" * 64) + b"\n"

def fake_git(_repo_root: Path, *args: str) -> bytes:
if args[0] == "ls-tree":
return tree
if args[0] == "show" and args[-1].endswith(":requirements-other.txt"):
return pinned
if args[0] == "show" and args[-1].endswith(":requirements.txt"):
return b"-r requirements-other.txt\n"
raise AssertionError(args)

monkeypatch.setattr(materializer, "_git", fake_git)

assert materializer.base_hash_locks(tmp_path, "a" * 40) == [
("requirements-other.txt", pinned)
]


def test_base_lock_discovery_publishes_nested_requirements_directory_locks(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Path-aware discovery still publishes standalone ``requirements/ci.txt``."""
tree = (
b"100644 blob "
+ (b"0" * 40)
+ b"\trequirements/ci.txt\0"
+ b"100644 blob "
+ (b"1" * 40)
+ b"\tservice/requirements/package.txt\0"
+ b"100644 blob "
+ (b"2" * 40)
+ b"\trequirements.txt\0"
)
ci_lock = b"ci-demo==1 --hash=sha256:" + (b"a" * 64) + b"\n"
package_lock = b"service-demo==1 --hash=sha256:" + (b"b" * 64) + b"\n"

def fake_git(_repo_root: Path, *args: str) -> bytes:
if args[0] == "ls-tree":
return tree
if args[0] == "show" and args[-1].endswith(":requirements/ci.txt"):
return ci_lock
if args[0] == "show" and args[-1].endswith(":service/requirements/package.txt"):
return package_lock
if args[0] == "show" and args[-1].endswith(":requirements.txt"):
return b"-r requirements/ci.txt\n"
raise AssertionError(args)

monkeypatch.setattr(materializer, "_git", fake_git)

assert materializer.base_hash_locks(tmp_path, "a" * 40) == [
("requirements/ci.txt", ci_lock),
("service/requirements/package.txt", package_lock),
]
Loading