A strict cache mode, one cache-error class, and cross-uid-readable caches: an explicit cache that can't be read must say so (measured, 1.0.2) #486

Sequencing.

• The new error code proposed here is a public-API change, so it lands after the public-surface parity gate (a v1 public-surface parity gate #436), with the next ABI/API minor.
• Anything 1.0.x can carry without a new name ships first: wrong error classes mapped to existing codes, chtypes verify reporting when it verified nothing, and file modes.
• The discovery work (the per-schema server profile, Per-schema server profile: the server's zone and input-affecting settings, honoured by functions, DEFAULTs, literals and parsing (1.0.x, high priority) #419) is separate.

Issue-ready design for the public repository. Evidence comes from runs of the published 1.0.2 packages of Go, Python and TypeScript; Rust was read from its 1.0.2 source. Labels: measured, inferred (read from the source, not run) and unverified. The companion defect "a request for one line can load another line" is #481, and concurrent installs are #482; neither is repeated here.

1. The problem, measured

A downstream consumer mounts a cache directory and names it explicitly (CacheDir, or CHTYPES_CACHE). When that directory, or an entry in it, cannot be read, the SDK says nothing:

All rows are measured unless the column says otherwise. The positive control for every chmod was a Permission denied from ls or cat as the same uid, so the mode really took effect.

Three further measured facts shape the design:

• The four bindings treat system dirs differently.
  • Go returns the first root that has a match.
  • Python and Rust take the newest match across the cache and every system dir.
  • TypeScript reads an already-unpacked system dir not at all; it uses system dirs only to install from a pre-seed.
• The four bindings write different modes.
  • Go writes entry dirs 0700 and every file 0600.
  • Python writes verified.json 0600.
  • TypeScript and Rust follow the umask (0755/0644 at 022).
  • A cache that Go or Python wrote is unreadable to any other uid, and every reader skips it silently. That is the consumer's container case: a cache fetched on the host and read as uid 65532.
• They write different layouts. Python writes no oci-layout and no blobs. Go keeps the layer blob. TypeScript keeps no blobs. A strict check must not require what a correct writer omits.

2. What the design changes, in one table

3. Behaviour

3.1 Failure kinds

Every fault the fetch layer can meet in a cache root falls into one of these kinds:

Three rules follow from the table:

• A missing oci-layout is not K4 by itself. Python 1.0.x never writes one, and oras pre-seeds do. K4 needs the positive 0.x shape. A root that has unpacked/sha256/ is a v1 cache, whoever wrote it.
• K3 stays self-healing in both modes. That is the §1 contract, and the cross-binding interop job depends on it. Strict mode adds an error only where the default mode would silently lose an entry forever.
• The default mode keeps "unreadable is absent", as the shared-cache rule requires. It gains only a warning (§3.4), a 0.x hint, and the class fix for errors it already raises.

3.2 Strict mode and system dirs

In strict mode, the explicit cache is checked BEFORE any system dir is consulted:

• A K1, K2 or K4 fault in the cache is raised. Nothing falls through.
• A system dir that does not exist is skipped, as now; it is a default list.
• A system dir that exists but is unusable (K1/K2) is also an error in strict mode. An image whose baked layout is unreadable is misconfigured in the same way.

3.3 One root order, in both modes (C)

resolve_installed and list_installed read the cache, then every system dir in order. Among all records that answer the request, they pick the newest by (version, build). A tie goes to the earlier root.

• This is Python's and Rust's behaviour today.
• Go changes from first-root-wins.
• TypeScript starts reading unpacked records in system dirs (ensure.ts:157-169). listInstalled and verifyInstalled include them too (ensure.ts:257-283).

3.4 Warnings in default mode

A K1 or K2 fault in default mode adds one warning per path, per call:

chtypes: <path> could not be read (<errno name>); treated as not installed. Set CHTYPES_CACHE_STRICT=1 to make this an error.

The warning goes to Resolved.warnings when a result is returned. When none is, it goes into the MISSING error's message, and the CLI prints it to stderr. No new logging channel is introduced.

3.5 The error

• The new code is CHTYPES_CACHE_UNUSABLE, exit status 9. It is added to spec/fetch-v1/constants.json errors and generated into all four bindings and the fetch-v1.md §8 table.
• Why a new code. None of the ten fits:
  • the SOURCE_* codes are about a network source; Rust's mapping of local I/O to SOURCE_UNREACHABLE would make a retry loop retry a permission error;
  • the ARTIFACT_* codes are about an artifact's content;
  • UsageError is caller misuse.
• Fields (in addition to the message): path (the exact path that failed), reason (one of the table below), and os_error (the errno name, for example EACCES, when there is one).

unwritable replaces today's raw errors and UsageError on the autofetch path. That is the class fix A, and it applies in every mode.

4. The API, in four bindings

Precedence follows the existing rule: the option, then the environment, then the default.

The default, argued.

• For: an explicit directory should imply strict. Naming a directory is a statement of intent. An operator who mounts /var/cache/chtypes/v1 and gets a build from /opt/chtypes/v1 without a word has been misled, and the consumer's measurement shows exactly that.
• Against, in a patch release:
  1. fetch-v1.md §1 says unreadable is absent, and every binding's 1.0.x users run under that contract.
  2. A deployment that "works" today by falling through to a system dir would start failing on a patch upgrade, in production, with no code change of its own.
• Recommended:
  • 1.0.x (the next patch): strict is OPT-IN (option, environment or --strict). Change A (the class), C (root order), D (modes), E (truthful read-only commands), F (the 0.x hint) and the default-mode warnings ship to everyone. They change no outcome a caller depends on: every error that was raised is still raised, under a correct class.
  • 1.1.0: an EXPLICIT cache (CacheDir/cache_dir/cacheDir, or CHTYPES_CACHE) is strict by default. The implicit per-user default stays tolerant. CHTYPES_CACHE_STRICT=0 opts out. The 1.0.x warnings name the variable, so anyone who would be affected has seen it in their logs first.

5. Modes (D): readable across uids by default

Rule. Every directory and file the fetch layer creates is set to 0777 &^ umask or 0666 &^ umask. The temp directory or temp file is chmoded before its rename into place, so the final path never exists with the temp API's private mode.

• What this means in practice.
  • At the default umask 022, a cache written by uid A is readable by uid B.
  • umask 002 with a setgid root gives a group-writable cache for two writer uids.
  • umask 077 keeps a private cache.
• Go: os.MkdirTemp (0700) in unpack.go:44, and os.CreateTemp (0600) in layout.go:288 and :332. Python: tempfile.mkstemp and mkdtemp. TypeScript and Rust already comply, inferred for Rust.
• Why world-readable is safe. Everything in the cache is public, downloaded anonymously, and signature-verified. The library is re-hashed against its signed record at load (loader step 5). Readable is not writable: the umask never grants write to other.
• Rejected alternative: a mode option. The umask is the standard knob, and two bindings already follow it. A fifth spelling of the same idea would be per-language surface for no gain.

6. Truthful read-only commands (E) and the 0.x hint (F)

• chtypes verify. When it verified zero builds, it prints verified 0 builds under <root> to stderr in every mode. In strict mode it exits with CHTYPES_ARTIFACT_MISSING (exit 1) when it verified zero, so a mounted cache can be health-checked.
• resolve_installed, list_installed, verify_installed and ensure with offline never create directories or files.
  • Today TypeScript creates the layout before its offline branch (ensure.ts:294-295).
  • Go creates it once per configured system dir in resolve_installed (ensure.go:676-682).
  • Creation moves to the point where an install actually begins, so a read-only mount reads cleanly.
• The 0.x hint. MISSING from a root with the K4 shape appends: <root> holds a 0.x registry (<minor>/manifest.json); chtypes 1.x uses an OCI layout at ${XDG_CACHE_HOME:-~/.cache}/chtypes/v1 — point CHTYPES_CACHE at an empty or 1.x directory.

7. Tests

7.1 N×N in v1-cache-interop (extends scripts/fetch-v1/cache-interop.py)

Same loopback fixture server, the four real CLIs, and one new block. The job runs on Linux, so it runs real cross-uid reads.

Writers × readers × fault, 4 × 4 × 7. Each writer W runs chtypes fetch 26.8 into a fresh cache, as the runner user with umask 022. The harness then applies the fault. Each reader R runs fetch 26.8 --offline, list --offline and verify twice: once in default mode and once with --strict.

• Every row is also run with a system dir holding a readable unpacked 26.8. Default mode must answer from the system dir, with the warning. Strict mode must raise the same error as without one; there is no fall-through.
• Comparison. Each reader prints (exit status, code, reason, path). All four must print the same tuple, which is the "one answer" rule. The table goes to the job summary like the existing 4×4.
• The run user. These rows run as nobody (sudo -u nobody), never as root, because root ignores mode 000.
• Positive control, as a preflight per row. sudo -u nobody cat <the faulted path> must fail with Permission denied, or the row is reported invalid rather than passing.

Cross-uid, 4 × 4. W writes as the runner user at umask 022. R reads as nobody with fetch --offline and verify, and must succeed. Before fix D this is red exactly where W is Go or Python (measured by emulation on 1.0.2: copying each path's "other" bits onto the owner). This is the consumer's host-to-container case.

Read-only commands, 4 × 1. On a non-existent root and on a 0.x root, list --offline, verify and fetch --offline must leave the directory tree byte-for-byte unchanged. The script's existing tree_hashes does the comparison.

7.2 Per-binding unit tests

• Mode test. For each writer, after an install at umask 022, every path under unpacked/ is 0755 or 0644. At umask 077 it is 0700/0600. This proves the umask is followed, not hard-coded.
• Python version test. The K1 case on every supported Python (3.12, 3.13, 3.14) must give the same result.
  • This pins the measured divergence: Path.is_dir() re-raises EACCES before 3.14 and returns False from 3.14 (_layout.py:326).
  • The fix replaces is_dir() with an explicit os.scandir and an errno check.
• No hand-set values. Faults are produced by the real chmod and by a real writer's cache, never by a stubbed reader. Every binding's runner skips a fault row loudly when it runs as root.

8. Docs to change

• fetch-v1.md §1:
  • the K0 to K4 table;
  • strict mode;
  • the root order rule;
  • the modes rule;
  • replace "Read-only system directories are searched after the user cache" with the order and tie rule of §3.3.
• fetch-v1.md §6: --offline never writes.
• fetch-v1.md §8: CHTYPES_CACHE_UNUSABLE (generated).
• fetch-v1.md "Upgrading from 0.x": the hint, and strict's layout_0x.
• bindings-v1.md §4: the class row, and CacheUnusableError under ChtypesError in all four.
• bindings-v1.md §6: the strict_cache option in "The fetch options".
• Each binding's reference page and CHANGELOG. In 1.0.x, strict is opt-in; there is a class fix, a mode change and a verify stderr line. Announce the 1.1 default.

9. Related defects found by the same measurements (separate issues)

• Concurrent installs of one build can replace a finished install, in every binding; tracked separately, with the fix in 1.0.4.
  • TypeScript is the worst (measured, N=8). 5 of 24 TS processes failed with ARTIFACT_INCOMPATIBLE (dlopen ENOENT) or raw ENOTEMPTY, and readers from other bindings saw the entry vanish. The cause is commitStaging's unconditional rm -rf before rename (layout.ts:417-425).
  • Go, Python and Rust check for a good record, then later move aside whatever is there (inferred from the code). This narrower race was measured with fixture-sized layers in the 1.0.4 work. At production layer size no failure was observed in 32 Go and Python processes, which shows the window is small, not that it is closed.
• Go --frozen re-downloads the layer on a warm cache (measured: 3 requests, 42.5 MB, every run; Python 0, TypeScript 1).
• TypeScript fetch --lock pins only the host platform (measured). Go and Python pin all three, as fetch-v1.md §6 requires.
• Python's index.json gains a duplicate entry per install (measured: 7 or 8 for one digest after 8 racers).

🤖 Generated with Claude Code