Make attachment APIs stream-only and bound all byte collection #818

1. Background Context

Source: git-warp #818. Audited against main 94b40dac64034cd8caab9bb05efe14a0c22bd735. Template: feature; work type: type:bug.

Current scope and disposition: Start with existing asset ports and unbounded collectors. This internal safety contract must precede restored public attachment operations and completed migration.

Reported context (historical names and counts require the current disposition above):
Plan: https://github.com/git-stunts/git-warp/blob/plan/streaming-recursive-materialization/docs/plans/streaming-indexed-recursive-warp.md
Design PR: #813

2. Problem Description

Start with existing asset ports and unbounded collectors. This internal safety contract must precede restored public attachment operations and completed migration.

Historical source report; the current disposition above supersedes obsolete claims:
Node and edge content APIs can collect complete assets into Uint8Array, while large binary values can also enter through inline PropValue. The git-cas asset boundary is streaming, but eager callers discard that safety and can exhaust memory for videos or other large attachments.

2b. Proposed Solution

Start with existing asset ports and unbounded collectors. This internal safety contract must precede restored public attachment operations and completed migration.

Historical approach to reconcile:
Current architecture rules take precedence: parsing/encoding stays at adapters, domain concepts are runtime-backed, no trust casts are introduced, and tests run in Docker.

• Public and internal attachment APIs return handles, metadata, or streams.
• Remove eager complete-content methods from the v20 surface.
• Require an explicit maximum for any convenience collector outside the core API.
• Reject large inline binary values and direct callers to asset handles.
• Audit every production collectAsyncIterable() at asset, attachment, storage, proof, and migration boundaries.
• Preserve backpressure, cancellation, declared size validation, and streaming encryption frames.

2c. Alternatives considered and rejected

No additional alternatives are recorded as decided. Reject duplicate ownership, private-import escape hatches and a broken intermediate mainline; retain original alternatives below when present.

2d. Acceptance Criteria

• No public attachment method returns an entire arbitrary payload.
• No unbounded asset-to-buffer collection remains in production.
• A generated multi-gigabyte attachment drains under a small heap cap.
• A deliberately eager control fails under the same cap.
• Size mismatch, truncation, and overflow fail before excess allocation.
• Existing Retire legacy content attachment storage-plane boundaries #646 and Replace raw blob/tree ports with git-cas asset and bundle handles #737 are closed or narrowed with proof.

2e. Test Plan

Golden: Attach, replace, clear and stream node/edge content through supported package imports.

Edges: Missing owners/handles, size boundaries, history and concurrent updates.

Known failure modes: Staging failure, missing capability, truncation and cancellation publish no partial result.

Fuzz and stress: Generated large payloads, slow consumers and recursive cycles where in scope, under a Docker heap cap.

All tests and benchmarks execute in COPY-based Docker containers without host repository or Git-directory mounts. This planning audit does not claim those checks were run.

3. Prerequisites

No unsatisfied open-issue prerequisite established by this review. This is not proof that an unresolved design or external readiness condition is satisfied.

4. Scope

In: Start with existing asset ports and unbounded collectors. This internal safety contract must precede restored public attachment operations and completed migration.

Out: unrelated domain work and any expansion beyond the issue’s stated observable outcome.

Safe intermediate state: the PR builds and passes relevant checks after its listed prerequisites; existing supported behavior remains usable. Any preparatory step must be independently mergeable.

Domain review, 2026-10-01 — accepted ownership / scope correction:

Own the stream-only attachment contract consumed directly by the #902 acceptance gate as well as by #901 implementation. This direct edge records the contract, not additional sequencing beyond the existing path.

5. Why now

Maintainer ordering: memory correctness first, supported attachments next, then land eligible PRs. Preserve this card’s existing priority unless a separately recorded scope decision changes it.

6. Risks

Main risk: implementing the historical description instead of the current runtime contract. Preserve compatibility, causal/ownership invariants and bounded behavior relevant to attachments.

7. Definition of Done

The issue-specific acceptance checks pass, relevant validation evidence is attached, and the issue links the coherent PR and resulting mainline integration commit. No open item is hidden in a later repair PR.

8. Stakeholders

James Ross: maintainer, assignee and acceptance owner. Public API consumers rely on reachable node/edge attachment operations and honest ownership.

9. Related Issues

• Blocks completion of #822: Migration explicitly verifies attachment streaming and must avoid eager legacy asset collection. The common internal stream-only and bounded-collector contract must precede a completed migrator.
• Blocks completion of #901: The restored public API must ship stream-only reads, bounded inline values and cancellation/size guarantees. Land the internal stream contract before exposing its public intents; minimum consumer proof stays in Restore public node and edge content attachments after the v19 API migration #901.
• Blocks completion of #823: The final memory gate must exercise multi-gigabyte attachment streaming and the eager negative control.

Historical paths, counts, release names and shell examples in source material are evidence to reconcile, not authority to restore retired documentation or run host tests.

• Blocks completion of #902: The consumer acceptance card explicitly coordinates with Make attachment APIs stream-only and bound all byte collection #818 and checks streaming reads, cancellation, size mismatch and atomicity. Preserve this direct contract prerequisite even though Make attachment APIs stream-only and bound all byte collection #818 -> Restore public node and edge content attachments after the v19 API migration #901 -> Prove node and edge attachments through installed-package public APIs #902 already orders it. Review basis: explicit-contract.