refactor(storage): route Space writes through the Phase 4 portable seam

[Yuge Zhang (ultmaster)]

Summary

• Define backend-neutral async contracts for the Space collection (membership, World identity, create/delete/rename), node records, and the ordered Space write.
• Implement minimal Disk adapters over the existing layout and its caught-error rollback path, with no filesystem WAL.
• Route the server's lifecycle, node, executor, preprocessing, event, and change writers through those ports.
• Document the Phase 4 boundary and the Disk-only writers that still block non-Disk profiles.

Why

Phase 3 established the read-side seams. The earlier Phase 4 prototype also introduced process-crash recovery and a new HTTP/SSE/web reconciliation protocol, which made it much larger than the backend abstraction it was supposed to deliver.

This revision fixes only the minimum portable write contract needed before a SQL backend can be written against it.

Port shape

StructuredStore vends two things: spaces() for the Space collection in one backend namespace, and space(id) for one Space. A SpaceHandle is the versioned record — read() and write() sit directly on it — and its other members name the durable parts the Space holds: nodes, changes, tasks, events.

Verbs mean one thing throughout: read, list, create (rejects a duplicate), put (replaces one), append, update, delete, and write for the Space's own ordered multi-part mutation. Two review passes trimmed this surface before it acquired more callers; §12.4.1–2 record what came out and why.

Behavioral notes

• A normal in-process node → record → delta failure restores the pre-operation state.
• No process-crash or power-loss recovery, idempotency, durable outbox, or distributed transaction guarantee.
• Delete opens a beginDelete session that fences repository and composed blob mutations across the blob-first cleanup.
• HTTP status/schema and SSE shapes are unchanged. Conflict details now carry the logical label or title rather than a Disk filename or directory name.
• Shared schemas, web code, and the client protocol are unchanged.

Disk-behavior corrections (§12.4.3)

A review of the branch asked what routing the writers through the port changed underneath it. Four things had changed that the port never required, and all four are reverted:

	Before this fix	Restored
Executor batch	Rebuilt the node index every batch — reads and parses every sidecar in the Space, inside the canvas mutex. Measured 1.55 ms → 19.70 ms per one-node mutation in a 1000-node Space	1.62 ms; writeNode's own probes already establish ownership at readdir cost
Broken frontmatter	Content PUT and DELETE both 500, while the lenient GET kept rendering the node — visible, unrepairable, undeletable	Recovered as readNode recovers it. The read stays strict about reachability: only ENOENT is absence
Duplicate sidecars	deleteNode threw, stranding the node in the state a user resolves by deleting, and failing the whole enclosing executor batch	Deletes the indexed representative; the orphan is still reported by the duplicate-node surfaces
Missing World	beginDelete/rename resolved World through requireWorldCanvasId(), so damage to World/ turned every ordinary delete and rename into a 500	They ask only whether this Space is the protected one. worldId() still raises

Three pieces of surface went with them, on the rule §12.4.1 applied — ownershipValidated (both production callers bypassed the branch it guarded), updateNode's retry loop (unreachable continue, and exhausting it fabricated a rev-conflict that preprocessing turns into a thrown PERSIST_FAILED), and DiskDeltaLog (no production caller; the journal row is appended inside the ordered write, so its guard never ran on the only path that writes the file).

Merged main (§12.4.4)

main brought in feat!: remove intent and sketch gesture recognition, which deletes intent episodes from the product. SpaceIntents and the Disk intent half go with them.

That left SpaceHistory holding one member. The group was justified by two kinds of past record under one noun; with intents gone it is a member whose only job is to hold one other member, so events move back onto the handle as handle.events. What §12.4.2 said stays flat is unchanged — change-review records and Tasks are not history, whatever Disk's .history/ directory happens to contain.

Validation

pnpm run check passes end to end: lint (0 errors), format:check, typecheck, every workspace's tests, i18n parity, agent-team skills, and license headers. apps/server is 110 files / 943 passed / 2 todo.

The executor-batch numbers above were measured directly, by timing 20 one-node ordered writes against a seeded Space with and without the rescan.

[matluster]

This PR is a rewrite of #77

[Yuge Zhang (ultmaster)]

Adversarial review of 69a6a4e

I found two PR contract issues that should be addressed before merge.

Confirmed findings

1. P1 — a cold-cache unreadable sidecar can be treated as absent and overwritten. The filename index silently skips unreadable Markdown files (source). readNodeStrict() then cannot resolve the existing file and returns null before its strict read (source); CAS accepts expectedRevision: null as absence (source). With a real sidecar made unreadable before the first index scan, I observed read() => null, put({ expectedRevision: null }) => ok:true, and replacement of the original body. Non-ENOENT scan failures need to propagate on the strict repository path, with a cold-start regression proving the bytes remain unchanged. The underlying lenient Disk weakness predates this PR, but it violates the strict/CAS contract introduced here.

2. P2 — preprocessing converts every repository read rejection into a cache miss. tryCacheShortCircuit catches operational backend failures as well as corruption even though the port says environmental failures reject (source). That can start fresh extraction and remote/LLM work during a storage outage and postpone the root failure. Operational failures should propagate; only an explicit typed recoverable-corruption outcome should become a miss.

Separate pre-existing data-loss bug

Deleting a nonexistent stable ID that matches another Space's directory title deletes that real Space. On both exact base 973c22ce and this PR's HEAD, deleting /api/canvas/AliasVictim returned 200; the Space then returned 404 through its real canvas-* ID and its directory was gone. This is critical but not introduced by this PR, so I filed #90.

Verification completed

Stage	Result
Automated	186 storage tests, 123 changed-caller tests, full server suite with 943 passed / 2 todo, and server typecheck passed. PR CI was green.
Curl/API	Verified Unicode writes, malformed-YAML recovery, duplicate sidecars, label conflicts, empty-body protection, structural/node CAS, ordered event batches, path sanitization, tombstones, and delete fencing with no resurrection. One hundred concurrent title races each produced one 200 and one 409.
Historical compatibility	Exact merge-base 973c22ce wrote structural state, Markdown, events, tasks, and runs; HEAD loaded and updated them. Released v0.2.2 wrote genuine canvas.json, Unicode Markdown, and events; HEAD migrated, loaded, and updated them successfully.
Chrome DevTools	Created and saved a note through the UI, hard-reloaded it, and confirmed Chinese, Greek, and emoji content persisted in both canvas and detail views. Reload requests were all 200 with no console warnings or errors.

Not verified locally: crash/power-loss recovery, multi-process or distributed behavior, non-Disk backends, unknown remote outcomes, external directory rename behavior, or additional viewport sizes. Full monorepo lint/build was not rerun locally; green PR CI covered those checks.

Adversarial review performed by OpenAI Codex.