Repository navigation
feat(storage): activate Postgres and Azure profiles with async agent persistence - #185
Conversation
Every storage port method may now answer with a promise, so a host can back a conversation with a remote database instead of the local file stores. The instance awaits each write before it reports: an up-report persists through a per-thread queue that a `record`, `run` or `close` drains, and a run awaits its turn boundary before the first frame, so persist-then-notify survives a store that takes time to answer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
f0e8a13 to
d2b4c51
Compare
Conversation stores now dispatch on the active structured backend, with native Postgres thread, event and turn tables beside the SQLite ones, so a Space that is rows keeps its conversation where its records live. The application awaits persistence throughout, and profile validation admits every Postgres and Azure Blob pairing the adapters implement. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Postgres integration suite borrowed the agent's substrate helpers to put a row under a Space's extension, which pointed storage at one of its consumers. It brings its own owner table instead, and the helpers keep their own proof beside them, where the concurrent-pool case belongs. A module-boundary case holds the direction: the agent module reaches into storage, and nothing goes back. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fold the activated behavior into `canvas-storage.md`: what each profile is configured with, where a Space's conversation now lives and how its writes are ordered, and which limits remain. The proposal records Phase 6 as implemented, and the docs index follows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
d2b4c51 to
58ed0a7
Compare
Reconciles the async Agenetes persistence with two upstream changes. microsoft#218 made `close()` tear the runtime down before detaching persistence and notification wiring, so a failed driver teardown can be retried. The async close keeps that order and drains the thread's queued state writes before either notification scope ends: a snapshot reported during teardown is the driver's last word, and the scoped stream now closes with the listener rather than ahead of the drain. Its new suites — Agent close retry, close-before-rehome, and the Move lifecycle cases — await the persistence surfaces they assert on, and the Move service awaits `close()` and `rehome()` inside the phase boundaries that map failures onto redacted responses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Async snapshot races, cross-namespace queue coupling, stale-cursor handling, and incomplete caller migration must be corrected.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 1
Open (5)
What changed in this PR
Activates all Postgres/Azure storage profiles and migrates Agenetes persistence and application callers to asynchronous operations.
Changes:
- Adds asynchronous conversation persistence and Postgres-backed stores.
- Enables six structured/blob profile combinations with expanded integration coverage.
- Updates application callers, tests, and storage architecture documentation.
| File | Description |
|---|---|
external/agenetes/README.md |
Documents async persistence guarantees. |
external/agenetes/packages/agenetes/src/turn-store.ts |
Makes turn-store operations async-capable. |
external/agenetes/packages/agenetes/src/thread-store.ts |
Makes thread-store operations async-capable. |
external/agenetes/packages/agenetes/src/notifications.test.ts |
Tests asynchronous notification persistence. |
external/agenetes/packages/agenetes/src/mount.test.ts |
Adapts mounting test to async creation. |
external/agenetes/packages/agenetes/src/instance.ts |
Implements async lifecycle, reads, and ordering. |
external/agenetes/packages/agenetes/src/instance.test.ts |
Migrates instance tests to async APIs. |
external/agenetes/packages/agenetes/src/instance.rehome.test.ts |
Tests asynchronous rehome and compensation. |
external/agenetes/packages/agenetes/src/instance-log.test.ts |
Tests async history, paging, and tails. |
external/agenetes/packages/agenetes/src/event-log.ts |
Serializes asynchronous event persistence. |
external/agenetes/packages/agenetes/src/event-log.test.ts |
Covers async event ordering and failures. |
docs/README.md |
Updates Phase 6 documentation status. |
docs/proposals/multi-backend-storage.md |
Records completed backend activation design. |
docs/architecture/canvas-storage.md |
Documents active Postgres/Azure architecture. |
apps/server/vitest.storage.config.ts |
Includes all remote storage tests. |
apps/server/src/modules/storage/testing.ts |
Provisions all product storage profiles. |
apps/server/src/modules/storage/storage.ts |
Activates Postgres and Azure composition. |
apps/server/src/modules/storage/profile.ts |
Marks Postgres and Azure selectable. |
apps/server/src/modules/storage/profile.test.ts |
Tests newly selectable profiles. |
apps/server/src/modules/storage/ports/blob.ts |
Updates blob-backend contract documentation. |
apps/server/src/modules/storage/module-boundaries.test.ts |
Enforces storage dependency direction. |
apps/server/src/modules/storage/backends/postgres/structured-store.ts |
Updates Postgres adapter status. |
apps/server/src/modules/storage/backends/postgres/integration.remote.test.ts |
Removes agent dependency from storage tests. |
apps/server/src/modules/storage/backends/azure/product.remote.test.ts |
Runs product tests with Azure profiles. |
apps/server/src/modules/remote_fs/interactive-view.rfs.test.ts |
Awaits Agenetes lifecycle operations. |
apps/server/src/modules/canvas/space-move.service.ts |
Awaits conversation migration operations. |
apps/server/src/modules/canvas/space-move.agents.test.ts |
Tests moves across storage profiles. |
apps/server/src/modules/canvas/canvas-search.ts |
Makes conversation search asynchronous. |
apps/server/src/modules/canvas/agent-node-ownership.test.ts |
Updates async record mocks. |
apps/server/src/modules/canvas/agent-node-edit.ts |
Awaits persisted agent records. |
apps/server/src/modules/agent/substrate-store.remote.test.ts |
Tests Postgres extension helpers. |
apps/server/src/modules/agent/conversation-title.service.ts |
Migrates title persistence dependencies. |
apps/server/src/modules/agent/conversation-title.conversion.test.ts |
Tests async title conversion. |
apps/server/src/modules/agent/agent.service.ts |
Awaits agent persistence operations. |
apps/server/src/modules/agent/agent.service.test.ts |
Updates async agent-service tests. |
apps/server/src/modules/agent/agent.route.ts |
Awaits conversation route operations. |
apps/server/src/modules/agent/agent-thread.titles.test.ts |
Updates async Agenetes mocks. |
apps/server/src/modules/agent/agent-thread.service.ts |
Migrates thread orchestration to async. |
apps/server/src/modules/agent/agent-thread.service.test.ts |
Tests async binding resolution. |
apps/server/src/modules/agent/agent-node-lifecycle.ts |
Supports async lifecycle transitions. |
apps/server/src/modules/agent/agent-node-lifecycle.test.ts |
Awaits asynchronous transition callbacks. |
apps/server/src/modules/agent/agent-node-fsm.integration.test.ts |
Updates canonical execution setup. |
apps/server/src/modules/agent/agent-node-binding.ts |
Awaits record and history checks. |
apps/server/src/modules/agent/agenetes/sqlite-stores.ts |
Clarifies synchronous SQLite implementation. |
apps/server/src/modules/agent/agenetes/postgres-stores.ts |
Adds Postgres conversation stores and paging. |
apps/server/src/modules/agent/agenetes/conversations.remote.test.ts |
Runs conversation tests on SQL profiles. |
apps/server/src/modules/agent/agenetes/conversation-stores.ts |
Dispatches persistence by active backend. |
apps/server/src/modules/agent/acp/threads.route.ts |
Awaits persisted ACP thread lookup. |
apps/server/src/modules/agent/acp/service.ts |
Awaits log metadata before execution. |
apps/server/src/modules/agent/acp/external-agent-realization.ts |
Makes external realization async. |
apps/server/src/modules/agent/acp/external-agent-realization.test.ts |
Adjusts async realization synchronization. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Test campaign against this PR — 16 findings, 3 of them highI built an independent test suite for this PR and ran it against real services: PostgreSQL 18 in Docker, a live Azure Blob account, and real model turns through the configured provider. Six storage pairings were exercised end to end, including a full server-process restart mid-test. Everything below is reproduced, not inferred. Each defect has a runnable reproduction; each is left in the tree as an Verification baseline
High1. A Postgres connection that dies mid-transaction takes the server process down
Each borrows a pool client and never attaches an Reproduced directly — with a The error path is correct: the client is released as broken and the pool recovers (pinned by a 12-consecutive-failure test). It is the unhandled event that kills the process. Realistic triggers: a Postgres restart or failover, 2.
|
Follow-up: a seventeenth finding, and a correction to the one aboveI went back to check something I had claimed but not actually verified — whether agent turns are stored properly in Postgres. Row counts and end-to-end recovery were all I had looked at. Opening the rows turned up one more defect and one error in my own testing. Turns that complete are stored correctlyVerified at row level, not by count: Contiguous ordinals, 17. A turn interrupted before its fold is recoverable exactly once, then disappears
const persisted = await turnStore.list(namespace, threadId);
if (!withTail) return persisted.map(({ turn }) => turn);
const fence = persisted[persisted.length - 1]?.seqEnd ?? 0;
return materializeHistory(persisted, await eventLog.readRecords(namespace, threadId, fence));A turn commits its Tier-2 record only when its generator returns, so a process that dies mid-turn leaves Tier-1 events with no folded turn. That is by design — But the fence is the last persisted turn's
Reproduced against real Postgres, through the HTTP API rather than by inference. The database left by a browser run on No The events are still there. Nothing will ever read them again. Why it matters. The conversation opens with the model answering a question about "the Note you created earlier" — and there is no earlier. The Note sits on the canvas with no record of what created it. No error is raised anywhere; the thread simply has a hole at the front. It is worse than an ordinary "the crash lost the last turn", because the turn is not lost at first — it comes back on recovery, the user sees it, and it vanishes silently as soon as they send their next message. Is it this PR's fault? The mechanism is pre-existing: Worth noting separately: The correctionMy browser test was restarting the server as soon as the Note appeared on the canvas — which is mid-turn, because the tool call lands while the turn is still streaming. So it never exercised a committed Tier-2 turn surviving a restart, and the comment in it claiming "Tier 2 came back" was wrong. The statement in my comment above that the suite proves a folded turn survives a restart did not hold as written. Fixed: it now waits for the model's own reply before restarting, and ends by asserting two turns with the original prompt still first — the one assertion an uncommitted tail cannot satisfy, since
Also re-checked on Suggested shape of a fix (for 17)Either make the fence per-turn coverage rather than a high-water mark — materialize any Tier-1 run not covered by some folded turn's I'm preparing fixes for the findings in both comments as a separate series of commits. 🤖 Generated with Claude Code |
…edge cases Five defects found by exercising these backends against real services. Each only exists because the profile is now genuinely remote. A dying connection took the process with it. Every place that borrows a pool client ran its transaction without an `error` listener, and pg-pool removes its own while a client is checked out, so a backend that went away mid-transaction raised an unhandled 'error' event — an uncaught exception, with no handler registered anywhere in the server. The pool-level handler `PostgresStoreContext` installs only ever sees idle clients. Each borrowed client now carries a listener for its checkout; the statement's own rejection is still what the code acts on, and the pool still recovers as before. A history page could be torn. `PostgresTurnStore.page()` issued its five statements straight on the pool while every other multi-statement operation in that file runs under the thread's advisory lock. Read committed gives each statement its own snapshot, and statements on a pool need not even reach the same connection, so a `replace()` landing between any two left the page built from two arrangements: content from after it, cursors stamped with the generation from before — rejected as stale on the very next request — or no groups at all with `hasMore: true`. The transaction body is now shared by `mutate` and a new `readLocked`. An isolation level would not have helped, there being no transaction for one to apply to. `ensure()` built the storage before refusing the profiles that need an awaited init, so a pool was already memoized from the environment by the time the caller was told to call `initStorage()` — and following that advice after fixing the URL reused the context built from the old one. It also let a missing Azure environment answer instead of the message explaining the real problem. The refusal now comes first. `AzureBlobStore.init()` checked for `closed` before its await and set `open` after, so a `close()` landing in between was undone. Re-read on the far side, through a method call so the check is a fresh read rather than the narrowing the first one leaves behind. `deleteSpace`'s local sweep had been narrowed to profiles with local blobs. That reads as a saving and is not one — with remote bytes the root was never created and `rm` with `force` is already a no-op. The one deployment the two forms differ on is records-in-a-database with bytes moved to Azure, where the pre-migration files are still on disk and would outlive the Space. The `pg` doubles in two suites gain the EventEmitter surface a real checked-out client has. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… turn dies The storage ports became asynchronous in this phase, and every existing suite backs them with synchronous in-memory stores, so nothing could tell an ordering claim from a store that happens to answer immediately. The harness added here makes each method genuinely asynchronous and releasable out of call order, which is what turned three defects up. A failed up-report wedged a thread for good. The chain linked with a bare `.then(...)`, and `.then` on a rejected promise forwards it without running the callback, so the first failed state write stopped every later snapshot being attempted — session and resume tokens dropped in silence — and left a rejection in the queue that answered every later `record`, `run` and `close` with that same stale error. `EventLog.#serialize` neutralises its predecessor for exactly this reason. The queue is now always-settled the same way and the failure moves to a map the query surface drains: reported once, to whoever next asks, then cleared. One transient blip used to cost a thread its checkpointing for the rest of its life. `replace()` and `delete()` sat beside the per-thread write queue rather than in it, so an append issued while a wholesale write was in flight ran concurrently: it resolved, handing its caller a durable `seq`, and the wholesale write then erased the entry that seq named. Both now queue. That alone does not close the path it was reachable by — `rehome` reads the source log and writes the target from that snapshot — so `rehome` also refuses a thread whose turn is still streaming, which is what its "no live handle" precondition was always asking; a threaded Job never enters the live table even though its `run()` is logged. An interrupted turn survived exactly one recovery. A turn commits its Tier-2 record only when its generator returns, and replaying the uncommitted records is how a recovered conversation still holds the turn whose tool call already changed the Space. But the fence was the last folded turn's `seqEnd`, so the moment a later turn folded past it those records became unreachable — the Note stayed on the canvas and the exchange that created it vanished, with no error. Coverage is now per turn: `firstUncoveredSeq` answers from the folded ranges alone, so a thread with everything committed reads exactly the suffix it read before, and one with a gap reaches back far enough to find it. Not fixed, and now said so in the code: reads inherit a queued write's failure. It looks like over-reach and is load bearing — turn acceptance rests on it, and neutralising it made `runAgent` announce a turn whose Tier-1 start was never written. Removing it means giving acceptance a signal of its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`realize()` single-flights per thread, but the key is the thread alone and carries no notion of admission, while `realizeOnce` decides admission after the record read. So a caller passing `turnLeaseHeld: true` — that is, `agentThreadService.invoke`, which *is* the turn holding the lease — could join a flight started by a caller without it, for instance a mode change on the same thread. That flight failed admission against this very lease and handed its rejection to everyone waiting: the turn 409'd with "Thread … is preparing or running", naming itself. A caller holding the lease is now retried once on its own terms, which skips admission as it always would have alone. Sharing is unchanged for everyone else. This became reachable in this phase. While `readRecord` was synchronous the doomed flight existed for a microtask; awaited behind a Postgres round trip it sits in the map for the length of a database read, which is exactly when a second request arrives. The suite alongside holds each step open by hand rather than counting ticks — the PR's own edit to the existing realization test had to stop doing that, which is what pointed here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pushed: fixes for ten of the seventeen findings, plus the suitesFour commits on top of the merge,
Each of the first three passes its own area's suite on its own commit, so the series bisects. Fixed
Deliberately not fixedThree, each now saying so in the code rather than silently:
Verification
All with zero skips — every defect reproduction now runs rather than sitting skipped. Prettier clean, ESLint 0 errors. Finding 17's fix was checked against the database that had actually lost a turn: same rows, same query, and the first turn with its tool calls now reads back in the place it happened. The browser run ends with two folded turns ( Two things worth saying plainlyThe browser suite was weaker than its own comment claimed. It restarted the server as soon as the Note appeared on the canvas — which is mid-turn, since the tool call lands while the turn is still streaming — so it never exercised a committed Tier-2 turn surviving a restart. That is what finding 17 came out of. It now waits for the model's reply before restarting and ends by asserting two turns with the original prompt still first, which an uncommitted tail cannot satisfy. And one full remote run showed 24 failures that were not a regression: live-account contention between parallel suites, which is finding 16. The same file passes in isolation. Worth running the live-account suites one at a time. 🤖 Generated with Claude Code |
Two reads that answered with more than they were asked about. `records(namespace)` awaited every thread's queued state write, whatever namespace it belonged to. The durable thread table is partitioned per namespace (I4.1), so a write queued for a thread elsewhere cannot change the answer — but it could hold the enumeration open for as long as it stayed parked. The queue stays keyed by the globally unique `threadId` (I4.2), like every other per-thread table here, and now carries the namespace it was queued for, so an enumeration waits for its own partition and nothing else. The rejection half of this was never reachable: the queue is deliberately kept always-settled, with the failure moved aside to `reportFailures`. `historyPage` read the fold boundary, the tail events and the persisted page as three separate reads of a log a running turn is still writing. A turn that folded in between was returned twice — as the newest persisted turn and as the active tail — so a caller saw one turn in two states at once. The boundary is read again after paging, and a page assembled across a fold is rebuilt against the settled layout; a boundary that keeps moving answers from the persisted page alone, a turn behind rather than self-contradictory. `history` never had this: it reads the persisted side first, so a later fold can only appear in the tail. Both are regression-tested on the deferred store harness, and the mount test now awaits `create` rather than asserting `not.toThrow()` on an async call, which observed nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four `agenetes.create` / `close` calls stayed fire-and-forget after the lifecycle became asynchronous. Three of them sit immediately before a `mounted.reopen()` or an `afterEach` teardown, so the test reopened storage while the close was still in flight, and any rejection surfaced as an unhandled one attributed to whichever test happened to be running. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1bf7583 to
8f68ec3
Compare
Yuge Zhang (ultmaster)
changed the title
…t them `.env.example` is where an operator configures storage, and it still said Postgres was not implemented and Azure not available — in the change that activates both. It named none of the variables the new profiles require, so the only documented way to reach them was to read the source. It now states that the two axes are independent and every pairing of an implemented record backend with an implemented blob backend is a deployment, and documents `HUABU_POSTGRES_URL`, the two required Azure variables and the optional key prefix, each with what breaks without it. Two existing entries were SQLite-specific and are now written for any record backend that keeps no Workspace folder: the unset-`HUABU_WORKSPACE` requirement, and managed mode being disk-only. Also records two operational constraints that only existed in code comments: one Server per Postgres database, because the locking is not distributed admission; and that the Azure container is the operator's to create and keep private, since deleting a Space sweeps only that Space's own keys. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Async test races, inconsistent page-limit validation, and process-wide lifecycle serialization remain unresolved.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 4
Open (5)
Resolved since last review (5)
…form holes Copilot's review of the storage-profile acceptance suite found one assertion that could pass without proving anything and three ways the harness or its prose was wrong about the code underneath. The recovery test asked the restarted Agent to quote the Note's text back. That text is also on the Space, so an Agent whose history did not survive could answer by reading the canvas in front of it. The first turn now plants a per-run codeword that is never written to the Space — asserted against both the structure the server holds and every node body, since those are served by different routes — and the question after the restart asks for it. The Note's text stays in the assertion as the weaker claim, so a regression there cannot hide behind the codeword. `stageLlmCredentials` reported readiness from two filenames. Staging `encrypted-secrets.json` without `HUABU_SECRET_KEY` makes `initializeSecretStore` fail the server's start outright, inside globalSetup and before any `test.skip` could be reached, so the common half-configured machine lost the credential-free storage test too. The key is now checked before either file is copied. Stopping the backend signalled a negative pid, which is POSIX-only; on Windows both calls threw, were swallowed, and left the backend holding the port until `restartBackend` gave up. Termination now goes through the process tree, mirroring `killTree` in `scripts/dev-child-supervisor.mjs`. The Postgres client-death case carried a comment saying it was skipped because no error listener existed, citing an untracked scratchpad note. `locked` has installed that listener and released the broken client for some time, and the test is enabled; the comment now describes the regression it holds. The acceptance note hard-wrapped its prose against the repository's one-line- per-paragraph rule, and called the Postgres/Azure pairing one this PR adds — microsoft#185 added it, this suite validates it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Migrates Agenetes thread, event, and turn persistence to async operations and activates Postgres and Azure Blob across the application. With #184's adapter foundation merged, every pairing of implemented backends becomes selectable — six of them, including Postgres records with Azure Blob bytes.
Which six. Structured records and blob bytes are independent configuration axes (proposal §6.3): records go to the structured backend, bytes go to a file system, and the two share nothing. So the profiles are the product of the two axes, not a list of blessed combinations —
sqliterecords withdiskbytes is an ordinary profile, not a special case:HUABU_STRUCTURED_BACKENDHUABU_BLOB_BACKENDdisk(default)disk(default)diskazuresqlitedisksqliteazurepostgresdiskHUABU_POSTGRES_URLpostgresazureHow to select one. Two variables, each defaulting to
disk, so an existing deployment that sets neither keeps today's behavior exactly:.env.exampledocuments every one of these, each with what breaks without it, and records two operational constraints that previously lived only in code comments: one Server per Postgres database, because the locking is not distributed admission; and that the Azure container is the operator's to create and keep private, since deleting a Space sweeps only that Space's own keys.Optional per-backend overrides:
HUABU_SQLITE_PATH(default<HUABU_DATA_DIR>/storage/sqlite/huabu.sqlite) andHUABU_BLOB_ROOT(default<HUABU_DATA_DIR>/storage/disk/blobs, and never built at all when Disk keeps the records, because the bytes then live inside the Space folder the user can see).An unknown value on either axis fails startup by name rather than falling back, and a profile that names a backend without its configuration fails the same way before any connection opens —
Postgres requires HUABU_POSTGRES_URL,Azure blobs require HUABU_AZURE_STORAGE_CONNECTION_STRING and HUABU_AZURE_BLOB_CONTAINER.What a non-Disk profile gives up. Nine product features need a Space or Workspace to be a real directory on this machine; four of them additionally need the Space's bytes to be in that directory. They are declared in
capabilities.ts, reported at startup, and refused at their own call sites with a sentence rather than a stack trace — unavailable, not emulated (proposal §6.4.2, disposition A). Structured-axis features: reveal a Space's folder, adopt Markdown dropped in from outside, choose/create/reveal a Workspace folder, the cross-Space user memory document, and user-authored Workspace skills. The four that also need Disk bytes: bundle export, bundle import, the built-in agent file tools, and reaching a Space as files over RFS — the plane external agents mount.This PR is the implementation half. The live-service and browser test campaign that came out of this work is now a separate, stacked PR — see below. What remains here is the change under review plus the unit and integration coverage for it: 67 files, +5,115/−1,145, of which ~1,620 added lines are source and only two source files exceed 100 lines (
postgres-stores.ts, new, andinstance.ts).Reconciled with
mainsince the original branch.mainmoved under this work — the Agent Team runtime was dropped (#215), and the merged foundation added host metadata, namespace-scoped notifications, SQLite turn paging, and an invocation-lifecycle restructure. Three things follow from that:updateHostMetadataandhistoryPageare asynchronous. The host-metadata merge chains onto the thread's up-report queue, so a read-modify-write that now spans awaits still observes every earlier write; a refused patch rejects to its caller alone and is never queued, so it cannot fail a laterrecord.PostgresTurnStore.page()is implemented, with anagenetes_turn_generationstable mirroring the SQLite ordinal/generation scheme. A wholesale replacement bumps the generation, so a history cursor issued against the previous arrangement is refused the same way on both SQL backends.main's newer callers: binding confirmation, Agent Node lifecycle, the conversation title service, canvas search, and the agent routes.The backend suite was running fewer profiles than it claimed.
PRODUCT_STORAGE_PROFILESgated the Postgres and Azure pairings onHUABU_TEST_POSTGRES_URL, which the retiredscripts/test-storage-backends.mjsused to set; the foundation's Testcontainers harness provides its services throughinject()instead, so nothing set it and onlydisk/diskandsqlite/diskran. The list now asks the harness, which takes the suite from 183 to 298 checks.Two reads that answered with more than they were asked about (from the Copilot review, both reproduced with a failing test before fixing):
records(namespace)awaited every thread's queued state write, whatever namespace it belonged to. The durable thread table is partitioned per namespace, so a write parked for a thread elsewhere could not change the answer but could hold the enumeration open. Queue entries now carry their namespace, and an enumeration waits for its own partition alone. The queue stays keyed by the globally uniquethreadId, like every other per-thread table.historyPageread the fold boundary, the tail events, and the persisted page as three separate reads of a log a running turn is still writing. A turn that folded in between came back twice — as the newest persisted turn and as the active tail. The boundary is now re-read after paging and the page rebuilt against the settled layout; a boundary that keeps moving answers from the persisted page alone, a turn behind rather than self-contradictory.history({ withTail })never had this, because it reads the persisted side first.Also from that review: four
agenetes.create/closecalls the async migration left fire-and-forget, three of them immediately before a storage reopen or teardown; and amounttest assertingnot.toThrow()on an async call, which observed nothing.Review comments from #184 that belonged to this half. The port and adapter headers no longer describe Azure and Postgres as an unselectable foundation, and
sqlite-stores.tsno longer claims the Agenetes ports are synchronous.canvas-storage.mdkeeps §2c's detailed record of shipped behavior and updates it for activation, adding an "Agenetes conversation persistence" subsection covering the tables, their cascade, write ordering, and cursor invalidation, plus a code-entry-points row. Every Markdown paragraph this change adds or edits is one physical line.Dependency direction.
storage/no longer imports from the agent module in either the new code or the Postgres integration suite, which had been borrowing the agent's substrate helpers to seed a row under a Space's extension; it brings its own owner table, and those helpers keep their proof beside themselves. Amodule-boundaries.test.tscase holds the direction, since the agent module is the side that reaches into storage.Subtree and Huabu changes are in separate commits, per
.github/copilot-instructions.md.Validation —
pnpm checkpasses in full on this branch:pnpm typecheck,pnpm lint(0 errors),pnpm format:check,check:i18n,check:agent-team-skills, andscripts/check-headers.mjs.pnpm test:storage-backends: 298 checks against disposable PostgreSQL 18 and Azurite — six product profiles, and conversation coverage (round-trip, host metadata, paging, rehome compensation, restart, deletion) on all four SQL pairings.Live-account Azure validation, the browser suite, and multi-Server coordination are outside this PR.
🤖 Generated with Claude Code