fix(runtime): a self-hosted restart reads the kernel's own sys_metadata back into the registry

[objectstack-fleet]

Fixes #20071
Clause-②: no

What was wrong

createStandaloneStack (packages/runtime/src/standalone-stack.ts) is the composition behind os dev / os serve / os start. It stamps environmentId: 'env_local' (or whatever OS_ENVIRONMENT_ID names), and it built new ObjectQLPlugin({ environmentId, runPlatformMigrations }) with no hydrateMetadataFromDb.

ObjectQLPlugin.start() hydrates only if (this.environmentId === undefined || this.hydrateMetadataFromDb) (packages/objectql/src/plugin.ts:767). So every self-hosted boot:

• skipped reading sys_metadata;
• logged "Project kernel — skipping sys_metadata hydration (metadata sourced from artifact)".

An object published at runtime then answered 404 OBJECT_NOT_FOUND on the data API after a restart, while its row was still stored. This is the same deduction that the [#9380] note in the same file records for runPlatformMigrations.

The fix: a declaration, not a deduction

• createStandaloneStack constructs new ObjectQLPlugin({ environmentId, runPlatformMigrations: cfg.runPlatformMigrations ?? true, hydrateMetadataFromDb: true }). The [#20071] comment at that line carries the caution check below.
• StandaloneStackConfigSchema and the published StandaloneStackConfig type are unchanged.
• The only other edit in the file: The three kernel:ready migrations in assembleMetadataProtocol never arm on a self-hosted boot — the standalone stack stamps environmentId = 'proj_local', and the gate asks for undefined #9380's docblock pointer "line ~567 below" now reads "createStandaloneStack below".
• packages/objectql/** and packages/spec/** are untouched.

Opt-out: none. No caller of createStandaloneStack can make either clause of the caution false; adding a key later is its own card, declared Clause-②: yes (widening).

The plugin's caution, clause by clause

The caution is on ObjectQLPluginOptions.hydrateMetadataFromDb (plugin.ts:174-177): "Set this ONLY when the kernel's registry is per-instance isolated AND sys_metadata lives on the kernel's own local driver (no control-plane proxy)". Both clauses are properties of createStandaloneStack itself, not of its caller.

1. Per-instance isolated registry: holds.
   • The function constructs a fresh ObjectQLPlugin and passes no ql, so init() runs this.ql = new ObjectQL(hostCtx) (plugin.ts:397).
   • Each ObjectQL owns its registry: private _registry: SchemaRegistry = new SchemaRegistry(); (engine.ts:3061, whose comment says "Each engine now owns its registry so kernels are fully isolated").
   • SchemaRegistry (registry.ts:1819) has no static members.
2. sys_metadata on the kernel's own local driver: holds.
   • SysMetadataObject (packages/metadata-core/src/objects/sys-metadata.object.ts:17) declares no datasource, so it routes to the default driver.
   • The function composes exactly one datasource, DefaultDatasourcePlugin as default (ADR-0062 D1: "every unbound object routes to it").
   • Every databaseDriver kind the function dispatches (memory, sqlite, sqlite-wasm, postgres, mysql, mongodb, turso) is a direct driver, never a control-plane proxy.
   • The hydration read is this.engine.find('sys_metadata', { where: { state: 'active', organization_id: null } }) (loadMetaFromDb in packages/metadata-protocol/src/protocol.ts), through that same engine.

Stop valve: every caller of createStandaloneStack, measured

git grep createStandaloneStack at a4ca69a9 finds three production callers, plus tests and fixtures:

caller	what it is	sys_metadata	hydrates after this PR
packages/cli/src/commands/serve.ts:2740	os serve / os dev / os start with a config	the stack's own default driver	yes (declared)
packages/runtime/src/default-host.ts:163 (createDefaultHostConfig)	artifact-only boot	same	yes (declared)
packages/cli/src/utils/schema-migrate.ts:303 (bootSchemaStack)	the one-shot funnel for 13 os migrate * / os meta * commands	same	yes (declared), see below
runtime and cli tests and fixtures	memory or sqlite files	same	yes (declared)

No caller has a proxied or non-local sys_metadata, so the stop valve does not fire.

Outside this repository:

• An org-wide code search for createStandaloneStack returns only objectstack-ai/duly: tests and seed scripts of an ordinary standalone app on its own database.
• objectstack-ai/cloud is NOT MEASURED. Control leg: the same search for hydrateMetadataFromDb returns 0 hits outside this repository, although the plugin's own docblock says the cloud single-env tenant runtime sets that option. So the search cannot see that repository.
• Either way, both caution clauses hold inside the function, so an unmeasured embedder gets the same composition.

Why bootSchemaStack hydrates too

Ruling ① lets a read-only one-shot boot turn hydration off "if it genuinely does not need it". Measured, the one-shots need it, or are neutral:

• os migrate plan: its unmanaged-table sweep (packages/cli/src/utils/unmanaged-tables.ts header) says the question "is only answerable when the composed object set actually MIRRORS what this deployment's os serve boot registers". After this PR, os serve registers runtime-authored objects.
• os migrate files-to-references: it refuses an empty scan because "this command's verdict is what later authorises irreversible behaviour". Without hydration it would silently skip the file fields of runtime-authored objects.
• The read writes nothing: loadMetaFromDb is engine.find plus registry registration plus log lines. A boot that defers DDL still defers the Phase-3 tables of what it hydrated.
• Measured at c945f282: duplicates.integration.test.ts (boot included, database byte-identical after the run), platform-migrations-arming.integration.test.ts and the six schema-migrate* / unmanaged-tables integration suites all pass. The stack's construction is the same true at the current head.

Ruling ④: the false log line

"Project kernel — skipping sys_metadata hydration" is now unreachable on this stack. It sits only in the else of the gate above, and hydrateMetadataFromDb is a literal true that no caller can change. The new unit test pins the flag under every way an environment id is stamped.

Observed directly:

• on the unfixed source, the unit test's two boots each printed that line once;
• on the fix, the second boot prints Metadata restored from database to SchemaRegistry {"loaded":2,"errors":0,"invalid":0} and no skip line.

No edit to packages/objectql was needed.

Reach beyond one object (mechanism assumption 6)

packages/runtime/src/standalone-stack-hydrate-metadata.test.ts boots the real stack twice on one SQLite file:

• Boot 1 writes an env-wide object row and an env-wide app row through the protocol, and inserts one record.
• Boot 2 has both rows in the registry, and the record reads back through the engine.

The app is not in the registry on the boot that wrote it. On this composition, the protocol's write-through for non-object types returns early on a kernel with an environment id (see Acceptance notes). So boot hydration is the only thing that puts it there, which makes it a sharp second leg.

Org-scoped rows are deliberately not asserted. loadMetaFromDb reads organization_id IS NULL only (ADR-0005: per-org overlays are served on demand).

Diagnostics hydration now surfaces at boot, reported and not suppressed. loadMetaFromDb and its reportUnhydratableOrgScopedRows (#6190) now run on every standalone boot:

• a stored row that cannot register prints [Protocol] [metadata_field_type_refused] … at error, or [Protocol] Failed to hydrate TYPE/NAME: … / [Protocol] [metadata_spec_invalid] … at warn;
• org-scoped rows of types that are not per-org overridable get one aggregated line.

None of these fired in any suite run here. Against a real install's sys_metadata they are NOT MEASURED; the changeset names these lines for upgraders.

The pin

packages/cli/test/package-restart-acceptance.integration.test.ts: probe 2's it.fails is promoted to a plain it, and the file's header now describes the fixed state.

A log-line assertion I added beside it was removed in this PR. The ablation leg showed it stays green on the unfixed build, because os serve does not print that INFO line at the pin's log level.

Tests (head 46fd71b1)

• The pin: pnpm --filter @objectstack/cli exec vitest run --project integration --maxWorkers=2 test/package-restart-acceptance.integration.test.ts gives Tests 5 passed (5). ablation-dist-preflight shows the runtime dist carries the literal.

• Unit test: pnpm --filter @objectstack/runtime exec vitest run --maxWorkers=2 src/standalone-stack-hydrate-metadata.test.ts gives Tests 2 passed (2).

• Runtime suite: pnpm --filter @objectstack/runtime test gives Test Files 279 passed (279), Tests 3906 passed | 1 skipped (3907).

• Typecheck: pnpm --filter @objectstack/runtime typecheck exits 0, with check:test-typecheck: OK … 27 file(s) / 191 error(s) / 69 pinned signature(s) (ledger unchanged).

• Lint: full pnpm lint (eslint . --no-inline-config) exits 0.

• Gates: all 60 commands that node scripts/pm/dispatch-gates.mjs --commands derives at 46fd71b1 exit 0. The --ran reconciliation reads "60 derived, 60 run, 0 NOT-MEASURED, 0 UNRUN".

• At c945f282, before this revision:

  • the CLI unit project: Test Files 224 passed (224), Tests 3158 passed (3158);
  • pnpm --filter @objectstack/cli typecheck: exit 0;
  • CLI --project integration on 10 files (the pin, duplicates, meta.stored-flow-resolution, platform-migrations-arming, the five schema-migrate* files and unmanaged-tables): Test Files 10 passed (10), Tests 38 passed (38).

  This revision touches no CLI file, and the rest of the CLI integration project is declared to CI.

Ablation at 46fd71b1, from committed state. The mutation flipped the literal's true to false through node scripts/ablation-replace.mjs, anchored on the construction line.

• Mutation landed: anchor x1 to x0, blob b10937d6 to dc0eab77, and on-disk counts true=0 false=1.
• Reached dist: after rebuilding @objectstack/runtime, ablation-dist-preflight reports the true spelling absent from all 6 dist files and the flipped spelling present in 2.
• Unit test: 2 failed. The restart row fails at "hydr_widget is registered after the restart: expected undefined", the registry check just before its data read.
• Pin: 1 failed | 4 passed. Probe 2 fails with "GET /data/leave_request after a restart: {"error":"Object 'leave_request' is not registered","code":"OBJECT_NOT_FOUND"} … expected 404 to be 200".
• Restore:
  • blob b10937d6 == HEAD, and git status --porcelain is empty;
  • runtime rebuilt: the literal is present in 2 dist files and false absent from all 6;
  • the unit test gives 2 passed and the pin 5 passed.

Acceptance notes (observations, not filed)

• Same deduction, non-object write-through: packages/metadata-protocol/src/protocol.ts applyRegistryWriteThrough returns early for every non-object type when this.environmentId !== undefined. That is the same deduction class, one package over. On a standalone kernel, a runtime-saved app is absent from the registry on the boot that wrote it, until a listing or the next boot hydrates it; the new unit test's boot 1 measured this. No user-visible failure was measured, so it is recorded here, not filed.
• Comment drift: packages/objectql/src/plugin.ts (the Phase-2 bridge comment) and loadMetaFromDb's comment still call SchemaRegistry a process-wide singleton, while engine.ts:3054-3061 says each engine now owns its own. Both files are read-only for this lane.

Relations

#17676 remains open: that card belongs to the engine seat, and this PR delivers the runtime half that its acceptance pin (PR #20069) waits on.

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/runtime, touching 4 documentable anchor(s).

8 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

• content/docs/api/environment-routing.mdx (via createStandaloneStack (symbol, a top-level function))
• content/docs/api/wire-format.mdx (via /api/v1/meta/* (route, a path literal in a comment in createStandaloneStack))
• content/docs/data-modeling/drivers.mdx (via createStandaloneStack (symbol, a top-level function), env_local (literal, a string literal in a comment in StandaloneStackConfigSchema; a string literal in a comment in createStandaloneStack))
• content/docs/deployment/cli.mdx (via env_local (literal, a string literal in a comment in StandaloneStackConfigSchema; a string literal in a comment in createStandaloneStack))
• content/docs/deployment/single-project-mode.mdx (via createStandaloneStack (symbol, a top-level function), env_local (literal, a string literal in a comment in StandaloneStackConfigSchema; a string literal in a comment in createStandaloneStack))
• content/docs/deployment/validating-metadata.mdx (via /api/v1/meta/* (route, a path literal in a comment in createStandaloneStack))
• content/docs/kernel/cluster.mdx (via /api/v1/meta/* (route, a path literal in a comment in createStandaloneStack))
• content/docs/plugins/index.mdx (via createStandaloneStack (symbol, a top-level function))

⛔ 1 release-owned page(s) also name something this change touched. These are read-only:

• content/docs/releases/v15.mdx (via /api/v1/meta/* (route, a path literal in a comment in createStandaloneStack))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see

• the SDK route bridge reached 60 of 215 client-bound route-ledger rows — the other 155 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 155: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 100 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
• a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 26 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json d624002eb907a92aa9e72d463c2e3ab78697d85c → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 42ab9d28765db80526f139a2833a0a1959de6343 — the merge of head 46fd71b1e605a3cb5434b36c40169db538eb5f4c into base d624002eb907a92aa9e72d463c2e3ab78697d85c, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 42ab9d28765db80526f139a2833a0a1959de6343 && git checkout 42ab9d28765db80526f139a2833a0a1959de6343
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin d624002eb907a92aa9e72d463c2e3ab78697d85c 46fd71b1e605a3cb5434b36c40169db538eb5f4c && git checkout -B drift-repro d624002eb907a92aa9e72d463c2e3ab78697d85c && git merge --no-ff 46fd71b1e605a3cb5434b36c40169db538eb5f4c

node scripts/docs-audit/affected-docs.mjs --json d624002eb907a92aa9e72d463c2e3ab78697d85c

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs d624002eb907a92aa9e72d463c2e3ab78697d85c → pass the list as
args.docs, on the commit named under Which tree this was computed on.

[objectstack-fleet]

Face review (post-hoc, released)

Served-tier: CONTRACT_REVIEW_TIER
Merge-sha: fa00ebf44759b7fcfb0a1e9f55256265068399a8
Local-runs: none

Scope: the changeset (and named docs) of PR #20100, released; this review cannot block anything and exists to find false released prose.

Sentence verdicts

(.changeset/20071-standalone-hydration.md at the merge; no docs page in scope)

1. "An object created and published at runtime on a self-hosted install ... answered 404 OBJECT_NOT_FOUND on the data API after the next restart, while its sys_metadata row was still there and GET /api/v1/meta/object/name/published still served it." TRUE. Measured on the card (runtime: the standalone stack stamps environmentId: 'env_local', so ObjectQLPlugin skips sys_metadata hydration on every self-hosted boot — an object published at runtime answers 404 on the data API after a restart #20071 body, three-probe table) and pinned by packages/cli/test/package-restart-acceptance.integration.test.ts:380-386 (probe 2 promoted from it.fails to it; the ablation in the dev report reproduces the 404 with that code).
2. "Every os dev / os serve / os start boot was affected." TRUE. packages/cli/src/commands/serve.ts:2740 (standalone mode composes createStandaloneStack); dev and start spawn serve.
3. "createStandaloneStack stamps environmentId: 'env_local' on every boot (or whatever OS_ENVIRONMENT_ID names), and ObjectQLPlugin.start() read any environment id as 'a per-project kernel ...', so it skipped reading sys_metadata at boot. It logged Project kernel — skipping sys_metadata hydration (metadata sourced from artifact)." TRUE. packages/runtime/src/standalone-stack.ts:568; packages/objectql/src/plugin.ts:767-770.
4. "This is the same deduction that runPlatformMigrations was declared out of." TRUE. standalone-stack.ts:223-230 (the The three kernel:ready migrations in assembleMetadataProtocol never arm on a self-hosted boot — the standalone stack stamps environmentId = 'proj_local', and the gate asks for undefined #9380 docblock).
5. "createStandaloneStack now declares hydrateMetadataFromDb: true to ObjectQLPlugin, where it used to be deduced from the environment-id stamp." TRUE. standalone-stack.ts:757-791 (literal in the constructor call); option declared at plugin.ts:179.
6. The two caution reasons (per-instance registry; sys_metadata on the kernel's own default datasource, every dispatched driver kind direct). TRUE as stated in the construction comment standalone-stack.ts:758-786; measured in the dev report (per-plugin ObjectQL, per-engine SchemaRegistry, no proxy driver in the repo).
7. "every env-wide sys_metadata row (organization_id NULL), any metadata type, is registered again. Org-scoped rows are still served on demand and are not read at boot (ADR-0005)." TRUE. plugin.ts:1838 (restoreMetadataFromDb calls protocol.loadMetaFromDb); packages/metadata-protocol/src/protocol.ts:21882-21895 (organization_id: null filter).
8. "a stored row that cannot register now says so at boot ...: [Protocol] [metadata_field_type_refused] ... at error, [Protocol] Failed to hydrate type/name: ... and [Protocol] [metadata_spec_invalid] ... at warn. The same boot also reports org-scoped rows of types that are not per-org overridable, on one aggregated line." TRUE. protocol.ts:22043 (console.error), :22062 (console.warn), :21934 (console.warn), and the org 作用域的 flow overlay 只在「本进程内发布后」绑定触发器,重启后静默失绑——冷启动两条读路径都把 organization_id 非空的行滤掉了 #6190 aggregated warn after the loop (:~22106). The dev report records these were not driven against a real install (NOT MEASURED there); their existence and levels are verified here by source.
9. "the one-shot os migrate * / os meta * commands hydrate too, so a plan or a scan covers runtime-authored objects the way the serving boot registers them." OVERSTATED on the os meta * half. Every os migrate * command boots through bootSchemaStack (packages/cli/src/utils/schema-migrate.ts:189,303, twelve callers), and so does os meta resync (packages/cli/src/commands/meta/resync.ts:132); but os meta get / list / register / delete are not one-shot boots at all, they call a running server through createApiClient (packages/cli/src/commands/meta/get.ts:5,50-58). The reader-facing outcome still holds for them (the server they talk to hydrates), so this is a mechanism overstatement, not a false promise. Precise wording: "the one-shot os migrate * commands and os meta resync hydrate too".
10. "The read itself writes nothing, and a deferred-DDL boot (os migrate plan, os migrate duplicates) still defers the tables of what it read." TRUE. schema-migrate.ts:146-169 (DeferSchemaDdlPlugin arms the driver before boot sync), :301; plan.ts:137-140 and duplicates.ts:896-900 pass deferSchemaDdl: true; the plugin's post-hydration installRegisteredSchemas runs under that deferral.

Still true on origin/main?

Yes. standalone-stack.ts:791 still passes the literal; plugin.ts:767-770 gate unchanged; schema-migrate.ts:303 still composes createStandaloneStack. packages/runtime/CHANGELOG.md:1680 carries the reviewed text verbatim under fa00ebf.

Finding

NONE at the correction bar. The one overstatement (sentence 9, os meta * where only os meta resync is a one-shot boot) does not promise a behaviour a user fails to get; the seat may fold the precise wording into the next @objectstack/runtime changeset if it wants the CHANGELOG exact.

Reviewed-by: session_01VvcEokUG1tvVxkceYfR5XB

VERDICT: CLEAN

---

Generated by Claude Code