Repository navigation
fix(driver-sql): the first boot of a new database no longer prints a sys_migration DATABASE_ERROR (#20768) - #20818
Conversation
…stion leaves the warn channel The first boot of a new database printed one `[sql-driver] DATABASE_ERROR ... no such table: sys_migration` line. The read is the engine's migration-gate read, reached through the ADR-0104 media-arm resolver the driver asks at the start of its first initObjects, before its schema pass has created any table. The resolver now runs inside an async scope, and the read-exit terminal routes one class off the warn channel inside that scope: a missing table, recognised by the shared isMissingTableError over the envelope's declared target. Every other refusal still warns, and so does a missing table read outside the scope. The resolver's answer is unchanged. Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…te database, and the refusals that still warn The driver pin runs a resolver shaped like the engine's through initObjects on a new file; the runtime pin boots a real ObjectQL engine over a real SqlDriver twice. Controls: a malformed read on an existing table, a missing table named by another relation, and a missing table read outside the question all still warn. Adds the patch changeset. Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…wn reads log Measured under the reverse verification: with the fix reverted, the boot's own sys_migration line landed in the same capture and turned the control red. A control must not move with the defect it sits beside. Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…rst-boot-migration-gate-read # Conflicts: # packages/drivers/driver-sql/src/sql-driver.ts
…rst-boot-migration-gate-read
📓 Docs Drift Check3 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 11 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 80a01a70cd2eb54b2992c760d01c3fa074f3c656 && git checkout 80a01a70cd2eb54b2992c760d01c3fa074f3c656
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 157baa75f276e5f9e157cf83c10b4774449cf481 c5ae3a6f8e2559353acb5a5ebe5f02de7b559dc7 && git checkout -B drift-repro 157baa75f276e5f9e157cf83c10b4774449cf481 && git merge --no-ff c5ae3a6f8e2559353acb5a5ebe5f02de7b559dc7
node scripts/docs-audit/affected-docs.mjs --json 157baa75f276e5f9e157cf83c10b4774449cf481 |
Contract reviewServed-tier: PR #20818 (card #20768), branch ① Derived judgmentsDiff: 4 files, +541 / -9.
② Semver level
③ Boundary flags
Implemented-by: VERDICT: PASS Generated by Claude Code |
…_ERROR for the tables whose DDL it deferred (objectstack-ai#20821) (objectstack-ai#21093) Fixes objectstack-ai#20821 Clause-②: no `os migrate plan` against a database that does not exist yet printed six `[sql-driver] DATABASE_ERROR … no such table` lines on stderr. The plan boots with the SQL driver's DDL deferred, lists every table as pending `create_table`, and then the same boot reads `sys_metadata`, `sys_metadata_activation` and `sys_migration`. Every one of those reads was refused, and every reader already answered from the refusal. Only the driver's warn line was wrong. This follows the seat answer on the card (5924276655): Q1 **A**, a driver-side demotion keyed on the driver's own deferral; Q2 **A**, the pin is scoped to the three deferred tables. ## The six readers (traced with a stack per line at `b253fadfb`; unchanged at this head) | table | reader | when | |:--|:--|:--| | `sys_metadata` | `ObjectQLPlugin.restoreMetadataFromDb` → `ObjectStackProtocolImplementation.loadMetaFromDb` (`isMissingTableError` → `loaded: 0`) | `ObjectQLPlugin.start()`, phase 2 | | `sys_metadata` | `readAuthoredTranslationLayer` (`core`, `fallbacks/authored-translation-sync.ts`) → `null` | `kernel:ready` | | `sys_metadata` | `ObjectQLPlugin.readAuthoredHookRows` via `resyncAuthoredHooksNow` → `null` | `kernel:ready` | | `sys_metadata` | `ObjectQLPlugin.readAuthoredActionRows` via `resyncAuthoredActionsNow` → `null` | `kernel:ready` | | `sys_metadata_activation` | `ObjectStoreActionActivationStore.probe` via `ObjectQLPlugin.hydrateActionActivations` → its own functional warn | `kernel:ready` | | `sys_migration` | `ObjectQL.announceOpenMigrationGates` → `readMigrationFlagVerified` → "not verified, not conclusive" | `kernel:bootstrapped` | The `DATABASE_ERROR` line is written inside `SqlDriver.backendStatementFault`, before any of these readers sees the error. So a reader-side catch cannot remove it. "Not asked" was measured and declined: no reader can see the deferral, because no `IDataDriver` member and no kernel key carries it. Deferred is also not the same as absent: on an existing database the deferral records every object before any `hasTable`. Making "not asked" honest would need a new contract fact, and `Clause-②` would become yes. ## The change `packages/drivers/driver-sql/src/sql-driver.ts`, `SqlDriver.backendStatementFault`, in the warn decision only. A refused statement goes to `logger.debug` instead of `logger.warn` only when all three conditions hold: 1. this driver has DDL deferred (`deferredDdl`); 2. the targeted table is in this driver's own `deferredSchemaObjects`; 3. `isMissingTableError(envelope, object)` holds. This is the one shared predicate, asked over the envelope's declared target. No second message regex. The throw and the envelope (`DATABASE_ERROR`, status 500) are unchanged. Every other refusal still warns: a malformed statement on a table that exists, a missing table the driver did not defer, and anything after `flushDeferredSchemaDdl` (which clears both the flag and the set). The new branch sits beside PR objectstack-ai#20818's pre-DDL scope check. It is disjoint from the PR objectstack-ai#20988 hunks, the JSON-column gate region and the JSON-membership helpers. ### Kept on purpose - **The stdout WARN from `ObjectQLPlugin.hydrateActionActivations`** (`sys_metadata_activation is registered but could not be read — packaged-ACTION enable/disable is UNAVAILABLE on this deployment …`) is kept. It is the reader's own functional line, not the driver's, and the stdout-identity pin requires it. On a dry run against an absent file it is a false alarm, and that is recorded here. - **The `examples/app-crm` app-hook lines** (`sys_position` ×3, `sys_permission_set` ×3) are kept. They come from the app's `onEnable` `kernel:bootstrapped` hook, which reads tables the plan's composition never declares. That is a different door: objectstack-ai#21054 is not addressed here, and the pin does not count those lines. ## Before and after on `examples/app-crm` `node ../../packages/cli/bin/run.js migrate plan [--json] --database-url file:ABSENT.sqlite`. The base is `576afc17b` (before the fix). The head is `ea9309b80`, rebuilt. | reading | base | head | |:--|:--|:--| | `DATABASE_ERROR` lines on stderr | 12 | 6 | | … naming `sys_metadata` / `sys_metadata_activation` / `sys_migration` | 6 (4 / 1 / 1) | **0** | | stderr diff | — | exactly those 6 lines removed, 0 added | | human stdout, normalised for timestamps and `Nms` durations | — | `diff` exit 0 | | the plan block (`ℹ Database:` … `Apply with:`) md5 | `e102064c7b13bce96303c1e9b698693d` | `e102064c7b13bce96303c1e9b698693d` | | `--json` stdout with `duration` removed | — | byte-identical (`cmp` exit 0) | | database file left behind | none | none | ## Tests - **Driver pin, new:** `packages/drivers/driver-sql/src/sql-driver-20821-deferred-ddl-missing-table.test.ts`, on a real SQLite file. Every case asserts the envelope `code` and `status`. - ① DDL deferred, the table in the deferred set and missing: the refusal goes to `debug` (with `no such table`), and `warn` stays empty. After `flushDeferredSchemaDdl` the same read answers `[]` with nothing logged. - ② Control: a malformed read (40,000 bound variables) on an existing table whose DDL is deferred still warns. - ③ Control: a missing table outside the deferred set, on the same deferred driver, still warns. - **CLI pin, new:** `packages/cli/src/commands/migrate/plan.deferred-reads.integration.test.ts` (integration tier: it spawns `bin/run-dev.js`). Its fixture is one host config with a lookup field. On the base, that fixture reproduced exactly the card's 6 lines (4 / 1 / 1). In human mode and under `--json`, it asserts: - 0 `DATABASE_ERROR` lines naming the three tables; - the reads still answered: the open `[value-shape]` gate is announced, and the activation WARN is present; - each of the three tables is still pending `create_table`; - no file is written. - **Suites at `ea9309b80`:** - `pnpm --filter @objectstack/driver-sql exec vitest run --maxWorkers=2`: 204 files passed, 11 skipped; 3285 tests passed, 188 skipped. The skips are the live PG and MySQL cells, which are not provisioned here. - `@objectstack/cli` `--project unit`: 240 files, 3419 tests passed. - `@objectstack/cli` `--project integration`, run on this PR's file plus `schema-migrate.readonly-probe` and `schema-migrate.deferred-ddl`: 3 files, 9 tests passed. - **NOT MEASURED: the full cli integration tier.** Reason: it ran past the 10-minute foreground cap here (exit 124 at 595s). It is declared to CI. - `typecheck` for `driver-sql` and `cli` is green, including `cli`'s `check:test-typecheck` (debt unchanged). The new cli pin is inside `tsc`'s program, because `src` is the `include`. ## Reverse verification (from the committed fix, through `scripts/ablation-replace.mjs`) 1. **The deferred-set condition removed** (anchor `this.deferredSchemaObjects.has(targetedTable) &&`, 1 → 0; blob `c626b59d` → `fecb8704`). The driver pin turned **red on ③ only** (`expected [] to have a length of 1 but got +0`), and ① and ② stayed green. The driver pin imports `src`, so no build was involved. Restore: blob `c626b59d` equals HEAD, and `git diff HEAD` is empty. 2. **The debug branch removed** (the whole `if` block, 1 → 0; blob `c626b59d` → `e7acc7e7`). `driver-sql` was rebuilt. `ablation-dist-preflight --absent 'this driver deferred its DDL'` found the marker absent from all 6 built files. The CLI pin turned **red on both cases** (`expected [ …(6) ] to deeply equal []`). Restore: blob equals HEAD and `git diff HEAD` is empty. After a rebuild, the preflight found the marker present in 2 built files and the tree clean against HEAD, and both pins were green again. This leg was run again at `fd9f151a4`, after the pin's spawn harness changed: same result. ## Gates - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` at `ea9309b80` derived 66 families. All 66 were run, with exit codes recorded before any pipe, and all exited 0. `--ran` reports: `66 derived, 66 run, 0 NOT-MEASURED, 0 UNRUN`. - `check:dual-build-cjs-loads` and `check:i18n-coverage` first answered `PREREQUISITE NOT MET` (exit 3) on the earlier merge head, because nine packages had no `dist/`. Those packages were built and both gates re-ran green. - The derivation warned that the tree was behind `origin/main` and named two of its inputs as changed: `lint.yml` gained timeout keys and comments, and `engine-double-contract.pinned.json` gained entries for other files. Neither changes the family set. No commit since the merge touches `driver-sql` or `cli/src/commands/migrate`. - `node scripts/check-driver-conformance.mjs`, before and after: `OK — 50 covered cell(s), 0 in the DEBT ledger, 0 exempt.` - Lint, a proven narrowing: `eslint --no-inline-config --format json` over the 3 touched TypeScript files reports 3 files, 0 errors, 0 warnings. `eslint --print-config` resolves each of them, and none carries `parserOptions.project`. `eslint.config.mjs` enables no type-aware linting, so this diff cannot move the verdict on any file it does not touch. The repo-wide `pnpm lint` is CI's. ## Changeset `.changeset/20821-plan-deferred-ddl-reads.md`: a `patch` for `@objectstack/driver-sql`. `@objectstack/cli` ships only `dist`, `README.md` and `CHANGELOG.md`, and `tsconfig.build.json` excludes `src/**/*.test.ts`, so the pin publishes nothing and gets no entry. ## Acceptance notes - In this worktree, one gate run wrote `examples/app-crm/dist/objectstack.json` (gitignored). With that compiled artifact present, `os migrate plan` takes the artifact boot path: `onEnable` does not run, and the six objectstack-ai#21054 lines disappear too. The after-measurement above was taken with that artifact removed, so it matches the base's no-artifact path. This is noted for whoever takes objectstack-ai#21054, since whether the app hook runs depends on that file. --- _Generated by [Claude Code](https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Fixes #20768
Clause-②: no
On the first boot of a new database, the SQL driver printed one
[sql-driver] DATABASE_ERROR ... no such table: sys_migrationline on its warn channel. The read behind it is the engine's migration-gate read. The driver's own pre-DDL question reaches that read before the schema pass has created any table. The driver now asks that question inside an async scope. Inside the scope, one class of refusal goes todebuginstead ofwarn: a missing table, recognised by the sharedisMissingTableErrorover the envelope's declared target. Every other refusal still warns, and so does a missing table read anywhere else. No gate's answer changes.Reproduction on the base
objectstack dev --database file:NEW.sqlite -p PORTonexamples/app-crm, base96e724475c(every package in the app's closure built), stdout and stderr captured separately:DATABASE_ERRORlinesconsole.warnsink)The line, verbatim:
The mechanism: which dispatch hypotheses held
H1 held. For one boot, I added a temporary stack-trace line to the built
packages/objectql/dist(restored afterwards,cmpbyte-identical on all four bundles, marker count 0). The first gate read at boot isObjectQL.readMigrationFlagVerified, reached throughreadFileReferencesFlagRowandhaveFileColumnsMoved. It is called by the closure thatregisterDriverhands the driver, whichSqlDriver.resolveFileColumnsMovedasks at the start ofSqlDriver.initObjects, from the schema pass's firstsyncSchema, before any DDL. The other gate readers run after the schema pass has createdsys_migration, so on the first boot they read without error:isValueShapesMigrationVerifiedfrom write hooks, andannounceOpenMigrationGatesatkernel:bootstrapped.H2 falsified. The engine cannot tell whether the table exists without asking the backend:
hasTable.conclusivemust befalse. The same pass creates the table moments later, and a memoized "not verified" would freeze a whole boot's posture from a moment when nothing could answer. That is what the refused read already returns, so it is unchanged.hasTable) would give "not asked", but it needs a new public driver method or a newIDataDrivermember. That enlarges a public surface, against this card'sClause-②: no, so this PR does not add one.H3 held.
SqlDriver.backendStatementFaultwrites the line before the engine sees the error. The engine'sfindOnepath does not log, and its catch inreadMigrationFlagVerifiedanswers{ verified: false, conclusive: false, columnsMoved: false }silently. So the demotion is in the driver, keyed onisMissingTableError, and limited to this read: the async chain of the driver's own question.H4: measured at
c5ae3a6f8e. The second boot prints nothing, as on the base.os migrate plan --database-url file:Xon the database the boots created prints noDATABASE_ERROR. On a path that does not exist yet, the plan printed 7 lines on the base, 2 of them onsys_migration, and prints 6 now. The onesys_migrationline left is theadr-0104-value-shapesread fromannounceOpenMigrationGates, outside the driver's question. The other 5 aresys_metadata(4) andsys_metadata_activation(1). They are not this card. See the acceptance notes.Landing site:
packages/drivers/driver-sql, notpackages/objectql/src/engine.tsThe dispatch expected the engine, with the driver only if a demotion was needed. As H2 and H3 show, the demotion is needed, and the line's producer is the driver. The premature read is the driver's own pre-DDL question.
engine.tsis unchanged.What changed
packages/drivers/driver-sql/src/sql-driver.ts:AsyncLocalStorage(PRE_DDL_QUESTION_SCOPE).resolveFileColumnsMovedruns the resolver inside it, and nothing else does. It is module-level rather than per instance because the question's read goes to whichever driver serves the ledger, which on a multi-datasource composition can be another instance in the same async chain.backendStatementFaultcomposes the envelope first. If the scope is active andisMissingTableError(envelope, object)holds, it writes the same dialect text tothis.logger.debug?.(...)and returns the envelope. Otherwise it warns as before. The throw, the envelope, and what the resolver hears are unchanged.debugchannel. The default sink has none.isMissingTableErroris imported from its home,@objectstack/types.@objectstack/metadata/errorsre-exports the same symbol, and driver-sql cannot depend on@objectstack/metadata. No second message regex..changeset/20768-first-boot-migration-gate-read.md:@objectstack/driver-sqlpatch.@objectstack/runtimegains only a test file, which itsfiles[](dist,README.md,CHANGELOG.md) does not ship, so it gets no changeset entry.After: this branch at
c5ae3a6f8e, builtDATABASE_ERRORlinessys_migrationmentionsobjectstack dev --database file:NEW.sqlite(examples/app-crm), first bootos migrate planon that fileNormalised for timestamps, port and file path, the first-boot stderr differs from the base run in exactly one line: the removed one. Both ledgers end in the same state after the first boot: both flags verified by the fresh-datastore attestation, and
columns_moved_atnull.Tests
packages/drivers/driver-sql/src/sql-driver-20768-pre-ddl-question-missing-table.test.ts(new, SQLite on a new temp file). The resolver is a stand-in that reads the ledger the way the engine does.debugline (no such table). The resolver gets the envelope (code: 'DATABASE_ERROR',status: 500), and the arm stays "not moved".columns_moved_atrow written between the boots makes the arm "moved", so the read reached the row. Nothing is logged.warn: ④ a malformed read on an existing table inside the question (40,000 bound variables, SQLite refuses the statement); ⑤ a view over a dropped table inside the question (a missing table named by another relation); ⑥ a missing table read outside the question.packages/runtime/src/first-boot-migration-gate-read.integration.test.ts(new). A realObjectQLengine over a realSqlDriveron a new file,SysMigrationregistered, andengine.syncSchemas(), which is the samesyncSchema,initObjects, resolver, gate-read chain.sys_migrationDATABASE_ERRORon warn, and one demoted line on the first boot only.c5ae3a6f8e:--project local: 293 files, 4207 tests passed, 1 skipped.check:test-typecheckdebt is unchanged, andtsc --listFilesshows both new tests inside their packages' programs.Reverse verification (from the committed fix)
packages/drivers/driver-sql/src/sql-driver.tswas restored to its base blob withgit restore --source=96e724475c: on-disk blob2462eccd= base, marker count 0. driver-sql was rebuilt, andablation-dist-preflight --absent PRE_DDL_QUESTION_SCOPEfound the marker absent from all 6 built files.expected [ Array(1) ] to deeply equal []). ③ to ⑥ stayed green.Restore:
git checkout HEAD -- packages/drivers/driver-sql/src/sql-driver.ts. Bloba3647fd1= HEAD, andgit diff HEADwas empty. After a rebuild, the preflight found the marker in 2 built files and a clean tree, and both pins were green again.The first run of this verification also turned the runtime control red: it had counted the boot's own
sys_migrationline together with its own. It now counts only its own reads, and the second run above is from that commit.Gates
node scripts/pm/dispatch-gates.mjs --commandsatc5ae3a6f8ederives 63 commands, and all 63 ran on that head with exit 0.--ranreconciliation: "63 derived famil(ies) accounted for — 63 run, 0 NOT-MEASURED (a DERIVED zero — all 63 recorded an exit code and none of them is 3)". On the first pass,check:dual-build-cjs-loadsanswered PREREQUISITE NOT MET (9 packages had nodist/). I built those 9 and re-ran it, and it exited 0.Lint, narrowed:
eslint --no-inline-config --format jsonover the 3 touched.tsfiles counted 3 files, 0 errors, 0 warnings. The population is the config'spackages/**/*.{ts,tsx,mts,cts}and**/*.{ts,...}objects. The config never enables type-aware linting (noparserOptions.project), so this diff cannot change the verdict on an untouched file.The branch merged
maintwice. The second merge brought in #20794, the serial constraint. Neither merge touched this diff's files, apart from the tracker-number wording in the sameDATABASE_ERRORline, which this branch now carries asmainspells it.Acceptance notes
os migrate planagainst a database that does not exist yet still prints 6DATABASE_ERRORlines atc5ae3a6f8e:sys_metadata4,sys_metadata_activation1, andsys_migration1. The last is theadr-0104-value-shapesread byannounceOpenMigrationGatesatkernel:bootstrapped, which on a deferred-DDL plan runs over tables the plan never creates. It is the same false-alarm family, on a different door, and is reported to the seat. It is not fixed here.debug. No production composition handsSqlDrivera logger today, so with the default sink the line is dropped. The engine's catch records nothing either.PRE_DDL_QUESTION_SCOPEcovers any read in the resolver's async chain, including one a hook starts inside it. That is intended: every such read is issued for the question, before the schema pass has run.Generated by Claude Code