Repository navigation
fix(driver-sql,driver-turso)!: refuse an upsert whose conflict lands on another organization's row (#21185) - #21225
Conversation
…sal (red on main) Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp Co-authored-by: Claude <noreply@anthropic.com>
…utside the written organization An upsert resolves its conflict against the whole table, so a tenant-scoped call could merge into, and re-parent, a row of another organization. The merge leg is now fenced to rows of the organization the row is written under (in-statement predicate on SQLite/PostgreSQL/libsql; a transaction-bound check on MySQL), a conflict elsewhere is refused with UNIQUE_VIOLATION/409 and writes nothing, and the tenant column joins insertOnlyUpsertColumns so no upsert re-parents a row. Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp Co-authored-by: Claude <noreply@anthropic.com>
… context; pin the caller-transaction refusal Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp Co-authored-by: Claude <noreply@anthropic.com>
…sert refusal (BREAKING, minor) Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp Co-authored-by: Claude <noreply@anthropic.com>
…ires Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp Co-authored-by: Claude <noreply@anthropic.com>
…sert-cross-org-refusal
📓 Docs Drift CheckThis PR changes 3 package(s): 15 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 5 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 140 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 76d50acf09f9fac6f4b96d547c2d7617d3fd08f0 && git checkout 76d50acf09f9fac6f4b96d547c2d7617d3fd08f0
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 7c5a311a5829ae3b550a3eeb9bb984f06be8b865 35d01c9d7fedae909f93a171ac2c319ed17000a6 && git checkout -B drift-repro 7c5a311a5829ae3b550a3eeb9bb984f06be8b865 && git merge --no-ff 35d01c9d7fedae909f93a171ac2c319ed17000a6
node scripts/docs-audit/affected-docs.mjs --json 7c5a311a5829ae3b550a3eeb9bb984f06be8b865
|
…IOLATION provenance for driver-sql and its ADR-0087 semantic entry Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp Co-authored-by: Claude <noreply@anthropic.com>
…sert-cross-org-refusal
Contract reviewServed-tier: Inputs: card #21185 (body; comments 5933970973, 5934416748, 5934820328, the ruling 5934879010, the claim 5935961155, the round-0 os-dev-report 5937894588, the claim amendment 5937974892, the round-1 os-dev-report 5938821270); PR #21225 (body as edited by the seat, 9-file list, the net diff against ① Derived judgmentsEvery accept-set or public-surface change the diff implies, each named right or wrong.
② Semver level
③ Boundary flagsRound 0 deviations (5937894588):
Round 1 deviation (5938821270): the entry file is
Out-of-scope findings:
Check-runs on Implemented-by: VERDICT: PASS Generated by Claude Code |
…objectstack-ai#21227) (objectstack-ai#21239) Fixes objectstack-ai#21227 Clause-②: no ## What changes `SqlDriver.create` and `SqlDriver.bulkCreate` ran `builder.insert(...).returning('*')` and answered what the statement answered. MySQL has no `RETURNING`: knex's MySQL compiler drops the clause (it logs `.returning() is not supported by mysql`) and answers `[insertId]`. That is ONE element whatever the row count, and `0` for this driver's string primary key. Both doors now answer the stored row on every dialect: - **SQLite and PostgreSQL families**: unchanged. They answer from `RETURNING` in one statement. - **MySQL family, and any client the driver recognises as neither family**: the INSERT is issued without `.returning()`, and the rows are read back by the ids that were written. That is one `SELECT` per `create`, and one per `bulkCreate` batch. The switch is one protected capability getter, `insertReturnsStoredRows` (`isSqlite || isPostgres`), next to `isMysql`. One private helper, `readBackInsertedRows`, serves both doors. No caller was patched: the auth adapter, the engine and the protocol are untouched. ### Measured before the fix (live MySQL 8.0.46, `origin/main` `62b90d74`, driver called directly) | call | answered | stored | |:--|:--|:--| | `create`, generated id | `0` | the row | | `create`, supplied id | `0` | the row | | `bulkCreate`, 3 rows | `[0]` (length 1) | all 3 rows | | `bulkCreate`, 1 row | `[0]` | the row | SQLite and live PostgreSQL 16.14 answered the full stored rows in the same probe, including `done: false`, which only the column DEFAULT supplies. That is the control. ### The doors, before and after (`pnpm dev:crm -- --fresh --database mysql://...` on that MySQL 8.0.46) | door | at `62b90d74` | with this change | |:--|:--|:--| | `POST /api/v1/auth/sign-up/email`, first user (`--no-seed-admin`) | `400 FAILED_TO_CREATE_USER`; `sys_user` row stored, no `sys_account` row | `200`; user, `credential` account and session stored | | `POST /api/v1/auth/sign-in/email`, same credentials | `401 INVALID_EMAIL_OR_PASSWORD` | `200` | | `--seed-admin` at boot | `dev admin seed skipped: Failed to create user`; orphaned `sys_user` | seeded; admin sign-in `200` | | `POST /api/v1/data/crm_account` as the seeded admin | not reachable (no session could exist) | `201`, with the full record including the stamped `organization_id` | | boot: `curated capability ... has no platform row and could not be seeded` warnings | 9 | 0 | ## Decisions the card left open - **H3: read back only where `RETURNING` does not answer the stored row.** Reading back on every dialect would add one round trip to every `create` on SQLite and PostgreSQL, where `RETURNING` already answers the stored row (measured above). It would also move the control cells onto new code. Cost on MySQL: +1 `SELECT` per statement. Cost on SQLite and PostgreSQL: 0. The pin file counts the statements on every cell. The getter is a positive list on purpose. A client the driver does not recognise (a Client constructor, `redshift`, `mariadb`) reads back, which is correct on every dialect. `RETURNING` is the shortcut that only a dialect known to answer the stored row gets. `driver-sqlite-wasm` overrides `isSqlite` to `true`, so it keeps `RETURNING`. - **H2: one helper for `create` and `bulkCreate` only. `update` and `upsert` are byte-unchanged.** The four read-backs answer different things: - `update` reads by id under the caller's scope and answers `null` on a miss, which its contract allows. - `upsert` reads by the conflict-key values it matched on and falls back to the payload. - `create` must answer a row, and is keyed on ids it wrote. Sharing one helper would change one of those answers. It would also touch the `upsert` region, which the card fences off. - **H4: the read key is the written id, and that id always exists.** `create` and `bulkCreate` give every row its id before the statement is built: the caller's `id`, else `_id`, else a minted nanoid. The managed `id` column is `varchar(255)` PRIMARY KEY with no AUTO_INCREMENT, so no insert id is ever read. That is the only kind of key this path produces. - The column goes through `remoteColumn`, so an external `columnMap` that renames `id` is read by its physical column. - The table is the write target, a rotation shard included. - The tenant scope goes through `applyTenantScope`, scoped to the tenant(s) the rows were WRITTEN under (as `assertMergeLandedOnSuppliedIdentity` scopes its probe). For a batch that is the union through `tenantIds`. So an admin write that names another tenant in the row data is answered rather than missed. - The ids are this call's own and `id` is the PRIMARY KEY, so the read cannot answer another organization's row. - The read uses the caller's transaction when there is one. - **A written row that is gone before the read-back** (a concurrent delete, or a trigger) is refused with `DATABASE_ERROR` / 500. It is not answered with the payload, and the insert is not re-issued. The read-back runs outside the insert's `try`, so a read fault can never reach the autonumber collision retry. ## One conclusion per face of the invariant (`IDataDriver.create` answers the inserted record) 1. **`driver-sql`**: changed for the MySQL family. SQLite and PostgreSQL are already conformant and unchanged (evidence: the pin's control cells, green before and after). 2. **`driver-sqlite-wasm` and LOCAL-mode `driver-turso`**: inherit `SqlDriver.create` / `bulkCreate` and stay on `RETURNING`. Wasm overrides `isSqlite` to `true`; Turso local uses `better-sqlite3`. Their suites are green: wasm 36 files / 675 tests, turso 86 files / 2313 passed, 33 skipped. 3. **REMOTE-mode `driver-turso`**: already conformant. `RemoteTransport.create` issues its INSERT and then `SELECT * ... WHERE "id" = ?` and answers that row (`remote-transport.ts`). Its `bulkCreate` loops the driver's own `create`. 4. **`driver-memory`**: already conformant. `create` pushes the built record and answers a copy of it, and `bulkCreate` answers the pending records it pushed. 5. **`driver-mongodb`**: already conformant. `create` answers the document it inserted (minus `_id`), and `bulkCreate` answers the inserted docs in order. ## Pins `packages/drivers/driver-sql/src/sql-driver-21227-create-answers-stored-row.test.ts`, through `declareDialectCell`: SQLite always, and live PostgreSQL and MySQL where provisioned. The `Temporal Conformance (live PG + MySQL)` job runs them on both. There is also one cell that always runs: SQLite with the read-back path forced. It puts the read-back's ordering, tenant scope, transaction and refusal into every CI run, not only the job with a MySQL server. Each test pins one behaviour: - `create` with a generated id and with a supplied id; - `bulkCreate` of 3 rows, which answers 3 rows in written order from ONE insert, plus ONE read on the read-back path; - `bulkCreate` of 1 row; - a tenanted create; - an admin write naming another tenant (single and batch); - create and bulkCreate inside a rolled-back caller transaction; - the vanished-row refusal (forced cell only), asserting `code` and `status` and that the insert is not re-issued; - a per-cell check that the cell measures the path it claims to. Every answer is compared with the driver's own `findOne` and must carry the DEFAULT-only `done: false`. Reverse verification, both legs run with live PostgreSQL 16.14 and MySQL 8.0.46: - **Fix reverted** (`sql-driver.ts` at `62b90d74`, worktree only, restore by trap with a hash check): `13 failed | 20 passed`. - All 8 MySQL tests are red: `expected +0 to deeply equal {...}`, and `expected [ +0 ] to have a length of 3 but got 1`. - 3 forced-cell path tests are red. - The SQLite and PostgreSQL behaviour tests stay green (the control). Their path checks are red only because the getter does not exist before the fix. - **Fix in place** (HEAD `f42d0354c3`): `33 passed`. **Sign-up door pin: not added, declared.** No live-dialect harness runs the real auth stack against a datasource URL. The plugin-auth real-engine harness (`signup-existing-address-refusal.test.ts` and its siblings) hard-codes better-sqlite3. A MySQL door pin there would need a per-file database isolation helper in plugin-auth and a new CI step in the live job. Without that step, the pin is a named skip that never runs in CI. The door is measured above instead. The options are in the report for the PM. ## Verification (HEAD `ee7c024b92`, after merging `origin/main` with objectstack-ai#21225 in it) - `pnpm --filter @objectstack/driver-sql exec vitest run --maxWorkers=2` with `OS_TEST_POSTGRES_URL` and `OS_TEST_MYSQL_URL` (PG 16.14, MySQL 8.0.46), `OS_EXPECT_LIVE_DIALECT_MATRIX=1`, `TZ=America/New_York`: **223 files passed, 5369 passed, 1 skipped**. The skip is pre-existing, in `schema-drift.base-type-mismatch.test.ts`. - `pnpm --filter @objectstack/driver-sqlite-wasm test` (36 / 675 passed) and `pnpm --filter @objectstack/driver-turso test` (86 files, 2313 passed, 33 skipped): exit 0. - `typecheck` for driver-sql, driver-sqlite-wasm and driver-turso: exit 0. `tsc --listFiles` includes the new test file. - `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands` derived 63 commands at `ee7c024b92`, and all 63 exited 0. `--ran` reconciliation: `63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN`, all with recorded exit codes. This includes `check:tenant-chokepoint` ("every read builder routes through applyTenantScope()"). - `pnpm check:driver-conformance`: before the first edit (`62b90d74`) `50 covered cell(s), 0 in the DEBT ledger, 0 exempt`, dialect axis `8 conformance suite(s) ... 0 in the DIALECT ledger`; after the last commit (`ee7c024b92`), identical. - **Lint, a declared narrowing.** `eslint --no-inline-config --format json` was run on the two touched TypeScript files. - Population: eslint's own `--print-config` resolves rules for both (6 and 5 rules; neither file is ignored). - Count: 2 files, 0 errors, 0 warnings. - Invariance: `eslint.config.mjs` enables no type-aware linting (`parserOptions.project` and `projectService` are null for both files), so this diff cannot move a verdict on an untouched file. - The repo-wide `pnpm lint` is left to CI. ## Acceptance notes - `sql-driver-21163-autonumber-prefix-like-escape.test.ts`'s header says its cases read the stored row "Not from `create`'s return value: on MySQL that is not the row". After this change that sentence is stale. It is a test comment, not published; it is left for whoever next edits that file. - In the same MySQL boot, service-package's raw `CREATE TABLE IF NOT EXISTS sys_packages (... created_at TEXT DEFAULT CURRENT_TIMESTAMP ...)` is refused with `ER_INVALID_DEFAULT` (`Invalid default value for 'created_at'`), and later `SELECT * FROM sys_packages` reads answer `ER_NO_SUCH_TABLE`. No door was measured for it, so it is noted here and not filed. - The `sys_activity` boot failure is reported to the PM with a measured door, for the seat to file. It is not touched here. --- _Generated by [Claude Code](https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Fixes #21185
Clause-②: no (narrowing)
Executes ruling record 5934879010 on #21185 (letter A, refinements 1/2/3): a tenant-scoped
upsertwhose conflict lands on a row of another organization is refused withUNIQUE_VIOLATIONand writes nothing, and the tenant column is insert-only (insertOnlyUpsertColumns), so an upsert never changes a row's organization.update()stays the deliberate path. Classes and positions only; the reproduction stays with the seats.The two
packages/specedits (patch round 1, on the seat's answer A)The ruled answer needed two
packages/specedits. The dispatch held them for the seat's word, and the seat answered A on #21185 (claim amendment 5937974892), under the spec lane's standing pre-approval. Neither edit moves the ruledClause-②: no (narrowing): no accept set widens. The precedent is thedriver-sql-calendar-day-methods-removedchangeset, which carriesregisteredplusClause-②: no (narrowing)with@objectstack/specatpatch.82f8120c).driver-sqlnow stamps the registeredUNIQUE_VIOLATION(UPSERT_UNIQUE_VIOLATION_CODEinsql-driver.ts), so one row,'UNIQUE_VIOLATION', is added under the@objectstack/driver-sqlowner key inpackages/spec/src/api/error-code-ledger.zod.ts. It is provenance only: theErrorCodeunion is byte-unchanged, andapi-surfacerecords the ledger by name only.driver-tursothrows the same constructor throughSqlDriver, so it needs no row.82f8120c, then5682e8fe). The ruled FROM → TO is a migration prescription, so the disposition isregistered driver-upsert-cross-organization-conflict-refused. The semantic entry ispackages/spec/src/migrations/entries/semantic/18.driver-upsert-cross-organization-conflict-refused.ts. Its prefix is the protocol-18 step thatregistry.tsrecords as accumulating and uncut, as for the precedent18.driver-sql-calendar-day-methods-removed.ts.src/migrations/registry.tswas regenerated withgen:migration-registry, the one artifactcheck:generated --fixproved stale;spec-changes.jsonand the upgrade guide do not move until the protocol major reaches 18. The changeset front matter carries'@objectstack/spec': patch.What changed, per face
SqlDriver.upsertTenantGuard): it applies when the call is tenant-scoped (non-emptyoptions.tenantId) on an object with a tenant column. The value is the organization the row is written under, afterinjectTenantOnInsert, the reading drivers(sql): on MySQL an upsert with no conflictKeys — or one naming the primary key — still merges on a unique key the caller never named #8807's identity probe already used. It holds for any conflict target, the primary key included.ON CONFLICT … DO UPDATE SET … WHEREthe stored tenant columnIS(SQLite) /IS NOT DISTINCT FROM(PostgreSQL)excluded's. The read-back that already follows the statement now reads under the written organization exactly (the chokepoint scope plus equality on the tenant column), and a miss is the refusal.ON DUPLICATE KEY UPDATEtakes noWHERE, so drivers(sql): on MySQL an upsert with no conflictKeys — or one naming the primary key — still merges on a unique key the caller never named #8807's mechanism is reused. The statement and a read of the landed row under the written organization (assertMergeLandedInWrittenOrganization) run as one transaction whenever the call is fenced, not only when a rival UNIQUE key exists. Inside a caller's transaction they run in a nested transaction (a savepoint), so the refusal never depends on what the caller does next. When drivers(sql): on MySQL an upsert with no conflictKeys — or one naming the primary key — still merges on a unique key the caller never named #8807's check is also owed, the new read carries its verdict. A miss is told apart by reading each rival key's sent values under the written organization: a hit there means the merge landed through a rival key in the caller's own organization, and drivers(sql): on MySQL an upsert with no conflictKeys — or one naming the primary key — still merges on a unique key the caller never named #8807's refusal is kept; anything else answersUNIQUE_VIOLATION.RemoteTransport.upsert): the same in-statement predicate. Of the ruling's two options, the read-back is scoped, notrowsAffected. When no column is left to merge the statement isDO NOTHING, which affects zero rows for a same-organization conflict too, so arowsAffectedverdict would refuse the control. The transport answersnullon a fenced miss, andTursoDriverthrows the local face's refusal. Measured as part of this: the remote face did not stamp the caller's organization on an upsert at all, so its insert leg wrote a row with no organization. It now callsinjectTenantOnInserton entry, as the local face does.insertOnlyUpsertColumns, the one list both faces read.code: 'UNIQUE_VIOLATION',status: 409, with the engine's duplicate-record sentence. It names no organization, no tenant column and no value of the landed row, and carries nocause.isUniqueViolationErrorrecognises it.assertMergeLandedOnSuppliedIdentityno longer turns the payload's tenant into a scope on a call with no tenant context. With the tenant column insert-only, that scope missed a merge that really landed on the supplied id. This was measured on MariaDB 10.11 as a false drivers(sql): on MySQL an upsert with no conflictKeys — or one naming the primary key — still merges on a unique key the caller never named #8807 refusal before the adjustment.IDataDriver.upsert's docblock inpackages/spec/src/contracts/data-driver.tsis one line and states no merge semantics, soClause-②staysno (narrowing).Tests
The pins and the round-0 results below ran on
e746da26(after mergingmain, which brought in #21178's remote-transport change). Patch round 1's head,35d01c9d, mergesmainagain: #21163 (c6b68891, thescanMaxNumericTailregion ofsql-driver.ts, a clean merge) and #21067 (682873d9). On35d01c9dthe driver suites were re-run (SQLite cell:driver-sql211 files, 3545 passed;driver-turso86 files, 2313 passed;driver-sqlite-wasm675 passed), and so were the live PostgreSQL 16 and MariaDB 10.11 cells (92 of 92). Pins:packages/drivers/driver-sql/src/sql-driver-upsert-cross-organization-refusal.test.ts. One dialect-cell matrix: SQLite, PostgreSQL, MySQL.idtarget, on an explicit['id']target and on aunique: 'global'column, withcodeandstatusasserted. The other organization's row reads back raw and byte-identical, and no row is added.update()still moves it.packages/drivers/driver-turso/src/turso-local-remote-upsert-cross-organization-parity.test.ts. The same pins on bothTursoDriverfaces, plus a parity pin that holds the two faces' answers equal.Local results:
sql-driver-upsert-conflict-target-dialects.test.tspass 92 of 92. That includes drivers(sql): on MySQL an upsert with no conflictKeys — or one naming the primary key — still merges on a unique key the caller never named #8807's tenanted pins.@objectstack/driver-sql: 209 files, 3491 passed.@objectstack/driver-turso: 85 files, 2301 passed.@objectstack/driver-sqlite-wasm(inherits the door): 675 passed.MySQL 8.0 itself is NOT MEASURED locally (no server in the container). CI's
Temporal Conformance (live PG + MySQL)job runs this package's whole suite withOS_EXPECT_LIVE_DIALECT_MATRIX=1.Reverse verification. The fix was committed first. Then
main'ssql-driver.tswas restored under a trap and the new file was run on all three cells: 25 red and 14 green. The red ones are every refusal pin and half 2 on each cell. The fresh-id business-key pin is red on MariaDB only, because SQLite and PostgreSQL already answer it with their own unique violation. The file was then restored, with byte identity proved by blob hash and an emptygit diff HEAD. The turso file ran red onmain(10 of 15), including the remote insert leg writing no organization.Gates. On
35d01c9d,dispatch-gates --commandsderives 93 families (the spec paths added 30), and--ranreconciles 93 derived, 93 run, 0 NOT MEASURED, all exit 0. Among them:check-adr-0087-registration --base origin/main,check:error-code-provenance(336 stamp sites: 317 listed, 19 waived),check:generated(15 artifacts up to date),check:migration-registry,check:spec-changes,check:upgrade-guide,check:api-surfaceandcheck:error-code-casing.check:driver-conformancereads the same before and after (50 covered, 0 DEBT, 0 exempt), andcheck:tenant-chokepointpasses. ESLint, narrowed to the 8 changed.tsfiles, reports 0 errors and 0 warnings (--format json). The config enables no type-aware linting (eslint.config.mjssays so), so this diff cannot move the verdict on an untouched file.Acceptance notes
injectTenantOnInsert's documented admin authority, so it still merges into that organization's row.update()under the caller's scope would not reach that row. Under thegroupposture the fence is the written organization alone: an upsert onto a row of another member organization is refused, whereupdate()'s union scope reaches it.cold.upsert(object, row, ['id'])carries no tenant context, so it gets half 2 only. The sandbox body runner'sql.upsertfalls back toinsert, and the knowledge adapters'upsertis a different interface.driver-sqlite-wasminherits the fence. Its knex dialect extends the SQLite3 compiler and its suite is green, but it has no tenant-scoped upsert pin of its own.SqlDriver.createon the MySQL family answers the insert id, not the record: filed as driver-sql on MySQL: create() answers the insert id (0) instead of the inserted record, so sign-up answers 400 FAILED_TO_CREATE_USER, the dev admin seed fails and no user can sign in #21227. Measured by this card's round-0 dev and then at the public doors:builder.insert(...).returning('*')returns[insertId]on MySQL, socreate()returns0for a string-id table. Sign-up answers400 FAILED_TO_CREATE_USERand no user can sign in.bulkCreatehas the same pattern. This PR's pins read the stored row back rather thancreate's return value, andupsert's own read-back is unaffected, so it does not bear on this change.Generated by Claude Code