Repository navigation
fix(driver-sql, driver-turso): four filter-refusal doors read the provenance mark before naming a read scope field or comparand (#20020) - #20037
Conversation
…mark Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
…hor where existing pins name the predicate Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
…nce mark Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
… provenance; mark author where existing pins name the predicate Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
…provenance mark Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 2 package(s): 8 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 1 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 13 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin e3beb770363735732f0f51dfb0ab28082b6a6e53 && git checkout e3beb770363735732f0f51dfb0ab28082b6a6e53
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 246314dffef9af072b6172d0f1f9678fb7363dec 5fb4b25b1218a23759a2d7d45e55bd5d553da899 && git checkout -B drift-repro 246314dffef9af072b6172d0f1f9678fb7363dec && git merge --no-ff 5fb4b25b1218a23759a2d7d45e55bd5d553da899
node scripts/docs-audit/affected-docs.mjs --json 246314dffef9af072b6172d0f1f9678fb7363dec
|
Contract reviewServed-tier: The merge base with ① Derived judgments
② Semver level
③ Boundary flagsChangeset: every sentence is TRUE, including the four door bullets, "reads the mark on the node it refused", the withheld and Shipped strings: every sentence of the four driver-sql withheld messages and the five PR body:
The dev's open questions and out-of-scope findings, judged for truth:
Other flags:
Implemented-by: VERDICT: FAIL What must change:
Isolated reviewer: a contract-review-tier subagent, fed only the card, the contract file, the PR and AGENTS.md; adopted by the seat. |
…for an unmarked where and the full text for an author-marked one Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
… change from SqlDriver Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: Second round, after FAIL 5823428898 at Scope read:
Measured in the reviewer's own detached worktree at head, which was removed afterwards. ① Derived judgments
② Semver level
③ Boundary flagsChangeset: the added sentence is TRUE (measured in C), and every earlier sentence stays TRUE. The analytics HTTP half of "Measured on both faces" remains UNMEASURED by this review. PR body, the new sentences:
Carried from round 1, not verdict-moving: the per-face compile-surface roster and the unreachable Docs: no hand-written Implemented-by: VERDICT: PASS Isolated reviewer: a contract-review-tier subagent, second read on this PR, fed only the card, the prior record, the PR and AGENTS.md; adopted by the seat. |
… of transactions it does not run (objectstack-ai#20053) Fixes objectstack-ai#20016 Clause-②: no `packages/drivers/driver-turso/README.md` said that `RemoteTransport` runs `beginTransaction`, `commit` and `rollback`. It does not: `RemoteTransport` declares no transaction methods, and `TursoDriver` refuses all three on the remote face with `NOT_IMPLEMENTED` / 501. The README also named none of the remote face's other refusals. This PR corrects the remote branch of the architecture tree and adds one list, **What remote mode refuses**, written from `turso-driver.ts` and `remote-transport.ts` at `adbbc5d01e`. Docs and a changeset only: no source file changes. ## What changes (each sentence, before and after, with the code it was written from) 1. **Architecture bullet, remote mode.** - Before: "Delegates all operations to `RemoteTransport` which uses `@libsql/client` SDK directly (HTTP/WebSocket). No local SQLite or Knex dependency needed." - After: "Sends CRUD, bulk writes, `aggregate`, raw `execute`, schema sync and `dropTable` to `RemoteTransport`, which uses the `@libsql/client` SDK directly (HTTP/WebSocket). Transactions and the other calls listed under What remote mode refuses are refused instead." - From: the `if (this.isRemote)` arms of `TursoDriver.find` / `findOne` / `create` / `update` / `upsert` / `delete` / `count` / `aggregate` / `bulkCreate` / `bulkUpdate` / `bulkDelete` / `updateMany` / `deleteMany` / `execute` / `syncSchema` / `syncSchemasBatch` / `initObjects` / `dropTable`, each forwarding to `this.remoteTransport`. "All operations" was false: `beginTransaction`, `commit`, `rollback`, `setDeferredDdl(true)`, `detectManagedDrift` and `planMediaColumnMove` refuse in `TursoDriver` and never reach `RemoteTransport`. - The dropped sentence "No local SQLite or Knex dependency needed" was false too. `TursoDriver.toKnexConfig` hands the `SqlDriver` constructor a `better-sqlite3` Knex config in remote mode, and that constructor builds Knex straight away. Measured end to end (see the out-of-scope findings): with `better-sqlite3` unresolvable, `new TursoDriver({ url: 'libsql://…' })` throws. I did not replace that sentence with a new dependency claim; the finding is for the code. 2. **Tree, local branch.** "Inherited: applyFilters (MongoDB-style + array-style)" → "Inherited: applyFilters (MongoDB-style)". From: `SqlDriver.compileFilters` in `driver-sql`, which throws `filterArrayReachedDriverError` for any non-empty filter array. The dispatch allowed this fix because the line is in the same tree and plainly false. 3. **Tree, remote branch.** Removed "RemoteTransport: beginTransaction, commit, rollback". Added "Refused, NOT_IMPLEMENTED / 501: beginTransaction, commit, rollback, setDeferredDdl(true), detectManagedDrift, planMediaColumnMove". From: `TursoDriver.beginTransaction` / `commit` / `rollback` (each calls `refuseRemoteTransaction`), `setDeferredDdl` (`refuseRemoteDeferredDdl` when armed), `detectManagedDrift` (`refuseRemoteDriftDetection`) and `planMediaColumnMove` (`refuseRemoteMediaColumnMove`). A grep for a `beginTransaction`, `commit` or `rollback` method declaration in `remote-transport.ts` finds 0; the same pattern shape finds its `find`, `create` and `execute` declarations (3). 4. **Remote example comment.** "// All CRUD operations work the same as local mode" → "// The CRUD methods are the same as in local mode; some calls are refused (below)". The old comment was false for `create` / `bulkCreate` / `upsert` on an object with an empty `autonumber` field (`refuseUngeneratableRemoteAutonumber`), and for any CRUD call that carries `options.transaction` (`assertRemoteTransactionUnsupported`). 5. **New subsection "What remote mode refuses".** Every sentence and where it comes from: - "refused … with `code: 'NOT_IMPLEMENTED'` and `status: 501`": the five builders in `turso-driver.ts` and the two reachable ones in `remote-transport.ts` all set `StandardErrorCode.enum.NOT_IMPLEMENTED` and `501`. - "the call is valid, and the remote transport does not have the capability": the reason each builder's message gives (a capability gap, not a mistake in the request). This is described, not quoted. - "The check runs before the call reads or writes anything": in each method the refusal is the first thing that runs, or it runs before the first `remoteTransport` call. The pins assert this: the transaction suite asserts "the client is never asked for one", and the groupBy suite asserts no statement is captured. - Transactions row: `refuseRemoteTransaction` is reached from the three transaction methods, and from `assertRemoteTransactionUnsupported`, which the 15 `RemoteTransport` methods in the tree plus `aggregate` and `syncSchemasBatch` call first. That is 17 doors, counted by grep. `initObjects` takes no `options`, and the inherited `distinct` / `findWithWindowFunctions` are not guarded, so the row names the 17 doors and does not say "every method". The alternative ("run Knex transactions and honour `options.transaction`") comes from the local arms calling `super` and from the refusal's own remedy text. - Record-numbers row: `refuseUngeneratableRemoteAutonumber` is called from `create`, `bulkCreate` and `upsert` when `mayMerge` is false. `mayMerge` is true when the data carries `id` or `_id`, or when `conflictKeys` is non-empty. The empty test is `isEmptyAutoNumberSlot` (`undefined` / `null` / `''`). The alternatives come from the refusal message. - Deferring-DDL row: `setDeferredDdl` refuses only when `deferred && this.isRemote`. `os migrate plan` boots with `deferSchemaDdl: true` (`commands/migrate/plan.ts`), and that makes the CLI's `DeferSchemaDdlPlugin` call `setDeferredDdl(true)`. The alternative comes from the refusal message. - Drift row: `detectManagedDrift` refuses on remote with or without `objects`. The alternative comes from the refusal message. - Media-column row: `os migrate files-to-references` calls `planMediaColumnMove` from its `runColumnStep`. The local and replica arms call `super.planMediaColumnMove()`. - Transactions bullet: the remote arm of `supports` sets `transactionsUnsupported: this.transportMode === 'remote'`. `driverSupportsTransactions` (spec `driver.zod.ts`) returns false when that is set. `engine.transaction()` then throws `TransactionUnsupportedError` under `require: true`, or calls `warnTransactionUnsupported`, which warns once per datasource name, and runs the callback without a transaction (`objectql/src/engine.ts`). - Record-numbers bullet: the registry is `autoNumberFields`, filled by `registerRemoteFieldMetadata` → `SqlDriver.registerExternalObject` from `completeRemoteSchemaSync`, which the three remote schema doors call. The "logs a warning naming the row" behaviour is `reportUnnumberedRemoteRow`. I watched it fire during the test run below (row id `never-seen`). - Aggregation bullet: `RemoteTransport.aggregate` calls `refuseDateBucketedGroupBy` for any `dateGranularity`, and `refusePerAggregationFilter` for a non-empty `filter`. `engine.aggregate` takes its in-memory path when `allStructuredSupported` is false (remote publishes `queryDateGranularity: {}`) or when `hasAggregationFilter` is true. On that path it strips `groupBy` / `aggregations` / `having` before `driver.find`. The local face (`SqlDriver.aggregate`) throws `unsupportedAggregationFilterError` for the same `filter`. The tree's other remote lines (`find … upsert`, the bulk line, `syncSchema, dropTable`, `execute (raw SQL)`) are true and unchanged. The other local-branch lines are true: each local arm calls `super`, or the method is not overridden. ## The remote refusal set, and the proof that it is complete Tree: `origin/main` `adbbc5d01e` (`packages/drivers/driver-turso/src`, excluding `*.test.ts` and `*.testkit.ts`). - `git grep -n "status = 501"` → **8 hits**: `turso-driver.ts` 5 (`refuseRemoteAutonumber`, `refuseRemoteTransaction`, `refuseRemoteDeferredDdl`, `refuseRemoteDriftDetection`, `refuseRemoteMediaColumnMove`) and `remote-transport.ts` 3 (`uncompilableAggregateFunctionError`, `refusePerAggregationFilter`, `refuseDateBucketedGroupBy`). - `git grep -c "StandardErrorCode.enum.NOT_IMPLEMENTED"` → the same 5 + 3. A census of every `StandardErrorCode.enum.X` in the package's source finds no other 501 route: INVALID_FILTER 1, INVALID_QUERY 2, NOT_IMPLEMENTED 8, TIMEOUT 1, VALIDATION_ERROR 4, plus the unsafe-identifier constants `INVALID_REQUEST` / 400. - A wider spelling sweep (`status: 501`, a bare `501` in code position) finds only 2 hits, both in docblock prose. - Positive control: the same `status = 501` grep over `packages/drivers/driver-sql/src` finds its 3 known producers. The built bundle `dist/index.js`, built at `031e3a45e`, carries exactly 8 `status = 501`. - `driver-sql`'s own three 501 producers are reachable only from `SqlDriver.aggregate`, through `mapAggregateFunc`, whose only caller is that method. Remote `TursoDriver.aggregate` never calls `super.aggregate`, so none of them reaches the remote face. - **Not listed, on purpose:** `uncompilableAggregateFunctionError` cannot fire today. `DECLARED_AGGREGATE_FUNCTIONS` (spec `AggregationFunction`: count, sum, avg, min, max, count_distinct) equals `REMOTE_AGGREGATE_FUNCTIONS` key for key, so the declared-but-uncompiled class is empty (`remote-transport-aggregate-function-refusal.test.ts` asserts that). Listing it would name a refusal the code never raises. ## Local verification (all at `031e3a45e`) - **Tests that read this README: 0.** `git grep` for `driver-turso/README` across tests and scripts finds only `package.json` `files`. No test file names the README. - **Build:** `pnpm --workspace-concurrency=2 --filter '@objectstack/driver-turso...' build`, run under the verify lock, ended `VERDICT command-exit 0`. It was needed for the dist-reading gates and the probes. - **The refusal pins the new list draws on, re-run at this head:** 8 files (`turso-remote-transaction-refusal`, `turso-remote-autonumber-refusal`, `turso-remote-deferred-ddl`, `turso-remote-drift-detection-refusal`, `turso-remote-media-column-move-refusal`, `remote-transport-groupby-node`, `remote-transport-aggregate-function-refusal`, `turso-transactions-unsupported-declaration`). Result: `Test Files 8 passed (8)`, `Tests 117 passed (117)`. - **Gates:** `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 51 commands at `031e3a45e`: 41 before the changeset existed, plus 10 changeset families after. I ran all 51 and recorded each exit code. Reconciled with `--ran`: "51 derived famil(ies) accounted for — 49 run, 2 NOT-MEASURED (2 DERIVED from a recorded exit 3)". 49 exited 0, including `check:nul-bytes`, `check:doc-authoring`, `check:issue-citations`, `check:published-files`, `check:dts-closure`, `check:driver-conformance`, `check-changeset-no-major`, `check-empty-changeset` and `check-adr-0087-registration`. - NOT MEASURED: `check:dual-build-cjs-loads`, reason: PREREQUISITE NOT MET (exit 3). It loads every built entry across 78 packages, and only the driver-turso closure is built here. - NOT MEASURED: `check:lean-entry-closure`, reason: PREREQUISITE NOT MET (exit 3). It needs `packages/objectql/dist`. - Neither gate reads a README or a changeset. CI runs both on a full build. - `node scripts/check-issue-citations.mjs --base adbbc5d` → exit 0: "no issue citations added against adbbc5d (0 file(s) read)". - `node scripts/check-published-readme-links.mjs` → exit 0: "178 outbound link(s) across 61 published markdown file(s)". - `node scripts/check-published-readme-exports.mjs` → exit 3, PREREQUISITE NOT MET (39 unbuilt packages). NOT MEASURED. This diff adds no `import` line to any README code block (0 added lines match `import`). ## Changeset `.changeset/20016-turso-readme-remote-refusals.md`, `"@objectstack/driver-turso": patch`. The rule applied: `.claude/agents/os-dev.md` Definition of done, "发布面动了要 changeset;`skip-changeset` 判据是没动:已发布 = 各包 `files[]` 实际发运内容". `README.md` is in `@objectstack/driver-turso`'s `files` (`["dist", "README.md", "CHANGELOG.md"]`), so it ships, which is route 1 of the Check Changeset job ("It releases something -> run 'pnpm changeset'"). Route 2's "docs/" means the repository's `docs/` tree, not a packaged README. Repo precedent: the README-only `patch` changeset in `a90272ae6` (`@objectstack/metadata` README). The level is `patch` because no public surface moves. ## In-flight overlap PR objectstack-ai#20037 (head `5fb4b25b12`, draft) edits `remote-transport.ts`. Its patch changes the filter-refusal doors (`nonListCombinator`, `nonBooleanNullComparand`, `nonBooleanExistsComparand`, `unsupportedOperator` gain a provenance argument). A grep of that patch for `NOT_IMPLEMENTED`, `501` or `StandardErrorCode` finds no changed line. It touches none of the 501 builders and not `aggregate`, so no sentence written here goes false when it lands. ## Out-of-scope findings (for the seat to file; not fixed here) 1. **Class (a): remote mode cannot be constructed without `better-sqlite3`, which the README and the package's own manifest say it does not need.** Probe (end to end, `dist/index.js` built at `031e3a45e`): make `better-sqlite3` unresolvable (`Module._resolveFilename` throws `MODULE_NOT_FOUND` for it), then call `new TursoDriver({ url: 'libsql://probe-db.example.turso.io', authToken: 'x' })`. It throws "Knex: run / $ npm install better-sqlite3 --save". Control: the same construction with the module resolvable returns `transportMode=remote`. Mechanism: the remote arm of `TursoDriver.toKnexConfig` returns `client: 'better-sqlite3'` with a `connection`, then `SqlDriver`'s constructor calls `knex(...)`, whose `Client` constructor calls `initializeDriver()` → `require('better-sqlite3')` (knex 3.3.0 `lib/client.js`). Shipped face: the README's "Dependencies by Mode" row for Remote, "better-sqlite3 is NOT required", "you don't need to install it", the Remote (Cloud-Only) section's "no local SQLite needed", and `package.json` `peerDependenciesMeta.better-sqlite3.optional: true`. Following the Vercel/Edge instructions verbatim ends in the throw above. Dedupe words: `turso remote better-sqlite3 required` · `toKnexConfig remote better-sqlite3` · `driver-turso optional peer better-sqlite3 remote`. 2. **Class (a): the inherited `SqlDriver` methods that `TursoDriver` does not override answer from the placeholder `:memory:` Knex database in remote mode.** Probe (`dist` at `031e3a45e`; remote = `mode: 'remote'` over a libsql `file:` client; control = local `:memory:` face, same two rows): - `introspectSchema()`: remote `[]` (silently empty, although `probe_t` exists and `find` returns 2 rows); local `["probe_t"]`. - `distinct('probe_t','name')`: remote throws `DATABASE_ERROR`/500 ("no such table: probe_t" kept server-side); local `["x","y"]`. - `findWithWindowFunctions`: remote throws a raw `SQLITE_ERROR` with no envelope; local returns 2 rows. - This is the same placeholder shape that `detectManagedDrift` and `planMediaColumnMove` were refused for. The other un-overridden public async methods are `analyzeQuery`, `explain`, `applyMigrationEntries`, `previewDeferredSchemaWork`, `flushDeferredSchemaDdl`, `reclaimSpace`, `rotateShards`, `commitTransaction` and `rollbackTransaction` (not probed). - Dedupe words: `turso remote introspectSchema empty` · `turso remote inherited placeholder knex` · `turso remote distinct no such table`. ## Acceptance notes - **Capabilities table, not changed.** Its header says "TursoDriver declares enhanced capabilities beyond the base SqlDriver". The `supports` getter in `turso-driver.ts` records that those bits (`savepoints`, `queryCTE`, `fullTextSearch`, `jsonQuery`, `connectionPooling`) were retired from `DriverCapabilities`. The table reads as libSQL SQL features reachable through raw `execute`, and I did not measure those over a remote client. Its remote "Savepoints ✅" cell has no driver path, because the driver's savepoints are Knex nested transactions and remote refuses transactions. I left the table alone rather than assert an unmeasured ❌. Carrier: none. - "Ideal for Vercel, Cloudflare Workers …" is not measured. For Vercel/Edge it depends on finding 1. - The probe process in finding 2 did not exit after `disconnect()` and was killed by PID. I did not isolate the cause, and it is not claimed as a finding. --- _Generated by [Claude Code](https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…ugh the withheld provenance seam, so a read scope refused there names no field or literal (objectstack-ai#20039) (objectstack-ai#20093) Fixes objectstack-ai#20039 Clause-②: no ## What changed A read scope is the RLS, sharing or tenant predicate that `plugin-security` (ordinary reads) and `service-analytics` (`ObjectQLStrategy.withReadScope`) AND into the caller's `where`. Both merges mark the scope `'policy'` and the caller's own predicate `'author'` (objectstack-ai#8220, `packages/spec/src/data/filter-subtree-provenance.ts`). The contract for `'policy'`: "A refusal raised from inside it keeps the objectstack-ai#7929 redaction: identity (`INVALID_FILTER` / 400) and capability statement on the wire, operands in the server log." The fail direction: "Unmarked or ambiguous ⇒ withheld." PR objectstack-ai#20037 brought four `SqlDriver` doors under that seam. This PR closes the class: **every refusal on `SqlDriver`'s filter-compile path now goes through `withheldFilterError` with the node it was raised from**, and one pin enumerates the builders so that a new one cannot disclose by omission. **The nine `SqlDriver` builders converted** (each of them called `unsupportedFilterError` directly at base): | builder | the node the seam resolves | |:--|:--| | `icontainsComparandError` | the comparand if it is an object, else the field's operator map | | `likePatternComparandError` | the same | | `danglingLikeEscapeError` | the field's operator map | | `unrenderableTextComparandError` | the comparand (an object or array here) | | `unbindableListMemberError` | the member if it is an object, else the list | | `undefinedComparandError` | the nearest object holding the `undefined`: the node carrying the field, the operator map, or the list | | `assertFilterNode` | the element if it is an object, else the `$and` / `$or` list that holds it, or the node carrying `$not` | | `unknownLogicalOperatorError` | the node CARRYING the key, never its value | | `filterArrayReachedDriverError` | the array itself (it is the `where` root). Not on the card: the enumeration found it, and it printed the whole array | The walk's refusals need the node that encloses the refused position. The spec's shared walk hands its hooks only a value, a key and a path, so `sqlFilterVerdictHooks` recovers the node from the path index it already kept for the combinator door (objectstack-ai#20020). It adds one map of `$and` / `$or` lists, keyed by the combinator key's position. The spec hook signature is unchanged. Per door, what stays on the wire is the class statement: which kind of refusal fired and the capability sentence. The field, the operator variant (`$like` vs `$ilike`, which of the three list operators, which of the five text operators), the comparand, the list index and the filter path go to the server log. `'author'`-marked subtrees get the text the door answered before, byte for byte. The source templates are the same literals, now passed as the diagnostic. Code and status are unchanged on every path (`INVALID_FILTER` / 400), and the set of refused filters is unchanged. **Turso remote transport.** `RemoteTransport.buildWhereSQL` refuses these classes on its own (measured, see below). Its twins now go through `withheldRefusal`, with the same node rules (`refusalNode` is `refusalSubtree`'s twin): `icontainsComparand`, `likePatternComparand`, `danglingLikeEscape`, `undefinedComparand`, `uncompilableSubFilter` (a non-node `$and` / `$or` element or `$not` operand), `undeclaredCombinator` (both of its tails share one withheld sentence) and `uncompilableWhere` (a non-object top-level `where`). The transport's remaining two refusals that named the field go through too, as a bounded in-place fix within this claim's file surface: `emptyFieldFilter`, whose `driver-sql` twin has withheld since objectstack-ai#8197, and `unsupportedOperator`'s `$between` arm. `TursoDriver` never reaches the `$between` arm (it lowers `$between` first). After this, `buildWhereSQL` has no refusal outside the seam. For six classes the withheld sentence is `driver-sql`'s behind the `[RemoteTransport]` prefix, pinned through `TursoDriver` local vs remote. ## Measured: H1–H3 **H1.** The probe (scratch, not committed) ran a real `ObjectQL` over a real `SqlDriver` (better-sqlite3), with a first-registered middleware copying `plugin-security`'s merge and marking. It marks the injected scope `'policy'` and the caller's verbatim `where` `'author'`, then composes `{ $and: [where, scope] }`, or the scope alone when there is no caller `where`. The scope used distinctive names (`policy_stage`, `PSECRET…`). A cell counts as a leak when one of them appears in the response message. | class | base (`b76aad5f6f`) | this branch | |:--|:--|:--| | empty `$icontains` | field | none, field in the log | | non-string `$icontains` | field, comparand | none, both in the log | | non-string `$like` | field, comparand | none, both in the log | | `$like` trailing escape | field, pattern | none, both in the log | | object `$contains` comparand | field, comparand | none, both in the log | | object `$in` member | field, comparand | none, both in the log | | `undefined` comparand (direct, `$eq`, `$in` member) | field | none, field in the log | | non-node element of `$or` / `$not` operand | the literal | none, literal in the log | | undeclared node combinator | the key | none, key in the log | | scope handed over as an array | whole array (field and literal) | none, both in the log | The policy-scope cells are 13 shapes × {with, without} a caller `where`, 26 cells in all. All 26 leaked at base and none leak at this branch. The author arm used the same 13 shapes in the caller's own `where`: 13 of 13 full messages are byte-identical at base and at this branch. (Six of those 13 are refused by the engine's own comparand or `where`-shape door before any middleware runs, and are identical on both sides for that reason.) Base was measured by the whole-file ablation below, which reverted `sql-driver.ts` and rebuilt `dist/`. **H2: the enumeration.** A TypeScript-parser walk over `sql-driver.ts` lists the functions calling `unsupportedFilterError` directly, and the functions calling `withheldFilterError`. - **Base:** 12 direct callers (`grep -c 'unsupportedFilterError('` = 13, including the definition), and 12 seam callers. - **This branch:** 3 direct callers (grep 4): the seam itself, the `'author'` re-issue in `resolveWithheldFilterRefusal`, and `unresolvableFilterColumnError`. That last one is raised from a dialect error after the statement ran, and resolves its own provenance by name (`unresolvableColumnProvenance`, objectstack-ai#20020). 21 seam callers (`withheldFilterError(` grep 22). - **Positive control:** the doors PR objectstack-ai#20037 converted (`retiredFilterOperatorError`, `unsupportedFilterOperatorError`, `assertFilterNodeList`, `nonBooleanNullComparandError`, `nonBooleanExistsComparandError`) are in the base seam set, and its column door is in the direct set. - **The eighth door (H1's PM-grep guess):** `assertFilterNode` is confirmed. The enumeration adds a ninth, `filterArrayReachedDriverError`. `RemoteTransport`: base has 11 direct `invalidFilterError` callers and this branch 2, the seam and the re-issue. The seam callers go from 5 methods to 13. **H3.** Measured on `RemoteTransport` with a `'policy'`-marked `where`. The transport refuses these on its own, naming `'object.field'` and the comparand: `$icontains`, non-string `$like`, trailing escape, `undefined` comparand, non-node `$or` element and `$not` operand, undeclared combinator (including a misplaced field operator), non-object `where` (the whole array), empty operator map, un-lowered `$between`. An object `$contains` comparand and an object `$in` member were already withheld there: both are `uncompilableComparand`, which objectstack-ai#8197 converted. Through `TursoDriver` in REMOTE mode, `toRemoteFilter` rebuilds every node, so no mark reaches the transport. Every mark-reading refusal there withholds for every caller, author included. That is pinned as the declared fail-closed cost. **H4: consumer pins.** Two sweeps, over test files in `packages/`, `examples/` and `apps/`: - **Wording grep:** the nine author texts' distinctive fragments and the remote twins'. 24 files, of which 9 are outside the three driver packages: `driver-memory` / `driver-mongodb` (their own wording) and a `packages/spec` test (its own). Control: the re-pinned `driver-sql` / `driver-turso` files are among the 24. - **Assertion-shaped grep** over the test files outside the three drivers that reference a SQL-family driver and `INVALID_FILTER`: 55 files, 17 of which construct a real SQL-family driver. Every hit is `service-analytics`' own normalizer or read-scope compiler wording (`comparand at … is undefined`, `normalizeAnalyticsFilterTree`), `plugin-auth`'s `$regex` face-3 pin (the retired door, unchanged here), or a negative assertion. **No consumer pin outside the three driver packages reads these doors' wording.** None was changed, and the suites PR objectstack-ai#20037's round 2 named are green (below). ## Compile surfaces (per `references/compile-surfaces.md`, located again at this head) | # | face | this PR | |:--|:--|:--| | 1 | `driver-sql` `applyFilterCondition` (`sql-driver.ts:15907`); `driver-sqlite-wasm` and Turso local by inheritance | CHANGED: nine builders, and the class is closed | | 2 | Turso `RemoteTransport` `buildWhereSQL` (`remote-transport.ts:2667`) | CHANGED: seven twin classes plus the empty-operator-map and `$between` arms; class closed | | 3 | service-analytics `compileScopedFilterToSql` (`read-scope-sql.ts:582`) | not touched: outside the claim; its refusals are the withheld `READ_SCOPE_COMPILE_FAILED` / 500 | | 4 | service-analytics `lowerAnalyticsWhere` (`filter-normalizer.ts:1966`) | not touched (outside the claim) | | 5 | `formula` `matchesFilterCondition` (`matches-filter.ts:212`) | not touched, not measured | | half | objectql `applyHaving` / `matchesHaving` (`having-filter.ts:279` / `:292`) | not touched, not measured | | — | `driver-memory` / `driver-mongodb` | not touched; they do not read the mark, and disclosure there is not measured | ## Tests **New pins:** - `packages/drivers/driver-sql/src/sql-driver-compile-refusal-seam.test.ts`, 91 cases: - the enumeration (positive control, direct-caller set, seam-caller set equal to the table); - 21 builders × {policy, author, unmarked, merged `$and` in both arm orders}, where the array-root builder has no merged arm. Each row is bound to its builder by the error's own stack; - five node-lookup cases. - `packages/drivers/driver-turso/src/remote-transport-compile-refusal-seam.test.ts`, 96 cases: - the same enumeration over `remote-transport.ts`; - 13 methods plus 7 extra arms under every mark; - two node cases; - `TursoDriver` local vs remote: the withheld sentences of six classes are one sentence, and REMOTE withholds an author-marked `where`. Per door, the pins assert: policy-marked gives `INVALID_FILTER` / 400, the error's own keys exactly `code,status`, no secret in the message, and every secret in the log or sink; author-marked gives the full text, which the policy case logged; unmarked is byte-identical to policy. **Existing pins that read a converted door's full text through an unmarked `where`.** Each now marks it `'author'`, as a merge boundary marks a caller's own predicate, on a shallow copy where the case is a shared constant: - `driver-sql`: `filter-refusal-envelope` (array cases), `not-null-safe`, `out-of-contract-filter-input`, `silent-empty-predicate` (its objectstack-ai#7929 comment was updated to say the family is now narrowed too), `undefined-comparand-refusal`. - `driver-turso`: `bare-date-routing`, `boolean-identity`, `node-operator-refusal`, `not-operator`, `top-level-where`, `undefined-comparand-refusal`. - `driver-sqlite-wasm`: `out-of-contract-filter-input`, which also gains an unmarked withheld leg. Two remote shapes cannot get the author text back, and their pins say so: - a PRIMITIVE top-level `where` cannot carry a mark, so it is withheld for everyone and its pin asserts the class statement; - a refusal raised inside `$not` resolves against the NULL-safe REWRITE of the operand, which the resolver cannot find under the root. It is withheld even for an author, fail-closed by construction as `driver-sql` documents for its own `$not` rewrite. Those pins read the naming half from the diagnostic sink. `turso-local-remote-text-parity.test.ts`: a withheld remote answer must still name none of the case's mentions, except the ones the refusal's CLASS statement carries. The class statement is read off the local face's unmarked answer; for the `$icontains` rows it is `$icontains` itself. **Suites.** Driver suites at `8b4f1bb3ed` (after merging `main`, which brought objectstack-ai#20054's `turso-driver.ts` change), with `driver-sql` and the driver closures rebuilt: - `driver-sql`: 185 files passed, 11 skipped; 2936 tests passed, 170 skipped. - `driver-turso`: 68 files, 1606 tests passed. - `driver-sqlite-wasm`: 32 files, 606 tests passed. - `typecheck` exits 0 for all three. `tsc --listFiles` counts 1 each for the two new pin files and the changed sqlite-wasm pin. Consumer suites at `ab61f7a832`: the same driver code, before the `main` merge. The merged commits touch no `driver-sql` path, and in `driver-turso` only `turso-driver.ts` (objectstack-ai#20054's constructor seam). `driver-sql` and `driver-turso` `dist/` were rebuilt first. - `plugin-auth`: 114 files, 2440 tests passed. - `service-storage`: 40 files, 627 tests passed. - `cloud-connection`: 30 files, 396 tests passed. - `plugin-dev`: 8 files, 80 tests passed. - `connector-mcp`: 3 files, 23 tests passed. - `hono`: 5 files, 122 tests passed. - `cli` unit project (`--project unit`): 224 files, 3158 tests passed. The integration tier is declared to CI; this diff touches no cli file. - `plugin-security`: 133 files, 2648 tests passed. - `plugin-sharing`: 37 files, 913 tests passed. - `service-analytics`: 120 files, 2689 tests passed. **Ablations.** All ran on committed state. Each restore is proved by the blob equal to `HEAD` and an empty `git diff HEAD`. 1. **`sql-driver.ts` at the base blob** (`463b99cf41`; marker 1 → 0), `driver-sql` rebuilt, and `ablation-dist-preflight --absent` passed. The new seam pin gave 41 failed, 50 passed: - red: both enumeration assertions, the 35 policy/author/unmarked/merged cases of the nine builders, and four node cases; - green: the positive control, the 12 already-converted builders × 4, and the author-arm node case, which base satisfies by disclosing to everyone. The five re-pinned `driver-sql` files stayed 110/110 green at base, so the author text they read is unchanged. The same run produced the base column of the H1 table. Restored to blob `a89054ee3a`, rebuilt, and the preflight found the marker in `dist/`. 2. **`remote-transport.ts` at the base blob** (`55148ccc94`): the remote seam pin gave 67 failed, 29 passed. It imports `src`, so no build leg was needed. 3. **The `classifyKey` enclosing node replaced with `undefined`** (`ablation-replace.mjs`, anchor 1 → 0): 3 failed, exactly the unknown-combinator author and merged legs and the direct-`undefined` node case. 4. **The `assertNode` enclosing replaced with `undefined`:** 4 failed, the non-node element's author and merged legs and the two primitive-inheritance node cases. 5. **One builder bypassing the seam** (`icontainsComparandError` calling `unsupportedFilterError`): 6 failed, both enumeration assertions and its four rows. This is the red a new bypassing builder gets. ## Gates - `dispatch-gates --commands --repo objectstack-ai/objectstack` at `8b4f1bb3ed` (merge base `249172975b`, 18 paths) derived 62 families. They are the PM's 55 plus the seven that apply once the changeset exists. - All 62 exited 0. `check:dual-build-cjs-loads` first exited 3 (PREREQUISITE NOT MET), then 0 after a full `turbo run build --filter=./packages/* --filter=./packages/*/*` (72 of 72 tasks cached). The other dist-reading families were re-run on that build and exited 0. - `--ran`: "62 derived, 62 run, 0 NOT-MEASURED, 0 UNRUN (a DERIVED zero)". - The three roster families under `packages/` that the derivation flags as unreadable exited 0: `check:authz-resolver`, `check:error-code-casing` and `check:filter-alias-parity`. - `check-issue-citations --base 2491729`: exit 0, 56 citations resolve. - `check:driver-conformance`: "50 covered cell(s), 0 in the DEBT ledger, 0 exempt" at base (`b76aad5f6f`, read from an archived tree) and at this head. The ledger did not move. - ESLint (`--no-inline-config --format json`) on the 17 changed `.ts` files: 17 files, 0 errors, 0 warnings. `eslint.config.mjs` sets no `parserOptions.project` (the printed config for `sql-driver.ts` is `{"ecmaVersion":"latest","sourceType":"module"}`), so this diff cannot move a verdict on an untouched file. The repo-wide run is CI's. - A control-byte scan of the changed files found no hits. ## Acceptance notes - **What an unmarked caller loses:** its own diagnostic from these nine refusals. It gets the withheld wording with the same code and status. This covers no security plugin, system context, anonymous calls, a `where` holding a `{placeholder}` token, and Turso remote mode. A member's plain `where` under `plugin-security` is marked `'author'` and keeps the full text. The changeset says this. - **The two node-shape refusals inside `$not` on the remote transport** (`uncompilableSubFilter`, `undeclaredCombinator`) resolve against the rewritten operand, so they withhold even for an author-marked `where`. The comparand doors there still answer an author in full. *(Narrowed by the seat from contract review 5826632400.)* PR objectstack-ai#20037's `nonListCombinator` has the same limitation inside `$not`. It is fail-closed, and invisible in production, where `TursoDriver` never hands the transport a mark. - **Two findings are reported to the seat and not fixed here.** Both are outside this claim's file surface or class: - `TursoDriver`'s remote `$between` lowering (`toRemoteFieldSpec`) refuses a non-two-element range with a bare `Error` carrying no `code` / `status`. Local mode answers `INVALID_FILTER` / 400. - A second finding, in `RemoteTransport`'s handling of `where` keys, is routed to the maintainer. *(The seat removed its detail from this body before landing, pending the maintainer's decision.)* --- _Generated by [Claude Code](https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…dmission before composing it (objectstack-ai#20232) Fixes objectstack-ai#19995 Clause-②: no The analytics ObjectQL face now asks the engine's own `where` admission, `IObjectQLEngine.judgeFilter` (objectstack-ai#20157, ruling C), about each row-level read scope on its own, before composing it into the `where` it hands `executeAggregate`. A scope the engine refuses is refused in the withheld `READ_SCOPE_COMPILE_FAILED` / 500 (objectstack-ai#5367). A scope the engine serves is still served. The caller's own `where` keeps the engine's answer. The close condition in the ruling is met on the final head: both analytics HTTP doors were re-measured over every class the ruling names (the four here, the eleven withheld by PR objectstack-ai#20017 / objectstack-ai#20046 / objectstack-ai#20072, and the four `driver-sql` doors from PR objectstack-ai#20037), and no response body carries policy content. ## What changed All in `packages/services/service-analytics/src/`. - **`read-scope-sql.ts`: new `assertReadScopeAdmittedByEngine(scope, objectName, context, host)`.** It calls the host's judge on the scope alone, under the verb every engine-bound merge runs (`'aggregate'`) and the context that merge forwards. - An `ok: false` verdict is raised through the module's one envelope helper, `readScopeCompileError`. The engine's sentence stays in the thrown message for the operator's log, and the 500 declaration withholds it on the wire. The verdict's own `code` / `status` describe a caller's mistake, so they do not travel either. - A throw from the judge itself (a fault, not a verdict) is raised in the same envelope. - No judge, or an `undefined` answer, means the scope is not judged here ("cannot answer, do not block"). - It is exported from the file only. The package entry is unchanged. - The header gains a ruling-C section, and the paragraph that said these doors were out of reach is updated. - **`strategies/objectql-strategy.ts`: called at both engine-bound merges**, `withReadScope` (direct path and cross-object base aggregate) and `resolveFkAttr` (the referenced object's scope). It runs after the existing guards (vacancy, comparand faces, placeholders), so a scope they refuse keeps their sentence. It runs before the `'policy'` mark, like them. - **`strategies/types.ts`: `DatasetScopedStrategyContext.judgeFilter?`**, the package-local hook, typed from the contract member itself (`IObjectQLEngine['judgeFilter']`, made non-nullable) plus the `undefined` answer. This is the `declaredFieldType` / `sqlDialect` pattern. - **`analytics-service.ts`: `AnalyticsServiceConfig.judgeFilter?`**, passed to the strategy context untouched. A service configured with no judge logs one `warn` on its first unjudged scoped merge, naming the consequence and the remedy. - **`plugin.ts`: the judge is wired to the engine the `executeAggregate` auto-bridge executes on**, resolved per call through the same `tryGetDataEngine`. It is wired ONLY when the plugin bridges `executeAggregate` itself. The judge must be the executor, or it would refuse scopes the executor serves, and a host that supplies its own `executeAggregate` has not said which engine that is. - A `data` engine without `judgeFilter`: `undefined`, plus one `warn` from the plugin. - No engine at all: `undefined`, silently, because the executor refuses that query itself. - **`plugin.ts`, record-label fetch (the objectstack-ai#14329 door): the same checks as `resolveFkAttr`.** This is a fourth engine-bound merge. It `$and`s the referenced object's scope into `executeAggregate` to turn a lookup dimension's ids into labels, and it ran only the vacancy guard. Measured before this change: on the dataset door, a selection ordered by a lookup dimension runs the sort-key label pass, and that pass relayed the engine's 400 with policy content. It did so for the residue classes and also for classes every other merge already withheld (a list in the equality slot, an unknown placeholder). It now runs the comparand faces, the placeholder resolver and the engine's admission on the scope alone. See "Scope" under Acceptance notes. ⛔ **Not a catch around `executeAggregate`.** The caller's own `where` is never judged here. Pinned, and ablation E6 shows those pins turning red under a blanket catch. **Why a served scope stays served.** The judge is the executing engine, under the same verb and context. `judgeFilter` runs the engine's two admission stages, the same functions in the same order execution runs, and stops before any driver. Every object-form door judges a node against the field map and the context, never against its siblings. So the scope alone is admitted exactly when the scope inside `{ $and: [userFilter, scope] }` is. Ablation E8 turns the served-placeholder control red when the judge reads a different context. ## Premises, measured before writing the fix 1. **The contract member and its implementation** (`objectql-engine.ts:306`, `engine.ts:8783` on `ce70876e4c`). Read, and measured: for one refused filter, `judgeFilter(..., { operation: 'aggregate' })` returned the same `code`, `status` and message string that `aggregate` raised. 2. **What the `data` service hands out.** In a booted `LiteKernel` with `ObjectQLPlugin` and `AnalyticsServicePlugin`, `getService('data')` is the same object as `getService('objectql')`. It is an `ObjectQL` instance, and `typeof judgeFilter` is `'function'`. It is not a wrapper. 3. **The four classes on current main.** They relayed policy content on both doors at base `ce70876e4c` (table below). 4. **The verb.** `'aggregate'` reproduces the execution message exactly (premise 1). The verb changes only the message prefix, never the verdict. 5. **The existing guards.** Kept. An unwired host relies on them, and the pins below show such a host still withholds a guarded class while the four residue classes keep today's engine 400. ## Measurement: both analytics HTTP doors, before and after **How.** A scratch probe, never committed, lived in `packages/runtime/src` only while it ran. - **Kernel:** a real `LiteKernel` booted with `ObjectQLPlugin` and `AnalyticsServicePlugin`. The plugin auto-bridged `executeAggregate`, and after the fix it wired the judge, to a real `ObjectQL` over `SqliteWasmDriver`. The plugin options supplied `getReadScope` and `admitObjectRead`, and fixed `queryCapabilities` to the ObjectQL face. Nothing else was stubbed. - **Doors:** `@objectstack/runtime`'s dispatcher, `POST /api/v1/analytics/query`, and `@objectstack/rest`, `POST /analytics/dataset/query`. - **Runs:** before = base `ce70876e4c`; after = this branch with the `service-analytics` dist rebuilt (the new sentence is present in `dist/index.js` and `dist/index.cjs`). - **"Policy content"** = the synthetic policy field name or comparand appears anywhere in the response body. **"Log"** = the refusal's detail reached the door's error-log channel. | Read-scope class | Both doors, before | Policy content in body, before | Both doors, after | Policy content in body, after | Detail in the server log, after | |---|---|---|---|---|---| | Text operator over a non-text field | `INVALID_FILTER` / 400 | yes | `READ_SCOPE_COMPILE_FAILED` / 500 | no | yes | | Temporal comparand the field cannot interpret | `INVALID_FILTER` / 400 | yes | 500 | no | yes | | Filter on a virtual (formula) field | `INVALID_FIELD` / 400 | yes | 500 | no | yes | | Dotted path through a lookup | `INVALID_FIELD` / 400 | yes | 500 | no | yes | | A residue class in the BASE scope on the cross-object path | 400 | yes | 500 | no | yes | | A residue class in the REFERENCED object's scope (text operator; dotted path into a scalar) | 400 | yes | 500 | no | yes | | The nine comparand classes (PR objectstack-ai#20017 / objectstack-ai#20046): list in the implicit equality slot, list under `$eq`, scalar under `$in`, scalar under `$nin`, one-bound `$between`, plain-object member in `$in`, `undefined` comparand, plain-object comparand under `$eq`, `null` member in `$in` | `READ_SCOPE_COMPILE_FAILED` / 500 | no | unchanged | no | yes | | The two placeholder classes (PR objectstack-ai#20072): unknown placeholder; known placeholder the context cannot resolve | 500 | no | unchanged | no | yes | | Refused `$icontains` comparand (objectstack-ai#20068) | 500 | no | unchanged | no | yes | | The four `driver-sql` doors (PR objectstack-ai#20037): missing column; retired or unknown operator; combinator with a non-array operand; non-boolean `$null` / `$exists` | `INVALID_FILTER` / 400, withheld | no | unchanged | no | unchanged | | Record-label fetch, sort-key pass (dataset door): a residue class on the referenced object | 400 | yes | 500 | no | yes | | Record-label fetch, sort-key pass (dataset door): list in the equality slot; unknown placeholder | 400 / `FILTER_TOKEN_UNKNOWN` 400 | yes | 500 | no | yes | The `/analytics/query` door has no label pass. The display label pass catches a failed fetch and renders raw ids: 200 before and after, with the detail in the `warn` log. **Controls, identical before and after:** - A well-formed scope answers 200 with exactly its rows. - A scope with a placeholder the context resolves answers 200 on the dataset door. The dispatcher harness carries no user, so that door answers the withheld 500 both before and after. - A well-formed referenced-object scope buckets what it hides as `(restricted)`. - A well-formed referenced scope on the label pass is served. - The caller's own `where` in each of four shapes (text operator on a number field, uninterpretable temporal comparand, virtual field, dotted path) answers its 400 on both doors, and the body carries the caller's own diagnostic. ## Tests New file: `src/__tests__/objectql-read-scope-engine-admission.test.ts`, 19 cases. Each builds `AnalyticsServicePlugin`'s own composition over a real `ObjectQL` + `SqliteWasmDriver` as its `data` service. Only `queryCapabilities` is fixed to the ObjectQL face. - **Refusal pins** assert `code` `READ_SCOPE_COMPILE_FAILED` and `status` 500. They also assert the two reads every analytics HTTP door takes before relaying prose: `serverFaultProvenance(resolveThrownHttpError(err, 500))` is `'declared'`, and `declaredRefusalMessage(err)` is undefined. The thrown message, which is the log channel, still names the detail. - The four classes on the direct path. - A well-formed caller `where` beside a refused scope. - The cross-object base scope, and the referenced-object scope. - The record-label sort-key pass. - A judge that throws. - **Preservation pins:** - A well-formed scope, and a placeholder the forwarded context resolves, are served with exactly their rows. - A well-formed referenced scope keeps its `(restricted)` bucket. - The label pass with a well-formed scope is served, sorted by label. - The caller's own `where` (text operator, temporal comparand, virtual field) keeps `INVALID_FILTER` / `INVALID_FIELD` / 400 with its message and no server-fault declaration. - **Unwired tiers:** - `AnalyticsService` with no judge keeps the engine's 400 for a residue class, still withholds a guarded class, serves a well-formed scope, and logs exactly one line across three queries. - A plugin host with its own `executeAggregate` is not wired to a guessed engine: a residue class keeps the engine's 400. At the record-label fetch, the comparand and placeholder guards this PR adds there withhold a guarded class and an unresolvable placeholder; they are new on that host too. - A `data` engine without `judgeFilter` keeps the 400, and the plugin logs exactly once across two queries. **Results on the final head `f6dbebe5`:** - `pnpm --filter @objectstack/service-analytics test`: `Test Files 129 passed (129)`, `Tests 3041 passed (3041)`. - `pnpm --filter @objectstack/service-analytics typecheck`: exit 0. `tsc --noEmit --listFiles` includes the new test (count 1). - **Downstream consumers**, run because the wire envelope of already-refused scopes moves. Each run was against the rebuilt `service-analytics` dist: - `@objectstack/rest`: 10 `analytics-*` files, 148 tests green. - `@objectstack/runtime`: the 19 test files that touch analytics, 566 tests green. - `@objectstack/dogfood`: the 6 analytics-touching files, 36 tests green. These boot the real stack, where the judge is wired, and include the label-scope and RLS suites. - `@objectstack/client`: the analytics test, 7 green. ## Ablations Each leg ran from committed state through `scripts/ablation-replace.mjs` in WRAP mode, against the new test file. The anchor had to hit exactly once and the blob had to change. Every restore was proven: the blob equals HEAD, and `git diff HEAD` is empty. An outer shell trap restored all four source files from `HEAD` on any exit. The subject is imported relatively from `src`, so no dist is involved. Every direction was predicted before the run; E1 reddened two more pins than predicted (below). | Leg | Mutation | Result | |---|---|---| | E1 | Delete the `withReadScope` judge call | 9 failed. Predicted 7 (the four classes, the scope beside a caller `where`, the cross-object base scope, the throwing judge). Also red: both once-log pins, which need that call to ask at all. | | E2 | Delete the `resolveFkAttr` judge call | 1 failed: the referenced-object pin | | E3 | Delete the label-fetch judge call | 1 failed: the label sort-key pin | | E4 | Delete the label-fetch comparand guard | 1 failed: the unwired plugin host's label pin | | E4b | Delete the label-fetch placeholder guard | 1 failed: the same test's placeholder assertion | | E5 | Wire the judge from the `data` engine even when the host supplied its own `executeAggregate` | 1 failed: "not wired to a guessed engine" | | E6 | Wrap the direct `executeAggregate` in a blanket catch re-raised as the withheld 500 | 6 failed: the three caller-`where` pins and the three unwired-tier pins that expect the engine's 400 | | E7 | Judge the COMPOSED tree instead of the scope alone | 3 failed: the caller's own `where` was misattributed as the scope's 500 | | E8 | Judge with no context instead of the forwarded one | 1 failed: the served-placeholder control was refused | | E9 | Drop the service's once-flag | 1 failed: two warn lines | | E10 | Drop the plugin's once-flag | 1 failed: two warn lines | | E11 | The service never logs the missing judge | 1 failed: zero warn lines | | E12b | Relay the engine's verdict as-is (its code, status and message) | 8 failed: every judge-dependent refusal pin | The first E12 attempt was a no-op: its replacement contained its own anchor, so the anchor count stayed at 1, the tool refused, and no test ran. It was re-run as E12b with a replacement that does not contain the anchor. ## Gates - **Derived gates.** `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands` on `f6dbebe5` derived 61 commands, the same count as the dispatch-time list. All 61 exited 0, each exit code captured right after a single redirect. The `--ran` reconciliation read: `61 derived, 61 run, 0 NOT-MEASURED, 0 UNRUN (a DERIVED zero — all 61 recorded an exit code and none of them is 3)`. - `check:dual-build-cjs-loads` first exited 3 (PREREQUISITE NOT MET). After `turbo run build --filter=./packages/* --filter=./packages/*/*` (71/71 tasks) it exited 0. `check:dts-closure`, `check:sourcemap-no-sources-content`, `check:published-files` and `check:lean-entry-closure` were re-run on that build and exited 0. - `check-plugin-teardown-shape.mjs --self-test` first exited 3: the shallow checkout could not reach its pinned positive-control commit. After fetching that one commit it exited 0. - **Outside the derivation, run because the diff adds a `warn` in `plugin.ts` and in the service:** `check:startup-registry-verdict` 0, `check:durability-log-level` 0. - **Issue citations.** `node scripts/check-issue-citations.mjs`: 28 citations across 5 files, all resolve. - **Lint, narrowed to the 6 touched TypeScript files.** - `eslint --no-inline-config --format json` reported 6 files, 0 errors, 0 warnings, none ignored. - `eslint --print-config` for each file shows `parserOptions` limited to `ecmaVersion` / `sourceType`, with no `project` and no `projectService`. Type-aware linting is off, so this diff cannot move a verdict on an untouched file. - `pnpm lint` itself is CI's. ## Acceptance notes - **Exported types.** `AnalyticsServiceConfig` (exported from the package index) gains one optional member, `judgeFilter`. Its type, `ReadScopeFilterJudge`, is exported from `strategies/types.ts` only, not from the index; it reaches the published declarations through that member. `DatasetScopedStrategyContext` (not exported) gains `judgeFilter`. `AnalyticsServicePluginOptions` is unchanged. Changeset: `@objectstack/service-analytics` patch. - **Wiring.** The plugin wires the judge only when it auto-bridges `executeAggregate`. That is how every shipped composition boots (`os serve`'s capability provider, the verify harness): no host in this repository passes its own `executeAggregate`. A plugin host that does keeps today's behaviour, and logs one `warn`, everywhere except the plugin's record-label fetch, whose new comparand and placeholder guards run on every host that uses it (see Scope below). A host constructing `AnalyticsService` directly can pass `judgeFilter`, from the engine its `executeAggregate` runs on. - **Precedence.** When the caller's `where` and the scope are both refused, and only the engine would refuse the caller's clause, the scope's 500 answers first. This is the same precedence PR objectstack-ai#20017 and PR objectstack-ai#20072 recorded. - **Scope: the record-label fetch.** The fourth merge is repaired in place: same defect class, a mechanical repeat of the `resolveFkAttr` form, a file on the claim's surface, and no new gate family. On a host whose own `executeAggregate` runs on something other than ObjectQL, the label fetch now refuses off-contract scope shapes that such an executor tolerated. That is the same note PR objectstack-ai#20017 carried for `resolveFkAttr`; the ObjectQL executor refused every one of them already. The display label pass's catch is unchanged. - **The once-lines** are per service instance and per plugin instance, emitted on first use, never at init. They show up once per test file that builds a service without a judge. - **`origin/main`** moved by one commit after the base, `805af4f2` (`packages/cli` only). It shares no path or behaviour with this diff and is not merged. - **Observation, not filed (zero pull).** In a kernel with no `security` service, the analytics object-read admission bridge answers `PERMISSION_DENIED` / 403 on every query. The kernel's `getService` throws for a missing service, and the bridge reads a throw as "unusable" (fail-closed). Its init warning describes the opposite. Every shipped composition includes `SecurityPlugin`, and the failure direction is fail-closed. Measured incidentally by the probe. - **Files not touched:** `filter-normalizer.ts`, `preview-evaluator.ts`, `text-match-sql.ts`, `native-sql-strategy.ts`, `packages/objectql`, `packages/spec`. ## Seat append — patch round 1 (head `d9a1002f`) - The at-tier contract review of `f6dbebe5` (record `5856105439`) found two overclaims in the changeset prose. The dev corrected them, plus two adjacent imprecisions, in `d9a1002f` (`.changeset` only, +3/−3). - Two sentences of this body carried the same overclaim as the review's defect 2: the Tests "Unwired tiers" bullet and the Acceptance-notes "Wiring" sentence. The seat corrected both in place, using the dev's text from its round-1 report. No other byte of this body changed. --- _Generated by [Claude Code](https://claude.ai/code/session_01TEah6PeJGjxJfbHaySJjLQ)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Part of #20020. This PR covers the four
driver-sqlrefusal doors and their twins in the Turso remote transport. The three engine doors the card also names are not changed: measured, they see no provenance mark on any host except the analytics face, so applying the rule there would hide every CRUD caller's own diagnostic. That needs a decision; see "The engine half" below.Clause-②: no
What changed
A read scope is the RLS, sharing or tenant predicate that
plugin-security(ordinary reads) andservice-analytics(ObjectQLStrategy.withReadScope) AND into the caller'swhere. Both merges mark the scope'policy'and the caller's own predicate'author'(#8220,packages/spec/src/data/filter-subtree-provenance.ts). The contract for'policy': "A refusal raised from inside it keeps the #7929 redaction: identity (INVALID_FILTER/ 400) and capability statement on the wire, operands in the server log." And the fail direction: "Unmarked or ambiguous ⇒ withheld."Four
SqlDriverrefusal doors did not read the mark. They now go through the seam the cross-field (#7929) and target-field (#8197) refusals already use:withheldFilterError, resolved inapplyFilters/withWithheldFilterLogwithresolveFilterSubtreeProvenance.default:arm):retiredFilterOperatorErrorand the newunsupportedFilterOperatorErrorhand over the field's operator map as the node.$null/$exists(classifyFilterKey): the field's operator map is the node.$and/$orwhose operand is not a list (assertFilterNodeList): the operand is the node when it is an object. A primitive operand is judged by the node that carries the key. The shared walk in@objectstack/spechands that hook only the operand, the key and the path, sosqlFilterVerdictHooksrebuilds the node from the path. It records every node the walk announces throughassertNode, keyed by paths made only of combinator keys and list indices. The spec hook signature is unchanged.unresolvableFilterColumnRefusal, raised after the statement ran): the dialect reports only a column name.unresolvableColumnProvenancefinds every node in the query'swherewhose key (as written, throughmapSortField, or through an external column map) or{ $field }names that column. The name reaches the wire only when there is at least one such node and every one resolves'author'. Otherwise the refusal uses the unnamed wording the method already had for a dialect message it could not parse. The four callers (find,count,aggregate,distinct) now pass theirwhere.Code and status are unchanged on every path:
INVALID_FILTER/ 400. The set of refused filters is unchanged. An'author'-marked subtree gets the text the door answered before. The source templates are the same literals, now passed as the diagnostic and the author text.Turso remote transport (the claim's conditional arm).
RemoteTransporthas its own copies of three of these doors, and they did not read the mark either:unsupportedOperator, for a retired or unknown operator. The same method's arm for a non-operator key in an operator map names the target the same way and is converted with it.nonListCombinator.nonBooleanNullComparand/nonBooleanExistsComparand.They now go through
withheldInvalidFilterError, and the newwithheldRefusalhelper sends the diagnostic to the sink. The transport has no refusal for an unresolvable WHERE column:findanswers[]. That door is not touched here.pushLikeis not touched.Measured: what each 400 body carried, before → after
Probe (scratch, not committed): a real
ObjectQLoverSqliteWasmDriver(aSqlDriversubclass). The analytics face went through the realPOST /api/v1/analytics/queryroute:AnalyticsServicein ObjectQL-only mode, withgetReadScopereturning the scope. The CRUD face went throughql.findunder a middleware that copiesplugin-security's merge and marking code. Before = base26550c660. After = this branch, withdriver-sqldist rebuilt. The scope used distinctive names (policy_stage,PSECRET…), and a row counts as a leak when one of them appeared in the response message.$regex)$null$exists{placeholder}@objectstack/coreresolveFilterTokensvia engineresolveWhereTokensINVALID_FILTER/ 400. The server log carried the field and the comparand.'author'by the merge returned the same message before and after for all ten classes (first 220 characters compared). On the analytics face, the caller's ownwhereanswers were unchanged.SecurityPlugin, a CEL rulepolicy_unsynced == 'PSECRET1u'on a field that is declared but has no column: before, a 400 namingpolicy_unsynced; after, the unnamed wording. A CEL rule on an undeclared field answered 0 rows with no refusal, before and after.Host × mark: which marks a refusal door sees
How each column was read:
SecurityPlugin(a CELusingrule) over a realObjectQL+SqliteWasmDriver, reading thewherethe driver received.SecurityPluginrows were not recorded separately at that stage.withReadScope, and consistent with the after-probe: the caller's arm discloses and the scope withholds.whereplugin-security, member, plainwhere'author''policy'plugin-security, member, nowhere'policy'plugin-security, system contextplugin-security, anonymousplugin-security,whereholding{current_user_id}'policy'plugin-security, FilterArraywhere'author'(the lowered copy is vouched)'policy'withReadScope'author', scope'policy''author''policy'TursoDrivertoRemoteFilterrebuilds every node, measured by an'author'-markedwheregetting the withheld textplugin-sharingwas not measured end to end.What an unmarked caller loses at the four driver doors: its own field, operator, comparand and path. It gets the withheld wording with the same code and status. Measured cases: no security plugin, system context, anonymous, a
whereholding a placeholder token, and Turso remote mode. A member's plainwhereunderplugin-securitykeeps the full text.The engine half: not in this PR
text-operator-declared-type-door.ts,temporal-comparand-door.tsand the unknown-placeholder refusal (UnknownFilterTokenErrorfrompackages/core/src/utils/filter-tokens.ts, called by the engine'sresolveWhereTokens) all run on the caller'swherebefore the middleware chain. The first column of the host table is the measurement: on every host except the analytics face, thewherethese doors see carries no mark at all. Under "unmarked ⇒ withheld", every REST, SDK and flow caller would lose what these three doors name for its OWN filter:$containsfamily) over a field whose DECLARED type is not textual —INVALID_FILTER400 at the engine's field-aware door (option C of #14079); the textual-type vocabulary is the question #15661 ruling asks for both);{placeholder}is correctly rejected 400 (17.0.0 GA) #8690);On the CRUD face a read scope never reaches these doors (measured above), so they disclose policy content only on the analytics face. Options and a recommendation are in the
os-dev-reporton #20020. The seat took option C in claim amendment 5823437510: the analytics-face pre-judge belongs to the services lane (pointer 5823452098 on #19995), and this PR does not change these three doors.Compile surfaces (per
references/compile-surfaces.md)driver-sqlapplyFilterCondition(anddriver-sqlite-wasm/ Turso local, by inheritance)RemoteTransportbuildWhereSQLpushLikeis untouched, andunsupportedOperator's$betweenarm still names its target, which cannot be reached from a policy throughTursoDrivercompileScopedFilterToSqlREAD_SCOPE_COMPILE_FAILED/ 500lowerAnalyticsWhere$null/$existsis #20040formulamatchesFilterConditionapplyHaving/matchesHavingdriver-memory/driver-mongodbTests
packages/drivers/driver-sql/src/sql-driver-refusal-door-provenance.test.ts(34 cases) andpackages/drivers/driver-turso/src/remote-transport-refusal-door-provenance.test.ts(29 cases). Per door, they cover:$and.$not.'author': these pins read a door's author-facing text on an unmarkedwhere, so theirwhere(or a shallow copy of a shared case constant) is now marked'author', as a merge boundary marks a caller's own predicate.driver-sql: 11541, 17857, backend-fault-envelope, boolean-identity, filter-no-silent-drop, filter-refusal-envelope, icontains-and-retired-operators, null-operators (two assertions moved to code and status), out-of-contract-filter-input, text-case-conformance, unresolvable-where-column-refusal.driver-turso: bare-date-routing, comparand-refusal, node-operator-refusal, not-operator, null-comparand-refusal, text-predicates.driver-sqlite-wasm: icontains-and-retired-operators, which is outside the claim's file list, becauseSqliteWasmDriverinherits these doors.sql-driver-target-field-provenance.test.ts"an UNDECLARED operator still names its operator and field". It stood so that this widening would be a deliberate act, and it now pins both halves.remote-transport-text-predicates.test.ts($regexthroughTursoDriverremote) andturso-local-remote-text-parity.test.tsnow assert the withheld wording on the remote face, becausetoRemoteFilterdrops the mark. A withheld remote answer must name none of the case's mentions.Suites at head
3bf7a68ce(after mergingmain, which brought #20019):driver-sql: 183 files passed, 11 skipped; 2771 tests passed, 170 skipped.driver-turso: 65 files, 1473 tests passed.driver-sqlite-wasm: 31 files, 582 tests passed.Before that merge, at
2e36f756d(the same code as8e3c49fbc, which only adds the changeset):service-analytics: 118 files, 2560 tests passed.plugin-security: 130 files, 2561 tests passed.plugin-sharing: 37 files, 913 tests passed.typecheckexits 0 for all three driver packages at3bf7a68ce. The two new pin files and the changeddriver-sqlite-wasmfile are in their packages'tscprograms (--listFiles, 1 each).Consumer pin changed (patch round 1,
244f5321e):packages/plugins/plugin-auth/src/auth-contains-filter.test.ts, face 3 (#5893). It read the retired-operator door's full text through an unmarkedengine.findon a realSqlDriver, and it went red atb6f40dc95(CI Test Core 6/6). It is re-pinned to the production behaviour:INVALID_FILTER/ 400, the message contains "is RETIRED" and names neither$regexnor$icontains, and the spied driver log names both.'author'-marked leg: the full text,$regexand the$icontainsprescription, is kept for a vouched caller.A repo-wide sweep found no other consumer pin on these four doors' wording. The old-wording grep over
packages/,examples/andapps/tests gave 101 files, 19 of which reference a SQL-family driver, each read at its hit line. A trigger-token grep over the 311 test files outside the three drivers that use a real SQL driver found nothing more.Patch-round suites (
244f5321e, the same code as5fb4b25b1, which only edits the changeset), each with its own exit code 0:plugin-auth: 114 files, 2440 tests passed.service-storage: 40 files, 627 tests passed.cliunit project: 224 files; 3129 tests passed, 29 skipped. Two published-subpath pin files neededclibuilt first, and passed after the build.cloud-connection29/374,plugin-dev8/80,connector-mcp3/23 andhono5/122.5fb4b25b1:driver-sql,driver-tursoanddriver-sqlite-wasmgive the same counts as above.These are the rest of CI Test Core shard 6/6, which may not have run once plugin-auth failed, because the step runs turbo without
--continue.example-embed-objectqlis NOT MEASURED: it depends only on driver-memory, objectql and spec.Changeset (patch round 1,
5fb4b25b1): the frontmatter gains'@objectstack/driver-sqlite-wasm': patch, and one sentence states the inheritance: "SqliteWasmDriver(@objectstack/driver-sqlite-wasm) andTursoDriverin local mode extendSqlDriver, so they inherit this change from it: the same four doors answer the same way there."Ablations (committed state, restored and proved by blob hash plus an empty
git diff HEAD):sql-driver.tsreverted to base: the new driver-sql pin file had 32 of 34 failing. The 2 that passed are author-disclosure cases that base satisfies by disclosing to everyone.undefined(throughscripts/ablation-replace.mjs, anchor 1 → 0): 4 failing, exactly the primitive-operand author cases.remote-transport.tsreverted to base: 29 of 29 failing.Gates
dispatch-gates --commandsderived 62 families from this diff at3bf7a68ce. All 62 exited 0.check:dual-build-cjs-loadsandcheck:type-check-debtfirst exited 3 (PREREQUISITE NOT MET) and exited 0 afterturbo run build --filter=./packages/* --filter=./packages/*/*.--ranreconciliation: 62 derived, 62 run, 0 NOT-MEASURED, derived from recorded exit codes.b6f40dc95), the changeset and prose gates were re-run and exited 0: ADR-0087 registration, no-major, empty-changeset, deadline census, objectui-changeset, nul-bytes, doc-authoring.check-issue-citations --base 9d81af714: exit 0, 28 citations resolve.check:driver-conformance: "50 covered cell(s), 0 in the DEBT ledger, 0 exempt" before and after..tsfiles: 24 files, 0 errors, 0 warnings.eslint.config.mjssets noparserOptions.project, so this diff cannot change a verdict on an untouched file. The repo-wide run is CI's.Acceptance notes
Eight more
driver-sqldoors relay policy content on the CRUD face. Measured with the same probe and a policy-marked scope, at head:$icontains;$like;$likepattern with a trailing escape;$containscomparand;$inmember;undefinedcomparand;$or;Each 400 named the field or the comparand. They are outside this card's seven and the claim's file surface, so they are reported to the seat as a finding rather than fixed here. The seat filed them as driver-sql: eight more filter-compile refusal doors ignore the #8220 provenance mark, so a policy read scope refused there relays its field or comparand in the 400; close the class so every compile refusal goes through the withheld seam #20039.
Analytics caller
where. OnPOST /api/v1/analytics/query, a caller's{ f: { $null: 'x' } }or{ f: { $exists: 'x' } }answered 200. The driver received{ f: { $ne: null } }. Reported to the seat, which filed it as service-analytics: the caller-where lowering reads$null/$existsflags by identity, so a non-boolean flag ({ $null: "x" }) is answered 200 as IS NOT NULL where every other face refuses it (#5347 / #5369) #20040 (services lane).Stale prose, not pins: two test comments still say
driver-sqlrefuses$regex"by name", which is now true only for an'author'-markedwhere: plugin-authauth-where-operator-coverage.test.ts(header note) and runtimeundeclared-field-write-driver-split.integration.test.ts. Neither asserts that wording.Turso remote mode.
TursoDriver.toRemoteFilterrebuilds every filter node, so no mark reachesRemoteTransportin remote mode, and every mark-reading refusal there withholds for every caller. This fails closed and discloses nothing.Generated by Claude Code