Repository navigation
Commit 8d76c2d
Fixes #20039
Clause-②: no
## What changed
A read scope is the RLS, sharing or tenant predicate that
`plugin-security` (ordinary reads) and `service-analytics`
(`ObjectQLStrategy.withReadScope`) AND into the caller's `where`. Both
merges mark the scope `'policy'` and the caller's own predicate
`'author'` (#8220,
`packages/spec/src/data/filter-subtree-provenance.ts`). The contract for
`'policy'`: "A refusal raised from inside it keeps the #7929 redaction:
identity (`INVALID_FILTER` / 400) and capability statement on the wire,
operands in the server log." The fail direction: "Unmarked or ambiguous
⇒ withheld."
PR #20037 brought four `SqlDriver` doors under that seam. This PR closes
the class: **every refusal on `SqlDriver`'s filter-compile path now goes
through `withheldFilterError` with the node it was raised from**, and
one pin enumerates the builders so that a new one cannot disclose by
omission.
**The nine `SqlDriver` builders converted** (each of them called
`unsupportedFilterError` directly at base):
| builder | the node the seam resolves |
|:--|:--|
| `icontainsComparandError` | the comparand if it is an object, else the
field's operator map |
| `likePatternComparandError` | the same |
| `danglingLikeEscapeError` | the field's operator map |
| `unrenderableTextComparandError` | the comparand (an object or array
here) |
| `unbindableListMemberError` | the member if it is an object, else the
list |
| `undefinedComparandError` | the nearest object holding the
`undefined`: the node carrying the field, the operator map, or the list
|
| `assertFilterNode` | the element if it is an object, else the `$and` /
`$or` list that holds it, or the node carrying `$not` |
| `unknownLogicalOperatorError` | the node CARRYING the key, never its
value |
| `filterArrayReachedDriverError` | the array itself (it is the `where`
root). Not on the card: the enumeration found it, and it printed the
whole array |
The walk's refusals need the node that encloses the refused position.
The spec's shared walk hands its hooks only a value, a key and a path,
so `sqlFilterVerdictHooks` recovers the node from the path index it
already kept for the combinator door (#20020). It adds one map of `$and`
/ `$or` lists, keyed by the combinator key's position. The spec hook
signature is unchanged.
Per door, what stays on the wire is the class statement: which kind of
refusal fired and the capability sentence. The field, the operator
variant (`$like` vs `$ilike`, which of the three list operators, which
of the five text operators), the comparand, the list index and the
filter path go to the server log. `'author'`-marked subtrees get the
text the door answered before, byte for byte. The source templates are
the same literals, now passed as the diagnostic. Code and status are
unchanged on every path (`INVALID_FILTER` / 400), and the set of refused
filters is unchanged.
**Turso remote transport.** `RemoteTransport.buildWhereSQL` refuses
these classes on its own (measured, see below). Its twins now go through
`withheldRefusal`, with the same node rules (`refusalNode` is
`refusalSubtree`'s twin): `icontainsComparand`, `likePatternComparand`,
`danglingLikeEscape`, `undefinedComparand`, `uncompilableSubFilter` (a
non-node `$and` / `$or` element or `$not` operand),
`undeclaredCombinator` (both of its tails share one withheld sentence)
and `uncompilableWhere` (a non-object top-level `where`). The
transport's remaining two refusals that named the field go through too,
as a bounded in-place fix within this claim's file surface:
`emptyFieldFilter`, whose `driver-sql` twin has withheld since #8197,
and `unsupportedOperator`'s `$between` arm. `TursoDriver` never reaches
the `$between` arm (it lowers `$between` first). After this,
`buildWhereSQL` has no refusal outside the seam. For six classes the
withheld sentence is `driver-sql`'s behind the `[RemoteTransport]`
prefix, pinned through `TursoDriver` local vs remote.
## Measured: H1–H3
**H1.** The probe (scratch, not committed) ran a real `ObjectQL` over a
real `SqlDriver` (better-sqlite3), with a first-registered middleware
copying `plugin-security`'s merge and marking. It marks the injected
scope `'policy'` and the caller's verbatim `where` `'author'`, then
composes `{ $and: [where, scope] }`, or the scope alone when there is no
caller `where`. The scope used distinctive names (`policy_stage`,
`PSECRET…`). A cell counts as a leak when one of them appears in the
response message.
| class | base (`b76aad5f6f`) | this branch |
|:--|:--|:--|
| empty `$icontains` | field | none, field in the log |
| non-string `$icontains` | field, comparand | none, both in the log |
| non-string `$like` | field, comparand | none, both in the log |
| `$like` trailing escape | field, pattern | none, both in the log |
| object `$contains` comparand | field, comparand | none, both in the
log |
| object `$in` member | field, comparand | none, both in the log |
| `undefined` comparand (direct, `$eq`, `$in` member) | field | none,
field in the log |
| non-node element of `$or` / `$not` operand | the literal | none,
literal in the log |
| undeclared node combinator | the key | none, key in the log |
| scope handed over as an array | whole array (field and literal) |
none, both in the log |
The policy-scope cells are 13 shapes × {with, without} a caller `where`,
26 cells in all. All 26 leaked at base and none leak at this branch. The
author arm used the same 13 shapes in the caller's own `where`: 13 of 13
full messages are byte-identical at base and at this branch. (Six of
those 13 are refused by the engine's own comparand or `where`-shape door
before any middleware runs, and are identical on both sides for that
reason.) Base was measured by the whole-file ablation below, which
reverted `sql-driver.ts` and rebuilt `dist/`.
**H2: the enumeration.** A TypeScript-parser walk over `sql-driver.ts`
lists the functions calling `unsupportedFilterError` directly, and the
functions calling `withheldFilterError`.
- **Base:** 12 direct callers (`grep -c 'unsupportedFilterError('` = 13,
including the definition), and 12 seam callers.
- **This branch:** 3 direct callers (grep 4): the seam itself, the
`'author'` re-issue in `resolveWithheldFilterRefusal`, and
`unresolvableFilterColumnError`. That last one is raised from a dialect
error after the statement ran, and resolves its own provenance by name
(`unresolvableColumnProvenance`, #20020). 21 seam callers
(`withheldFilterError(` grep 22).
- **Positive control:** the doors PR #20037 converted
(`retiredFilterOperatorError`, `unsupportedFilterOperatorError`,
`assertFilterNodeList`, `nonBooleanNullComparandError`,
`nonBooleanExistsComparandError`) are in the base seam set, and its
column door is in the direct set.
- **The eighth door (H1's PM-grep guess):** `assertFilterNode` is
confirmed. The enumeration adds a ninth,
`filterArrayReachedDriverError`.
`RemoteTransport`: base has 11 direct `invalidFilterError` callers and
this branch 2, the seam and the re-issue. The seam callers go from 5
methods to 13.
**H3.** Measured on `RemoteTransport` with a `'policy'`-marked `where`.
The transport refuses these on its own, naming `'object.field'` and the
comparand: `$icontains`, non-string `$like`, trailing escape,
`undefined` comparand, non-node `$or` element and `$not` operand,
undeclared combinator (including a misplaced field operator), non-object
`where` (the whole array), empty operator map, un-lowered `$between`. An
object `$contains` comparand and an object `$in` member were already
withheld there: both are `uncompilableComparand`, which #8197 converted.
Through `TursoDriver` in REMOTE mode, `toRemoteFilter` rebuilds every
node, so no mark reaches the transport. Every mark-reading refusal there
withholds for every caller, author included. That is pinned as the
declared fail-closed cost.
**H4: consumer pins.** Two sweeps, over test files in `packages/`,
`examples/` and `apps/`:
- **Wording grep:** the nine author texts' distinctive fragments and the
remote twins'. 24 files, of which 9 are outside the three driver
packages: `driver-memory` / `driver-mongodb` (their own wording) and a
`packages/spec` test (its own). Control: the re-pinned `driver-sql` /
`driver-turso` files are among the 24.
- **Assertion-shaped grep** over the test files outside the three
drivers that reference a SQL-family driver and `INVALID_FILTER`: 55
files, 17 of which construct a real SQL-family driver. Every hit is
`service-analytics`' own normalizer or read-scope compiler wording
(`comparand at … is undefined`, `normalizeAnalyticsFilterTree`),
`plugin-auth`'s `$regex` face-3 pin (the retired door, unchanged here),
or a negative assertion.
**No consumer pin outside the three driver packages reads these doors'
wording.** None was changed, and the suites PR #20037's round 2 named
are green (below).
## Compile surfaces (per `references/compile-surfaces.md`, located again
at this head)
| # | face | this PR |
|:--|:--|:--|
| 1 | `driver-sql` `applyFilterCondition` (`sql-driver.ts:15907`);
`driver-sqlite-wasm` and Turso local by inheritance | CHANGED: nine
builders, and the class is closed |
| 2 | Turso `RemoteTransport` `buildWhereSQL`
(`remote-transport.ts:2667`) | CHANGED: seven twin classes plus the
empty-operator-map and `$between` arms; class closed |
| 3 | service-analytics `compileScopedFilterToSql`
(`read-scope-sql.ts:582`) | not touched: outside the claim; its refusals
are the withheld `READ_SCOPE_COMPILE_FAILED` / 500 |
| 4 | service-analytics `lowerAnalyticsWhere`
(`filter-normalizer.ts:1966`) | not touched (outside the claim) |
| 5 | `formula` `matchesFilterCondition` (`matches-filter.ts:212`) | not
touched, not measured |
| half | objectql `applyHaving` / `matchesHaving`
(`having-filter.ts:279` / `:292`) | not touched, not measured |
| — | `driver-memory` / `driver-mongodb` | not touched; they do not read
the mark, and disclosure there is not measured |
## Tests
**New pins:**
-
`packages/drivers/driver-sql/src/sql-driver-compile-refusal-seam.test.ts`,
91 cases:
- the enumeration (positive control, direct-caller set, seam-caller set
equal to the table);
- 21 builders × {policy, author, unmarked, merged `$and` in both arm
orders}, where the array-root builder has no merged arm. Each row is
bound to its builder by the error's own stack;
- five node-lookup cases.
-
`packages/drivers/driver-turso/src/remote-transport-compile-refusal-seam.test.ts`,
96 cases:
- the same enumeration over `remote-transport.ts`;
- 13 methods plus 7 extra arms under every mark;
- two node cases;
- `TursoDriver` local vs remote: the withheld sentences of six classes
are one sentence, and REMOTE withholds an author-marked `where`.
Per door, the pins assert: policy-marked gives `INVALID_FILTER` / 400,
the error's own keys exactly `code,status`, no secret in the message,
and every secret in the log or sink; author-marked gives the full text,
which the policy case logged; unmarked is byte-identical to policy.
**Existing pins that read a converted door's full text through an
unmarked `where`.** Each now marks it `'author'`, as a merge boundary
marks a caller's own predicate, on a shallow copy where the case is a
shared constant:
- `driver-sql`: `filter-refusal-envelope` (array cases),
`not-null-safe`, `out-of-contract-filter-input`,
`silent-empty-predicate` (its #7929 comment was updated to say the
family is now narrowed too), `undefined-comparand-refusal`.
- `driver-turso`: `bare-date-routing`, `boolean-identity`,
`node-operator-refusal`, `not-operator`, `top-level-where`,
`undefined-comparand-refusal`.
- `driver-sqlite-wasm`: `out-of-contract-filter-input`, which also gains
an unmarked withheld leg.
Two remote shapes cannot get the author text back, and their pins say
so:
- a PRIMITIVE top-level `where` cannot carry a mark, so it is withheld
for everyone and its pin asserts the class statement;
- a refusal raised inside `$not` resolves against the NULL-safe REWRITE
of the operand, which the resolver cannot find under the root. It is
withheld even for an author, fail-closed by construction as `driver-sql`
documents for its own `$not` rewrite. Those pins read the naming half
from the diagnostic sink.
`turso-local-remote-text-parity.test.ts`: a withheld remote answer must
still name none of the case's mentions, except the ones the refusal's
CLASS statement carries. The class statement is read off the local
face's unmarked answer; for the `$icontains` rows it is `$icontains`
itself.
**Suites.** Driver suites at `8b4f1bb3ed` (after merging `main`, which
brought #20054's `turso-driver.ts` change), with `driver-sql` and the
driver closures rebuilt:
- `driver-sql`: 185 files passed, 11 skipped; 2936 tests passed, 170
skipped.
- `driver-turso`: 68 files, 1606 tests passed.
- `driver-sqlite-wasm`: 32 files, 606 tests passed.
- `typecheck` exits 0 for all three. `tsc --listFiles` counts 1 each for
the two new pin files and the changed sqlite-wasm pin.
Consumer suites at `ab61f7a832`: the same driver code, before the `main`
merge. The merged commits touch no `driver-sql` path, and in
`driver-turso` only `turso-driver.ts` (#20054's constructor seam).
`driver-sql` and `driver-turso` `dist/` were rebuilt first.
- `plugin-auth`: 114 files, 2440 tests passed.
- `service-storage`: 40 files, 627 tests passed.
- `cloud-connection`: 30 files, 396 tests passed.
- `plugin-dev`: 8 files, 80 tests passed.
- `connector-mcp`: 3 files, 23 tests passed.
- `hono`: 5 files, 122 tests passed.
- `cli` unit project (`--project unit`): 224 files, 3158 tests passed.
The integration tier is declared to CI; this diff touches no cli file.
- `plugin-security`: 133 files, 2648 tests passed.
- `plugin-sharing`: 37 files, 913 tests passed.
- `service-analytics`: 120 files, 2689 tests passed.
**Ablations.** All ran on committed state. Each restore is proved by the
blob equal to `HEAD` and an empty `git diff HEAD`.
1. **`sql-driver.ts` at the base blob** (`463b99cf41`; marker 1 → 0),
`driver-sql` rebuilt, and `ablation-dist-preflight --absent` passed. The
new seam pin gave 41 failed, 50 passed:
- red: both enumeration assertions, the 35 policy/author/unmarked/merged
cases of the nine builders, and four node cases;
- green: the positive control, the 12 already-converted builders × 4,
and the author-arm node case, which base satisfies by disclosing to
everyone.
The five re-pinned `driver-sql` files stayed 110/110 green at base, so
the author text they read is unchanged. The same run produced the base
column of the H1 table. Restored to blob `a89054ee3a`, rebuilt, and the
preflight found the marker in `dist/`.
2. **`remote-transport.ts` at the base blob** (`55148ccc94`): the remote
seam pin gave 67 failed, 29 passed. It imports `src`, so no build leg
was needed.
3. **The `classifyKey` enclosing node replaced with `undefined`**
(`ablation-replace.mjs`, anchor 1 → 0): 3 failed, exactly the
unknown-combinator author and merged legs and the direct-`undefined`
node case.
4. **The `assertNode` enclosing replaced with `undefined`:** 4 failed,
the non-node element's author and merged legs and the two
primitive-inheritance node cases.
5. **One builder bypassing the seam** (`icontainsComparandError` calling
`unsupportedFilterError`): 6 failed, both enumeration assertions and its
four rows. This is the red a new bypassing builder gets.
## Gates
- `dispatch-gates --commands --repo objectstack-ai/objectstack` at
`8b4f1bb3ed` (merge base `249172975b`, 18 paths) derived 62 families.
They are the PM's 55 plus the seven that apply once the changeset
exists.
- All 62 exited 0. `check:dual-build-cjs-loads` first exited 3
(PREREQUISITE NOT MET), then 0 after a full `turbo run build
--filter=./packages/* --filter=./packages/*/*` (72 of 72 tasks cached).
The other dist-reading families were re-run on that build and exited 0.
- `--ran`: "62 derived, 62 run, 0 NOT-MEASURED, 0 UNRUN (a DERIVED
zero)".
- The three roster families under `packages/` that the derivation flags
as unreadable exited 0: `check:authz-resolver`,
`check:error-code-casing` and `check:filter-alias-parity`.
- `check-issue-citations --base 2491729`: exit 0, 56 citations
resolve.
- `check:driver-conformance`: "50 covered cell(s), 0 in the DEBT ledger,
0 exempt" at base (`b76aad5f6f`, read from an archived tree) and at this
head. The ledger did not move.
- ESLint (`--no-inline-config --format json`) on the 17 changed `.ts`
files: 17 files, 0 errors, 0 warnings. `eslint.config.mjs` sets no
`parserOptions.project` (the printed config for `sql-driver.ts` is
`{"ecmaVersion":"latest","sourceType":"module"}`), so this diff cannot
move a verdict on an untouched file. The repo-wide run is CI's.
- A control-byte scan of the changed files found no hits.
## Acceptance notes
- **What an unmarked caller loses:** its own diagnostic from these nine
refusals. It gets the withheld wording with the same code and status.
This covers no security plugin, system context, anonymous calls, a
`where` holding a `{placeholder}` token, and Turso remote mode. A
member's plain `where` under `plugin-security` is marked `'author'` and
keeps the full text. The changeset says this.
- **The two node-shape refusals inside `$not` on the remote transport**
(`uncompilableSubFilter`, `undeclaredCombinator`) resolve against the
rewritten operand, so they withhold even for an author-marked `where`.
The comparand doors there still answer an author in full. *(Narrowed by
the seat from contract review 5826632400.)* PR #20037's
`nonListCombinator` has the same limitation inside `$not`. It is
fail-closed, and invisible in production, where `TursoDriver` never
hands the transport a mark.
- **Two findings are reported to the seat and not fixed here.** Both are
outside this claim's file surface or class:
- `TursoDriver`'s remote `$between` lowering (`toRemoteFieldSpec`)
refuses a non-two-element range with a bare `Error` carrying no `code` /
`status`. Local mode answers `INVALID_FILTER` / 400.
- A second finding, in `RemoteTransport`'s handling of `where` keys, is
routed to the maintainer. *(The seat removed its detail from this body
before landing, pending the maintainer's decision.)*
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent d624002 commit 8d76c2d
18 files changed
Lines changed: 1706 additions & 147 deletions
File tree
- .changeset
- packages/drivers
- driver-sqlite-wasm/src
- driver-sql/src
- driver-turso/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
0 commit comments