Repository navigation
Commit 862f12c
Fixes #21178
Clause-②: yes (narrowing)
## What changes
`RemoteTransport.buildWhereSQL` (`@objectstack/driver-turso`, the filter
compiler every remote-mode `TursoDriver` read and filtered write uses)
now applies the JSON-column half of the filter contract exactly as the
local face (`SqlDriver`, which local and replica mode inherit) does,
from the same shared home in `@objectstack/core`:
- **Refusal.** On a field the driver stores as a JSON TEXT column, every
operator in `JSON_COLUMN_INCOMPATIBLE_OPERATORS` is refused with
`INVALID_FILTER` / 400 before any statement runs: in an operator map (at
the top of the per-operator loop, ahead of every arm), in the bare `{
field: value }` spelling, and in the bare `{ field: null }` spelling.
The message and the withheld diagnostic are
`jsonColumnOperatorRefusalText`'s output, byte for byte the local
face's, through this transport's existing withheld-refusal seam (#8220
provenance, diagnostic sink).
- **Membership.** `$contains` / `$notContains` on such a column answer
membership through `jsonMembershipPredicate('sqlite', ...)` (libSQL is
SQLite); the negated form sits inside `nullSafeNegative`, so a row with
no value satisfies `$notContains`. A scalar string column keeps the
substring test.
- ⛔ No copy of the set, the sentence or the construct in `driver-turso`;
no remote-only dialect.
### The widening (for the contract review)
The transport keeps no schema, so it learns "is this a JSON-stored
column" the way it learns every other declared fact, through an injected
resolver. This adds ONE optional public method on the exported
`RemoteTransport` class, and one type:
- `RemoteTransport.setJsonColumnResolver(resolver: JsonColumnResolver):
void`
- `type JsonColumnResolver`: a function taking `(object: string, field:
string)` and returning `boolean`, exported from `remote-transport.ts`
and NOT re-exported from `index.ts`, like its siblings
`NonTextColumnResolver` and `DeclaredValueShapeResolver`; it reaches the
published `.d.ts` only as the method's parameter type.
`TursoDriver`'s constructor wires it to the inherited
`SqlDriver.isJsonColumn`, beside `setDeclaredValueShapeResolver`
(constructor wiring only in `turso-driver.ts`; the `upsert` regions are
untouched). In remote mode `registerRemoteFieldMetadata` calls
`registerExternalObject`, which fills the same `jsonFields` registry the
local face's gate and membership reading ask, so both faces read one
population. A `RemoteTransport` driven standalone without the resolver
treats no column as JSON-stored and compiles as before. This is the
fourth sibling of `setFilterColumnSql`, `setNonTextColumnResolver` and
`setDeclaredValueShapeResolver`; the last of those shipped as "New
optional API" in commit `fb386074`'s changeset. The seat's answer to the
fork is on the card (claim amendment 5935032674, option A, open to the
maintainer's veto).
## Why (measured at base `0b12b9ea`, on the libsql SQLite stub harness)
Over a `multiple: true` lookup holding `["u1","u2"]` r1, `["u2"]` r2,
`["u3","u1"]` r3, `["u10"]` r4 and null r5, the remote face answered,
while the local face answered the right-hand column:
| filter | remote, before | local (the contract) |
|:--|:--|:--|
| `$contains: 'u1'` | r1, r3, r4 (u10 by substring) | r1, r3 |
| `$notContains: 'u1'` | r2, r5 | r2, r4, r5 |
| `$nin: ['u1']`, `$ne: 'u1'` | all five rows (fail-open) |
`INVALID_FILTER` / 400 |
| `$eq`, `$in`, bare equality | no row | `INVALID_FILTER` / 400 |
| `$lt` / `$lte` `'u1'` | r1-r4 (lexicographic over the serialization) |
`INVALID_FILTER` / 400 |
| `$startsWith: '['`, `$endsWith: ']'` | r1-r4 | `INVALID_FILTER` / 400
|
| `$icontains: 'u1'` | r1, r3, r4 | `INVALID_FILTER` / 400 |
| `{ owners: null }` | r5 | `INVALID_FILTER` / 400 |
| `json` field, `$contains: 'u1'` | r1, r4 (text inside the object) | no
row (array-only membership) |
## Pins
- `turso-local-remote-json-column-parity.test.ts` (new): one fixture on
BOTH faces; every case asserts remote equals local AND the canonical
answer. It iterates the refused set from
`JSON_COLUMN_INCOMPATIBLE_OPERATORS` as it stands (27 members at base),
asserting `code` + `status` + equality with
`jsonColumnOperatorRefusalText(...).message` (never the literal words);
the bare infix spellings in an operator map stay refused on both faces.
Also: bare equality, `null` equality (`{f: null}`, `$eq: null`, `$ne:
null`), the gate under `$and` / `$or` / `$not`, the population (a `tags`
field and a `json` field), `count()`; membership `u1` vs `["u10"]`,
`u10`, `$notContains` complement with the NULL row, `tags`, the `json`
object, bind alignment beside sibling predicates, `count()`; the scalar
text-field control (`$contains`, `$nin`, `$eq`, bare and null equality,
`$startsWith` unchanged); `$null` / `$exists` still answered.
- `remote-transport-compile-refusal-seam.test.ts`: the enumeration
requires every seam refusal method to have a row, so
`jsonColumnOperator` gets one per position (operator map, bare value,
bare null) on a half-2 transport told exactly one column is JSON-stored.
Policy / author / unmarked / merged-arm provenance all hold.
## Ablation (one-time proofs, via `scripts/ablation-replace.mjs`,
restore proven by blob hash and empty `git diff HEAD`)
The suite imports `./turso-driver.js` from source (vitest), so no
`dist/` leg applies.
- **Unwire the resolver** in `turso-driver.ts`
(`this.remoteTransport.setJsonColumnResolver(` replaced by a no-op call;
anchor 1 to 0, blob `a9affc72` to `b90200ef`): the parity suite goes
**24 failed / 19 passed** of 43, as predicted. Red: all 14 operator-map
refusals, bare equality, depth, population, both `count()` pins, `u1`
membership, `$notContains`, the `json` object, bind alignment, null
equality. Still green, as predicted: the 13 bare-infix rows (both faces
refuse as an object comparand whatever the gate), the set check, `u10`
and `tags` membership (substring coincides), the scalar controls,
presence.
- **Disable the membership arm** in `remote-transport.ts`
(`pushJsonMembership` answers false): **5 failed / 38 passed**, exactly
the five membership pins whose substring answer differs; every refusal
pin stays green.
- Restored: `git diff HEAD` empty, blob equals HEAD in both legs.
## Filter-semantics compile surfaces, one conclusion per face
1. `driver-sql` `applyFilterCondition` (`sql-driver.ts`): **already
conformant** — it is the contract; the parity suite's local column is
its answer, and `driver-sqlite-wasm` and local/replica `driver-turso`
inherit it. Not edited.
2. `driver-turso` `RemoteTransport.buildWhereSQL`
(`remote-transport.ts`): **changed** (this PR).
3. `service-analytics` `compileScopedFilterToSql` (`read-scope-sql.ts`):
**out of scope** — another face; it already reaches the shared
membership construct through `contains-membership-sql.ts` (imports
`jsonMembershipPredicate`).
4. `service-analytics` `lowerAnalyticsWhere`
(`strategies/filter-normalizer.ts`): **out of scope** — another face,
same shared construct as 3.
5. `formula` `matchesFilterCondition` (`matches-filter.ts`): **out of
scope** — another face, and seat 2's in-flight #20822 group 3b owns it.
6. `objectql` `applyHaving` / `matchesHaving` (`having-filter.ts`):
**out of scope** — another face (it already imports the shared set and
sentence), and seat 2's #20822 group 3b owns it.
7. `driver-memory` query path (refuses the family since `45ce12a4`,
`filter-refusal.ts` imports the shared set) and `driver-mongodb`
`translateFieldOperators` (`mongodb-filter.ts`): **out of scope** —
other faces, not touched.
## Tests and gates (all on HEAD `c67a136e` unless noted)
- `pnpm --filter @objectstack/driver-turso exec vitest run
--maxWorkers=2`: 84 files passed, 2286 tests passed, 33 skipped (exit 0,
via `os-verify-lock`).
- `pnpm --filter @objectstack/driver-turso typecheck`: exit 0; `tsc
--listFiles` includes both edited test files.
- `pnpm check:driver-conformance`: before (base `0b12b9ea`) 50 covered /
0 DEBT / 0 exempt; after (`c67a136e`) 50 covered / 0 DEBT / 0 exempt.
The ledger did not move.
- `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack
--commands` derived 63 commands at `c67a136e`; all 63 were run and
reconciled with `--ran` (each line recording its exit code): 62 run, 1
NOT MEASURED. `pnpm check:dual-build-cjs-loads` exited 3 (PREREQUISITE
NOT MET: it reads every package's `dist/`, and a repo-wide build is
CI's); narrowed in its place, `require` of `driver-turso`'s CJS build
and `import` of its ESM build both load and expose
`setJsonColumnResolver`. `check-plugin-teardown-shape --self-test` and
`check:lean-entry-closure` first exited 3 on prerequisites (a fixture
commit outside the shallow clone; objectql's `dist/`) and exited 0 after
fetching that commit and building objectql's closure.
- `check-adr-0087-registration --base origin/main`: the changeset reads
as declared-breaking with one disposition, `not-required
(no-migration-prescription)`; `check-changeset-no-major`: no major bump.
- Lint, narrowed to the 4 touched TS files: `eslint --no-inline-config
--format json` read 4 files, 0 errors, 0 warnings. All four are in
`eslint .`'s population (`--print-config` resolves each). The config
never enables type-aware linting (the resolved `parserOptions` are
`ecmaVersion` and `sourceType` only, and `eslint.config.mjs` says so in
its header), so this diff cannot move a verdict on any untouched file.
The full `pnpm lint` is CI's.
- Downstream consumers (`cli`, `qa/dogfood`, `runtime`,
`service-datasource`) are declared to CI: none references
`RemoteTransport`'s API, and the one remote-mode consumer test
(`date-bucket-parity-turso`) aggregates without a JSON-field filter.
## Changeset
`.changeset/21178-remote-json-column-gate.md`:
`@objectstack/driver-turso` `minor`, BREAKING banner, `Clause-②: yes
(narrowing)`, ADR-0087 disposition `not-required
(no-migration-prescription)`, the new optional API named, and the
migration text (write `$contains` for "holds this member", an `$or` of
`$contains` for any-of, `$not` around either for the exclusion, `$null`
/ `$exists` / `$empty` for presence).
## Siblings
- #21067 (in flight) owns the shared sentence's words in
`json-column-operator-refusal.ts`; this PR edits neither that file nor
that branch, and its pins compare against the builder's output, so the
reword cannot flip them. Whichever of the two lands second merges `main`
and re-runs its pins.
- #21185 is ruled to edit the `upsert` regions of `remote-transport.ts`
and `turso-driver.ts`; this PR is region-disjoint. Whichever lands
second merges `main`.
## Acceptance notes
- The local face's gate reads the operator, not the comparand, so on a
JSON column it refuses the equality spellings of a null comparand (`{ f:
null }`, `$eq: null`, `$ne: null`) although `IS NULL` is a well-formed
presence question there; this PR matches it on the remote face (that is
the parity invariant), and `$null` / `$exists` / `$empty` remain the
presence spellings on both. Noted, not filed: no declared contract says
otherwise (`$eq` is a declared member of the set). Carrier: none.
- Refusal order differs from the local face only for a doubly-wrong
filter: the remote gate runs ahead of each arm's own comparand gate (the
remote comparand gate lives inside each arm), so `{ owners: { $in: [{
... }] } }` reads the JSON-column sentence remotely and the comparand
sentence locally. Both are `INVALID_FILTER` / 400.
- The bare infix spellings (`=`, `in`, …) in an operator map are refused
on both faces as an object comparand, with different wording per face;
pre-existing, not JSON-specific.
---
_Generated by [Claude
Code](https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent ce8a6d2 commit 862f12c
5 files changed
Lines changed: 553 additions & 0 deletions
File tree
- .changeset
- packages/drivers/driver-turso/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
Lines changed: 32 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| 41 | + | |
41 | 42 | | |
42 | 43 | | |
43 | 44 | | |
| |||
51 | 52 | | |
52 | 53 | | |
53 | 54 | | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
54 | 60 | | |
55 | 61 | | |
56 | 62 | | |
| |||
171 | 177 | | |
172 | 178 | | |
173 | 179 | | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
174 | 190 | | |
175 | 191 | | |
176 | 192 | | |
| |||
238 | 254 | | |
239 | 255 | | |
240 | 256 | | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
241 | 270 | | |
242 | 271 | | |
243 | 272 | | |
| |||
341 | 370 | | |
342 | 371 | | |
343 | 372 | | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
344 | 376 | | |
345 | 377 | | |
346 | 378 | | |
| |||
0 commit comments