Skip to content

fix(driver-turso): a remote $between that is not two bounds is refused INVALID_FILTER / 400 through the transport's withheld seam, naming no field or value (#20094) - #20114

Merged
objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/issue-20094-turso-remote-between-envelope
Sep 25, 2026
Merged

objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/issue-20094-turso-remote-between-envelope

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #20094

Clause-②: no

What changed

In remote mode, TursoDriver.toRemoteFieldSpec lowers $between to $gte / $lte before the filter reaches RemoteTransport, so that a two-bound range inherits the whole-day upper-bound rule (framework#4081). A range that was not two bounds was refused right there, with a bare Error: no code, no status, and a message naming 'deal.field' and echoing the comparand. That method runs before the transport and so outside its refusal seam, which is why PR #20093's class closure never reached it.

  • turso-driver.ts, the $between arm of toRemoteFieldSpec. A range that is not two bounds is no longer refused here. The arm hands it to the transport as written, un-lowered. A two-bound range lowers exactly as before.
  • remote-transport.ts, the $between arm of unsupportedOperator. The arm now tells apart the two ways a $between can arrive:
    • A range that is not two bounds gets driver-sql's class statement for the same mistake (betweenArityError), behind this file's [RemoteTransport] prefix. It goes through withheldRefusal: INVALID_FILTER / 400, with the field and the comparand in the diagnostic sink only.
    • A well-formed range keeps its "must be lowered" sentence, unchanged.
    • The range is read off the operator map that the one call site already passes as subtree. The refusal carries the range itself when it is an object, and otherwise the map. That is driver-sql's refusalSubtree rule.

After this change toRemoteFilter, toRemoteFieldSpec and toRemoteUpperBound contain no throw. The remote face's filter refusals are raised in buildWhereSQL, behind the seam and its enumeration pin.

The remote withheld message, as it stands at this head:

[RemoteTransport] Operator "$between" in this filter requires a [min, max] value array. The field it was aimed at is withheld from the message; the full diagnostic is in the server log.

It is the local message behind the prefix, minus one thing: the local sentence carries the tracker id (#8197). Runtime text carries no tracker id, and check:doc-authoring's sibling-package prose ratchet reds a new one, so the copy leaves it out. The class statement itself is byte-equal to the local one. The pin compares the two faces after removing the local tracker id, so the day the local sentence drops it, the comparison still holds.

Measured

The probe was a scratch vitest file, never committed. It drove a real remote TursoDriver (the in-repo libSQL stub over better-sqlite3) and a local TursoDriver(':memory:') control, both seeded with the same four rows. It covered 7 shapes ([x], [x, y, z], a number, a string, null, {} and []), 3 caller classes (unmarked, 'policy'-marked where, 'author'-marked where), and both find and count: 42 cells per face. The field is secret_policy_col and the literals are distinctive, so a leak is a substring hit. The base column was measured on the base tree (8d76c2d38c), and again with both files put back at their base blobs in this worktree. The two base runs are byte-identical.

H1: confirmed

face base this head
remote 42 of 42 cells: bare Error, code and status undefined, no own keys. The message names 'deal.secret_policy_col' in all 42, and echoes the comparand in all 24 cells whose shape has a literal. Nothing is logged. 42 of 42 cells: INVALID_FILTER / 400, own keys exactly code,status. The message names neither the field nor the comparand. One warn line holds both.
local (control) INVALID_FILTER / 400 and one class for all seven shapes. Unmarked and 'policy': withheld, with one warn line naming the field. 'author': the field restored, nothing logged. The comparand is never echoed. 42 of 42 cells byte-identical to base

H2: falsified as stated, so the arm needed one branch

I drove the transport directly with the un-lowered $between, 7 shapes × 3 marks. At base it refused all 21 cells with INVALID_FILTER / 400 through the seam and executed no statement. The wording, though, was the "must be lowered to $gte/$lte before it reaches the transport" sentence. That is the class of a skipped lowering step, not the local face's arity class. Dropping the pre-check alone would therefore have answered the right envelope with a false cause. At this head all 21 cells answer the arity class. 'author'-marked cells get the full diagnostic, including the comparand.

Lowering controls, base against head, byte-identical in all six. Each control compared the lowered filter (toRemoteFilter), the statements the stub executed, and the rows and count on both faces. Local and remote agree in every control.

control lowered (identical at base and head) rows
numeric [10, 20] {"$gte":10,"$lte":20} r2, r3
bare days on a date field {"$gte":"2026-01-01","$lte":"2026-01-31"} r1, r2
bare days on a datetime field {"$gte":"2026-01-01T00:00:00.000Z","$lt":"2026-02-01T00:00:00.000Z"} r1, r2 (r2 is 13:45 on the last day; only the whole-day rule admits it)
instants on a datetime field $gte / $lte, unwidened r1
the bare-day control under $not $not of the widened pair r3, r4
numeric inside $or the lowered pair inside $or r2, r3, r4

H3: which text each caller class gets

caller local (base = head) remote base remote head
unmarked withheld class statement bare Error, field and comparand, to everyone withheld class statement
'policy' withheld class statement the same bare Error withheld class statement
'author' the full text, field restored the same bare Error withheld class statement

toRemoteFilter rebuilds every node, so no mark reaches the transport. On the remote face an author therefore reads the unmarked answer. PR #20093 declared that same cost for the transport's other classes; it errs in the withholding direction. I did not extend turso-local-remote-text-parity.test.ts. Its population is FILTER_TEXT_CASES, which has no $between row, so nothing there reaches this door. The local-against-remote table for withheld classes is Half 3 of remote-transport-compile-refusal-seam.test.ts. This PR extends that half with a table of its own, and adds no new file.

H4: the refused set did not move

Every one of the 42 remote cells was refused at base and is refused at head. I also measured 15 nested and multi-operator cells at both base and head: a malformed range under $not, inside $or, inside $and with null, beside a $gt in one map, and a two-bound range with an object member. Each of them was refused at base and is refused at head. The six well-formed controls answer the same rows. Declared patch, Clause-②: no.

Tests

remote-transport-compile-refusal-seam.test.ts grows from 96 to 130 cases:

  • An ARMS row for the arity branch ({ secret_policy_col: { $between: [7770123] } }), driven through all four legs of Half 2: 'policy', 'author', unmarked and merged $and in both orders. The stack binds it to unsupportedOperator. The existing well-formed $between row stays, so both branches of the arm are pinned.
  • Half 3b, one table over both faces. It covers 7 shapes × find / count (28 cases):
    • Unmarked and 'policy': INVALID_FILTER / 400 and own keys exactly code,status on both faces. The class statement is on the wire, with no field and no literal. The field is in each face's log, and the comparand is in the remote sink. The remote message equals the local one behind the prefix, minus the local tracker id. No statement is executed.
    • 'author': local restores the field, and remote equals its own unmarked answer.
  • Two well-formed controls. Numeric [10, 20], and bare days on a datetime field. The date control's expected rows include a row only the whole-day upper bound admits. Both faces must return the same rows, and count must agree.

Ablations, on committed state. Each ablation put one file back at its base blob with git restore --source, and the blob hash was checked equal to base before the run. trap restored the file from HEAD, the blob hash was checked equal to HEAD, and git diff HEAD was empty afterwards. The pin imports src, so no build leg was needed. I predicted the counts before running either one.

  1. turso-driver.ts at base (marker 1 → 0): 28 failed, 102 passed, as predicted. All 28 Half 3b shape cases failed. The two controls and the transport arity row stayed green.
  2. remote-transport.ts at base (marker 2 → 0): 18 failed, 112 passed, as predicted. The four legs of the arity row failed, along with the 14 unmarked and 'policy' Half 3b cases, whose wording at base is "must be lowered". The 14 'author' cases stayed green (remote stays withheld either way), and so did the controls. This is H2 as a test result.

Suites run, all at d148119880 unless noted:

  • driver-turso, the whole suite: 68 files, 1640 tests passed. typecheck exits 0, and tsc --noEmit --listFiles counts the pin file once.
  • driver-sql seam and provenance pins, which import the shared wording: sql-driver-compile-refusal-seam, sql-driver-target-field-provenance, sql-driver-refusal-door-provenance and sql-driver-cross-field-provenance. 4 files, 169 tests passed.
  • The scratch probe, at base and at head (see above).
  • driver-sql is untouched, and no export moves. The only wording removed ([TursoDriver] $between on … needs exactly two bounds) has no reader: a repo-wide git grep finds it nowhere. The control, the spec's own "A range needs exactly two bounds" message, was found in its 3 files.

Gates

  • dispatch-gates --commands --repo objectstack-ai/objectstack at d148119880 (merge base aa04ea2964, 4 paths) derived 62 families, and all 62 exited 0.
    • check:dual-build-cjs-loads and check:lean-entry-closure first exited 3 (PREREQUISITE NOT MET). They exited 0 after turbo run build --filter=./packages/* --filter=./packages/*/* (72 of 72 tasks). check:dts-closure was re-run on that build: 72 packages, 166 of 166 declaration files present.
    • --ran over the last record per command: "62 derived, 62 run, 0 NOT-MEASURED, 0 UNRUN".
  • The three roster families under packages/ that the PM's derivation flagged exited 0: check:authz-resolver, check:error-code-casing and check:filter-alias-parity.
  • check-issue-citations --base aa04ea2964: exit 0, 5 citations resolve.
  • check:driver-conformance: "50 covered cell(s), 0 in the DEBT ledger, 0 exempt" both at base (8d76c2d38c, read from an archived tree) and at this head. The ledger did not move.
  • check:doc-authoring: the sibling-package prose ids hold the baseline, with no growth.
  • ESLint (--no-inline-config --format json) on the 3 changed .ts files reports 3 files, 0 errors and 0 warnings, so no file was ignored. The printed config for remote-transport.ts has parserOptions {"ecmaVersion":"latest","sourceType":"module"} and no project. Linting is therefore not type-aware, and this diff cannot move a verdict on an untouched file. The repo-wide run is CI's.
  • A control-byte scan of the 4 changed files found no hits.

Acceptance notes


Generated by Claude Code

…sport's seam, INVALID_FILTER / 400 with the field and comparand withheld

TursoDriver.toRemoteFieldSpec threw a bare Error for a $between that is not
two bounds, outside the transport's refusal seam: no code, no status, and a
message naming 'object.field' and echoing the comparand to every caller.
The lowering now hands such a range over as written, and
RemoteTransport.unsupportedOperator's $between arm refuses it through
withheldRefusal with driver-sql's class statement for the same mistake.
A well-formed range lowers exactly as before.

Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx
Co-authored-by: Claude <noreply@anthropic.com>
… is not two bounds

Seven malformed ranges x find/count x unmarked/policy/author on both faces:
INVALID_FILTER / 400, own keys exactly code,status, the class statement on
the wire and the field (and, remotely, the comparand) in the log. A transport
arm row pins the arity branch under every mark, and two well-formed controls
(numeric, and bare days on a datetime column) hold the lowering still.

Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation tests tooling labels Sep 25, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

3 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see
  • the SDK route bridge reached 60 of 215 client-bound route-ledger rows — the other 155 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 155: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 100 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 6 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json fa00ebf44759b7fcfb0a1e9f55256265068399a8 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 260feffb77bdc9f5259da62bd3bf85ebd2a13bdc — the merge of head d14811988009df1869d3660cc2d3e26d8e075e72 into base fa00ebf44759b7fcfb0a1e9f55256265068399a8, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 260feffb77bdc9f5259da62bd3bf85ebd2a13bdc && git checkout 260feffb77bdc9f5259da62bd3bf85ebd2a13bdc
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin fa00ebf44759b7fcfb0a1e9f55256265068399a8 d14811988009df1869d3660cc2d3e26d8e075e72 && git checkout -B drift-repro fa00ebf44759b7fcfb0a1e9f55256265068399a8 && git merge --no-ff d14811988009df1869d3660cc2d3e26d8e075e72

node scripts/docs-audit/affected-docs.mjs --json fa00ebf44759b7fcfb0a1e9f55256265068399a8

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: d14811988009df1869d3660cc2d3e26d8e075e72

Scope: 4 files (+232/−13) on merge base aa04ea2964:

  • the $between arm of TursoDriver.toRemoteFieldSpec;
  • the $between arm of RemoteTransport.unsupportedOperator, with its doc paragraph;
  • the seam pin remote-transport-compile-refusal-seam.test.ts;
  • the changeset.

No governed surface is touched.

Read: card #20094 with its triage, unlock and claim, #20039, PR #20093 and its record 5826632400, the PR body, the diff, all check-runs, AGENTS.md, contract-review.md, and the code at head and base.

Measured in two detached worktrees (head and base), each with driver-turso's closure built. The probe drove a real remote TursoDriver over the in-repo libSQL stub beside TursoDriver(':memory:'), and it also drove RemoteTransport directly with a stub client and a diagnostic sink.

① Derived judgments

  • Every malformed shape: the premise is REAL at base and closed at head. The seven shapes were [x], [x,y,z], a number, a string, null, {} and []. Each ran under 3 marks through find and count on both faces: 84 cells per tree.
    • Base remote, 42/42: a plain Error with code and status undefined. It names 'object.field' in 42/42 and echoes the comparand in 24/24 literal cells, logs nothing, and is the same for every caller.
    • Head remote, 42/42: INVALID_FILTER / 400, with own keys exactly code,status. The field appears in 0/42 messages and the comparand in 0/24. There is exactly one warn line with the field and comparand, and 0 statements execute.
    • The head remote non-author message equals [RemoteTransport] plus the local class statement in 28/28 cells.
    • Local control: 42/42 cells are byte-identical at base and head.
    • aggregate, updateMany and deleteMany with [x] also answer INVALID_FILTER / 400 at head, with no row changed.
  • The refused set does not move: RIGHT. 17 nested and multi-operator shapes × {unmarked, 'policy'} × 2 faces = 68 cells, 0 moved. Well-formed shapes answer the same rows at base and head.
  • Well-formed ranges lower byte-identically: RIGHT. Six controls: a numeric range, bare days on a date field, bare days on a datetime field (the whole-day upper bound of framework#4081), instants, a range under $not and a range inside $or. The lowered filter, the executed SQL and args, the rows and the counts are identical at base and head in 6/6, and remote equals local in 6/6.
  • The pins discriminate: RIGHT, and both claimed counts reproduce. The seam pin passes 130/130 at head. With turso-driver.ts at its base blob it gives 28 failed / 102 passed; with remote-transport.ts at its base blob, 18 failed / 112 passed. Each file was restored by blob.
  • No throw remains in the remote lowering at head. The removed wording has no reader in the repository.
  • Public surface: RIGHT. Both changed methods are private. No export, index.ts, package.json or packages/spec file moves.
  • Existing pins: turso-remote-temporal-conformance.test.ts passes 67/67 at head.
  • CI at this head: 34 runs, 31 success, 3 skipped, 0 failures, and all seven required contexts are success.

② Semver level

@objectstack/driver-turso: patch, Clause-②: no: RIGHT.

③ Boundary flags

  • Changeset: every sentence is TRUE. The read-scope clause is measured through a 'policy'-marked where, not through the plugin itself.
  • PR body: no FALSE sentence. H1–H4, the 96 → 130 case count, both ablations and the acceptance notes reproduce. The whole-suite count and the gate families were not re-measured here; CI carries them.
  • The dev's deviations are answered.
    • Not extending the text-parity file is correct: it has 0 $between rows.
    • Omitting the tracker id from the new runtime string is justified by the prose-id baseline.
    • The remote author cost is accepted per ②.
    • The three out-of-scope observations are correctly left unfiled.

Implemented-by: claude/issue-20094-turso-remote-between-envelope
Reviewed-by: session_01Bvd69VPa6puiNzzPUroDBx

VERDICT: PASS

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review September 25, 2026 06:16
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Sep 25, 2026
Merged via the queue into main with commit 55daf89 Sep 25, 2026
35 of 36 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-20094-turso-remote-between-envelope branch September 25, 2026 06:32
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Sep 28, 2026
…efused by every driver that answers `$like`, instead of being cut at the NUL on SQLite (objectstack-ai#20041) (objectstack-ai#20124)

Fixes objectstack-ai#20041

Clause-②: yes (narrowing)

## What changed

On the SQLite faces `$like` / `$ilike` compile to `GLOB`, and SQLite
reads a pattern only up to its first U+0000. A pattern holding U+0000
was cut there and answered a different question, with nothing raised.
There is no NUL-safe SQLite pattern primitive to compile to instead
(`LIKE` cuts the same way, `replace()` cannot target U+0000, `instr()`
has no wildcards; measured on objectstack-ai#20024). So the accept set becomes one
contract: such a pattern is refused, the way a pattern ending in a lone
unpaired backslash already is.

- **`packages/spec`:** one exported predicate,
`hasNulInLikePattern(pattern)`, beside `hasDanglingLikeEscape` in
`src/data/filter.zod.ts`, with its own docblock. `hasDanglingLikeEscape`
is byte-identical. The converters (`likePatternToRegexSource`,
`likePatternToGlobPattern`, `matchesLikePattern`) are unchanged (see
H3).
- **Every door that refused a dangling escape now asks the new predicate
right after it**, in that door's existing envelope, `INVALID_FILTER` /
400:

| door | file | envelope |
|:--|:--|:--|
| `SqlDriver`'s filter walk (`classifyFilterKey`), before a dialect is
chosen | `driver-sql/src/sql-driver.ts` | new builder
`nulLikePatternError`, born in the withheld seam (`withheldFilterError`,
the operator map as the node, as for the dangling escape) |
| `RemoteTransport.buildWhereSQL`'s `$like` / `$ilike` arm, before
anything is sent | `driver-turso/src/remote-transport.ts` | new method
`nulLikePattern`, through `withheldRefusal`; its withheld sentence is
`driver-sql`'s behind the `[RemoteTransport]` prefix |
| `driver-memory`'s shape gate (`assertFieldConstraintShape`: the query
path and the reference matcher `match()`) |
`driver-memory/src/filter-refusal.ts` | new exported
`nulLikePatternError`, `driver-sql`'s author text word for word |
| `driver-memory`'s QueryAST `comparison` `like` / `ilike` arm, and the
`$like` translator floor | `driver-memory/src/memory-driver.ts` | the
same builder |

- The new check runs AFTER the dangling-escape check at every door, so a
pattern with both keeps the refusal it already had.
- **Messages.** The withheld class statement names neither the operator
variant, the field, the path nor the pattern. The author text (and the
server log) reads, for example: `Operator "$like" on field "v" at
filter.v.$like has a pattern holding the NUL character U+0000
("%\u0000"). ...` — the pattern goes through `JSON.stringify`, so U+0000
reaches no message as a raw byte.
- **Bounded in-place fix, declared:** the `$like` operator's
`.describe()` (`LIKE_DESCRIPTION`, same file) said "A pattern ending in
a lone unpaired backslash is refused (INVALID_FILTER)". It now reads "A
pattern ending in a lone unpaired backslash, or holding the NUL
character U+0000, is refused (INVALID_FILTER)", so the declared contract
names the refusal this PR enforces.
`content/docs/references/data/filter.mdx` is its regeneration
(`gen:schema && gen:docs`): 5 rows, that phrase only.
- **A pending release note corrected, which needs your confirmation:**
the last bullet of `.changeset/20024-sqlite-glob-stored-nul.md` (not yet
released) said "a pattern holding U+0000 is still cut there". At this
head that sentence is false, so it now says the pattern is refused by
every driver that answers `$like`. `node
scripts/check-empty-changeset.mjs` is RED on this by design: its
DELIBERATE CORRECTION class says "do NOT restore it -- say so on the PR
and get it confirmed". This is that statement. The alternative is to
drop the edit and leave that false sentence in the release's changelog
beside this PR's own entry.

## H1: every face, base vs head (measured, the PM's hypothesis holds on
the SQLite faces)

A probe (scratch, not committed) drove the public `find` of each face
over one row set: 13 stored values, 12 non-NULL, U+0000 at the start,
middle and end, alone, and none (`'a'` + U+0000 + `'b'`, `'ab'` +
U+0000, U+0000 + `'z'`, U+0000 alone, `'A'` + U+0000 + `'B'`, `plain`,
`''`, `ab`, `a`, `b`, `axb`, `AB`, NULL). Cases: 10 patterns holding
U+0000 (`'%'`+NUL, NUL+`'%'`, `'%'`+NUL+`'%'`, `'a'`+NUL+`'b'`,
`'a'`+NUL+`'%'`, `'_'`+NUL+`'_'`, backslash+NUL, NUL alone, `$ilike`
`'%'`+NUL+`'B'` and `'AB'`+NUL) and 9 NUL-free controls, each bare and
under `$not`. Base is `8d76c2d38c`, head is this branch (dists rebuilt
at each).

| face | U+0000 pattern, base | U+0000 pattern, head | NUL-free control,
base = head |
|:--|:--|:--|:--|
| `SqlDriver` on better-sqlite3 | 20 of 20 differ from `formula`, 0
refused | 20 of 20 refused, `INVALID_FILTER` / 400 | 0 of 18 differ on
the NUL-free rows; 12 of 18 differ over rows holding a stored U+0000
(objectstack-ai#20024 item 2 (ii), not this card) |
| `SqliteWasmDriver` | 20 of 20 differ, 0 refused | 20 of 20 refused |
same as above |
| `TursoDriver` local | 20 of 20 differ, 0 refused | 20 of 20 refused |
same as above |
| `TursoDriver` remote, `makeLibsqlSqliteStub` | 20 of 20 differ, 0
refused | 20 of 20 refused | same as above |
| `TursoDriver` remote, real `@libsql/client` `file::memory:` | 20 of 20
differ, 0 refused | 20 of 20 refused | same as above |
| `InMemoryDriver.find` (`$`-spelling) | 0 of 20 differ (answers
correctly), 0 refused | 20 of 20 refused | 0 of 18 differ |
| `InMemoryDriver.find`, QueryAST `comparison` `like` / `ilike` |
answers (`like '%'`+NUL gives the one value ending in U+0000) | refused
| `like 'ab%'` answers the same rows |
| `driver-memory` `match()` | answers (`true` for `'ab'`+NUL) | refused
| pinned equal to the query path |
| `@objectstack/formula` | the oracle | unchanged (see H2) | the oracle
|
| `driver-mongodb` `translateFilter` | 38 of 38 cases refused: `$like`
is not translated at all | unchanged, not touched (held by draft PR
objectstack-ai#19947) | refused |

- The five SQLite faces gave byte-identical answer lists on every case,
at base and at head. The control answers are byte-identical base vs head
on all six driver faces.
- Examples at base, SQLite faces: `$like: '%'` + U+0000 returned all 12
non-NULL rows, where `formula` returns the two ending in U+0000; `$like:
'a'` + U+0000 + `'b'` also returned `'a'`; `$like: '_'` + U+0000 + `'_'`
also returned `'a'` and `'b'`; `$ilike: 'AB'` + U+0000 also returned
`'AB'` and `'ab'`; `$not $like '%'` + U+0000 returned only the NULL row.
- Also measured at base, not touched: objectql `applyHaving` refuses
every `$like` (`INVALID_FILTER` / 400, "Unsupported operator '$like' in
`having`"); service-analytics `compileScopedFilterToSql` refuses it
(`READ_SCOPE_COMPILE_FAILED` / 500, fail-closed) and
`normalizeAnalyticsFilterTree` refuses it (`INVALID_FILTER` / 400).
`lowerAnalyticsWhere` alone passes the node through; the tree build
after it refuses.
- The Postgres and MySQL arms of `driver-sql` were not measured live (no
server here). At head the refusal fires on the walk before the dialect
is chosen, pinned by compiling with the `pg` and `mysql2` clients and no
server.

## H2: the doors (census)

`git grep -n -E '\bNAME\(' HEAD -- 'packages/**/*.ts' ':!**/*.test.ts'`,
comments and the definition excluded:

- `hasDanglingLikeEscape`: 7 call sites at base and at head. Five are
face doors: `filter-refusal.ts` (1), `memory-driver.ts` (2),
`sql-driver.ts` (1), `remote-transport.ts` (1). Two are the converters'
own backstops in `filter.zod.ts`. Positive control: the PM's list
(`sql-driver.ts`, `remote-transport.ts`, `memory-driver.ts` x2,
`filter-refusal.ts`) is exactly the five doors.
- `hasNulInLikePattern`: 0 at base, 5 at head, one beside each of the
five doors.
- Every one of the five doors refused a dangling escape at base, and
every one must refuse U+0000. `formula` does NOT refuse a dangling
escape: `matchesLikePattern` throws, and the arm answers `false`
(measured: `matchesFilterCondition({ v: 'abc\\' }, { v: { $like: 'abc\\'
} })` is `false`). So by the claim's condition its file is not touched,
and it still evaluates a U+0000 pattern.

## H3: where the predicate is called, option (a)

Converter consumer census, same grep over non-test sources:

- `likePatternToGlobPattern`: 3 calls: `sql-driver.ts`
(`likePatternPredicate`), `remote-transport.ts` (`pushLikePattern`), and
`driver-memory/src/memory-analytics.ts` (`globSubstringPattern`, the
analytics echo of a `$contains` comparand).
- `likePatternToRegexSource`: 3 calls: `memory-driver.ts` (2) and the
spec's own `matchesLikePattern`.
- `matchesLikePattern`: 2 calls: `driver-memory/src/memory-matcher.ts`
and `formula/src/matches-filter.ts`.
- 8 calls in all: `packages/drivers/**` 6, `packages/formula` 1,
`packages/spec` 1. `packages/services/**`, `packages/objectql` and
`packages/plugins/**`: 0 (the same grep, whose drivers count is the
positive control).

Option (b), a throw inside the converters, would have changed three
consumers beyond this surface: `memory-analytics`' `$contains` echo
would throw a plain `Error` on a comparand holding U+0000; `formula`'s
`$like` would answer `false` (its caught throw) instead of the right
rows; the reference matcher the same (though its shape gate runs first).
So the predicate is called at each door (a), and
`filter-like-nul-pattern.test.ts` pins that the JS translation keeps its
meaning.

## H4: the withheld seam

- `nulLikePatternError` has a row in
`sql-driver-compile-refusal-seam.test.ts` and `nulLikePattern` in
`remote-transport-compile-refusal-seam.test.ts`; without the row, "every
builder that goes through the seam is driven by a row below" is red. The
remote pin's local-vs-remote table gained the class too (the withheld
sentence is one sentence on both compilers).
- policy-marked: `INVALID_FILTER` / 400, the error's own keys exactly
`code,status`, no field or pattern on the wire, both in the log / sink.
`'author'`: the full text. Unmarked: byte-identical to policy. Merged
`$and`: the refusing arm's mark decides, in both arm orders.
- Through `TursoDriver` in remote mode no mark survives
`toRemoteFilter`, so an author gets the class statement there (the
objectstack-ai#20093 fail-closed cost, pinned in
`turso-20041-like-nul-pattern.test.ts`).

## H5: declaration

- `Clause-②: yes (narrowing)`: the PR adds one public export,
`hasNulInLikePattern` on `@objectstack/spec/data`
(`api-surface/data.json` +1), so the value is `yes` (AGENTS.md's
Clause-② rule, the PR objectstack-ai#20104 precedent), and the arm is `(narrowing)`
for the refused patterns. The claim carried `no (narrowing)`; corrected
in the patch round after contract review 5828387721. The changeset
grades `@objectstack/spec`, `driver-sql`, `driver-sqlite-wasm`,
`driver-turso` and `driver-memory` `minor`, with **BREAKING** and a `!`
title, following PR objectstack-ai#19971's changeset.
- `node scripts/check-changeset-no-major.mjs --base 8d76c2d`: "✓ This
diff introduces no `major` bump." (exit 0)
- `node scripts/check-adr-0087-registration.mjs --base 8d76c2d`: "✓
check-adr-0087-registration: 1 declared-breaking changeset(s), each
carrying an ADR-0087 disposition." — `[BREAKING+bang+clause-②-narrowing]
not-required (no-migration-prescription)` (exit 0). The gate chose
`not-required`: no key, schema, object definition or stored
representation moves, and no rewrite of a stored pattern keeps its
meaning.

## Compile surfaces (`references/compile-surfaces.md`, re-verified with
its grep: 92 hits in non-test sources)

| # | face | this PR |
|:--|:--|:--|
| 1 | `driver-sql` `applyFilterCondition` (`sql-driver.ts:15953`);
`driver-sqlite-wasm` and Turso local by inheritance | CHANGED: the walk
refuses a U+0000 pattern on every dialect |
| 2 | Turso `RemoteTransport.buildWhereSQL` (`remote-transport.ts:2668`)
| CHANGED: the `$like` / `$ilike` arm refuses it |
| 3 | service-analytics `compileScopedFilterToSql`
(`read-scope-sql.ts:602`) | not touched (claim excludes it): refuses
every `$like` already, measured |
| 4 | service-analytics `lowerAnalyticsWhere`
(`filter-normalizer.ts:2015`) | not touched: passes the node through;
`normalizeAnalyticsFilterTree` refuses every `$like`, measured |
| 5 | `formula` `matchesFilterCondition` (`matches-filter.ts:212`) | not
touched: it refuses no dangling escape, so it is not one of the doors;
it evaluates a U+0000 pattern correctly (the oracle above) |
| half | objectql `applyHaving` / `matchesHaving`
(`having-filter.ts:279` / `:292`) | not touched: refuses every `$like`,
measured |
| thawed | `driver-memory` `checkCondition` (`memory-matcher.ts:361`)
and the query path | CHANGED: the shape gate both run first refuses it;
the QueryAST arm and the translator floor too |
| thawed | `driver-mongodb` `translateFieldOperators`
(`mongodb-filter.ts:832`) | not touched (draft PR objectstack-ai#19947 holds it):
refuses every `$like` through its `default:` arm, measured through
`translateFilter` |

## Tests

New pins, each asserting `code`, `status` and the path (never a bare
`toThrow()`), plus NUL-free controls and the dangling escape beside
U+0000:

- `packages/spec/src/data/filter-like-nul-pattern.test.ts` (5): the
predicate, the escaped U+0000, its independence from the dangling
escape, and the converters unchanged.
- `driver-sql/src/sql-driver-20041-like-nul-pattern.test.ts` (84): 9
patterns x bare / `$not` / `$or` / `$and`, unmarked (class only, path in
the log, through `find` and `count`) and author-marked (operator, field,
path); the dangling escape keeps its refusal also beside U+0000; 8
controls equal `formula`; `sqlite`, `pg` and `mysql2` compiles refuse on
the walk.
- `driver-sqlite-wasm/src/sqlite-wasm-20041-like-nul-pattern.test.ts`
(16), the same through sql.js, controls checked against `formula`.
- `driver-turso/src/turso-20041-like-nul-pattern.test.ts` (17): local,
remote over the stub and remote over a real libSQL engine; no `FROM`
statement reaches the engine for a refused filter; 8 controls, equal on
all three transports.
- `driver-memory/src/memory-20041-like-nul-pattern.test.ts` (21): the
query path, the QueryAST spelling and `match()`, with controls.
- Consumer pins changed: the two seam enumerations above (one row each;
the remote one also one local-vs-remote row). No other consumer pin
reads a U+0000 `$like`: `git grep` over every `*.test.ts` that names
`$like` / `$ilike` and U+0000 finds none outside this PR.

Suites (`vitest run --maxWorkers=2`), at `5f2e6f246d` (code equal to
this head but for a comment in `memory-driver.ts`), dists rebuilt:

- `driver-sql`: 186 files passed, 11 skipped; 3024 tests passed, 170
skipped.
- `driver-turso`: 69 files, 1629 tests passed.
- `driver-sqlite-wasm`: 33 files, 622 tests passed.
- `driver-memory`: 53 files, 1269 tests passed (re-run at `049b3a6c95`,
same).
- `formula`: 35 files, 978 tests passed.
- `spec`: 570 files, 16369 tests passed, 2 todo.
- `plugin-auth` (consumer): 114 files, 2440 tests passed.
- `typecheck` exits 0 in `spec`, `driver-sql`, `driver-sqlite-wasm`,
`driver-turso`, `driver-memory`. `tsc --listFiles` counts each new
driver test once in its package program; the spec test once in
`tsconfig.test.json`, which `check:test-typecheck` compiles.

## Before-red and ablations (one-off, no file left behind)

All from committed state, through `scripts/ablation-replace.mjs` (anchor
1 -> 0 on disk, blob changed), restored with the blob equal to `HEAD`,
`git diff HEAD` empty and `git status --porcelain` empty.

1. **Every driver's predicate replaced by a never-true local** (the four
source files at once, which is base behaviour at every door).
`driver-sql` rebuilt and `ablation-dist-preflight` found the marker in
`dist/` before the run. Predicted and observed exactly:
- `sql-driver-20041`: 75 failed, 9 passed (the controls, the dangling
escape, and nothing else green);
- `sql-driver-compile-refusal-seam`: 4 failed, 91 passed (the new row);
- `sqlite-wasm-20041`: 10 failed, 6 passed; `turso-20041`: 8 failed, 9
passed; `remote-transport-compile-refusal-seam`: 6 failed, 96 passed;
`memory-20041`: 15 failed, 6 passed;
- the dangling-escape suites `sql-driver-like-pattern` (22) and
`memory-like-pattern` (15) stayed green.
Restored, rebuilt, and `ablation-dist-preflight --absent` passed with
the tree clean.
2. **The new builder bypassing the seam** (`nulLikePatternError` calling
`unsupportedFilterError`, `nulLikePattern` calling
`invalidFilterError`). Predicted and observed: `driver-sql` 78 failed of
179 (both enumeration assertions, the row's 4, and the 72 path / log
cases of the new suite); remote seam pin 6 failed of 102.

## Gates (at `049b3a6c95`)

- `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` from the real diff (17 paths, merge base
`8d76c2d38c`): 110 commands, the PM's 79 plus 31 the docs,
`driver-memory` and changeset paths add. All 110 run after a full `turbo
run build --filter='./packages/*' --filter='./packages/*/*'` (72 tasks);
`--ran`: "110 derived, 110 run, 0 NOT-MEASURED, 0 UNRUN".
- 109 exit 0. **One exits 1, on purpose:** `node
scripts/check-empty-changeset.mjs --base origin/main`, the DELIBERATE
CORRECTION of `.changeset/20024-sqlite-glob-stored-nul.md` above.
- Roster families beside these paths, run by hand:
`check-changeset-fixed`, `check:meta-url-spelling`,
`check:spec-changes`, `check:authz-resolver`, `check:error-code-casing`,
`check:filter-alias-parity`, `check:engine-double-contract`,
`check:error-status-conformance`: all exit 0.
- `pnpm --filter @objectstack/spec check:generated`: all 15 artifacts up
to date after `gen:api-surface`, `gen:export-origins`, `gen:schema` and
`gen:docs` (`api-surface/data.json` and `export-origins/data.json` gain
`hasNulInLikePattern`; `authorable-surface.base.json` untouched).
- `pnpm check:driver-conformance`: "50 covered cell(s), 0 in the DEBT
ledger, 0 exempt" at base and at head.
- `node scripts/check-issue-citations.mjs --base 8d76c2d`: 17
citations across 5 files, all resolve.
- `pnpm check:nul-bytes` passes, and a control-byte scan of the changed
files finds none. U+0000 is spelled `String.fromCharCode(0x00)` in every
file.
- **Re-run at the merged head `b88ca46d90`** (merge base `6780e34af5`,
17 paths, after a full `turbo run build`): the derivation gives the same
110 commands; 109 exit 0 and `check-empty-changeset` exits 1 on the
objectstack-ai#20024 correction; `--ran`: "110 derived, 110 run, 0 NOT-MEASURED, 0
UNRUN". `check-changeset-no-major` and `check-adr-0087-registration`
exit 0; `check:generated` all 15 up to date.
- ESLint, narrowed: `--no-inline-config --format json` over the 12
changed `.ts` files reports 12 files, 0 errors, 0 warnings.
`eslint.config.mjs` enables no type-aware linting (the printed
`parserOptions` for `sql-driver.ts` are
`{"ecmaVersion":"latest","sourceType":"module"}`), so this diff cannot
move the verdict of a file it does not touch. The full `pnpm lint` is
CI's.

## Acceptance notes

- **Not this card:** a NUL-free pattern matched against a STORED value
that holds U+0000 still differs on the SQLite faces (12 of 18 control
cases in the probe, unchanged). That is objectstack-ai#20024 item 2 (ii), and objectstack-ai#20024
remains open for it.
- `content/docs/protocol/objectql/query-syntax.mdx` still names only the
dangling escape among refused patterns. It is not false, only
incomplete, and it is outside this claim's file surface, so it is left
for the next PR that touches the page.
- `formula` keeps evaluating a U+0000 pattern (correctly). A write-side
`check` with such a pattern therefore answers where the read side
refuses; the read side refusing loudly means the two can no longer
silently disagree.
- `origin/main` at `6780e34af5` is merged into this branch (merge commit
`2b8dd90200`, no conflicts): the 10 commits past the old merge base
`8d76c2d38c` are `aa04ea2964`, `fa00ebf447`, `7b27bd00c7`, `7a13e0562a`,
`7c1039b388`, `55daf89d74`, `226e00c038`, `7b068877ce`, `0d73ff6245`,
`6780e34af5`. One of them touches this PR's paths: `55daf89d74` (PR
objectstack-ai#20114) changes `packages/drivers/driver-turso/src/remote-transport.ts`
and `remote-transport-compile-refusal-seam.test.ts`; both auto-merged,
and the branch's delta against the new merge base is still exactly its
17 files, +1017/−16. After the merge: `turso-20041-like-nul-pattern`
17/17 and `remote-transport-compile-refusal-seam` 136/136 (102 plus
objectstack-ai#20114's rows), `driver-turso` 69 files / 1663 tests;
`sql-driver-compile-refusal-seam` and
`sql-driver-20041-like-nul-pattern` 179/179; `sqlite-wasm-20041` 16/16,
memory 36/36, spec 31/31.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

2 participants