fix(service-analytics)!: the NativeSQL execute face and the /analytics/sql echo refuse a read scope the shared comparand faces refuse

[objectstack-fleet]

Fixes #20018

Clause-②: no (narrowing)

What changed

compileScopedFilterToSql (packages/services/service-analytics/src/read-scope-sql.ts) is the read-scope lowering behind two analytics faces:

• the NativeSQL execute face, NativeSQLStrategy.applyReadScope, for the base table and every joined hop;
• the /analytics/sql echo, ObjectQLStrategy.generateSql.

It is also a public export of the package. Once its own lowering returns, it now calls assertReadScopeComparandsRunnable on the scope. That is the helper PR #20017 added in the same module, and it runs @objectstack/spec/data's assertListComparandShapes and normalizeFilterComparandTypes on the scope alone.

A scope those faces refuse now gets READ_SCOPE_COMPILE_FAILED / 500 with the message withheld (the #5367 ruling, re-affirmed as #7598 Q2 = A) on the native face and the echo. The refusal comes before any statement is built or executed. That is the answer the ObjectQL execute face has given since PR #20017, so one read scope gets one verdict on all three analytics faces.

• Files: the call itself is one line. The rest of the diff is:
  • the module-header section (service-analytics: the NativeSQL read-scope compiler and the /analytics/sql echo compile two scope shapes the shared comparand faces refuse (plain-object comparand under $eq, null member in $in): one scope, two answers across faces #20018);
  • a note on the helper's docblock;
  • a one-paragraph note on the binary extra in comparand-shape.ts, whose "accepted in every bind position" is no longer true of the read-scope door;
  • tests and the changeset.
• Not touched: objectql-strategy.ts and native-sql-strategy.ts. Both faces reach the guard through the compiler, so neither needs a call site of its own.
• Also not touched: filter-normalizer.ts, preview-evaluator.ts and packages/spec.

Measurement, recorded before the fix

Everything in this section was measured on 9d81af714f (origin/main, pre-fix). The rows are executed, not read from the compiled string.

• Harness: a scratch probe that is not committed, plus measurement commit 8c80d9c3d2 (the new test file alone).
• Database: one real SqliteWasmDriver with four fixture rows. region is NULL on d3, and owner and amount are NULL on d4.
• ObjectQL face: a real ObjectQL engine.
• Echo: its SQL was run on the same database.
• Native: NativeSQLStrategy.execute through executeRawSql on the same database. It was not stubbed.
• Scopes: the getReadScope contract, filled by hand.

read-scope shape class	ObjectQL execute	echo (SQL executed)	native execute
plain-object comparand under $eq (the card's first shape)	READ_SCOPE_COMPILE_FAILED / 500	compiles; the database refuses the bind	DATABASE_ERROR / 500
null member in $in, ['emea', null] (the card's second shape)	500	d1	d1: the NULL member matches nothing
null-only $in	500	no rows	no rows
null member in $in under $not	500	d3	d3: only the NULL row, which the scope names as excluded
null member in $nin	500	d3	d3: only the NULL row, which the scope names as excluded
null comparand under $gt / $lte	500	no rows	no rows
null $between bound	500	no rows	no rows
blank $between bound	500	d1 d2 d4	d1 d2 d4
plain-object comparand under $ne / $gt; empty object under $eq	500	compiles; DB refuses	DATABASE_ERROR / 500
bigint beyond 2^53 (implicit, $in)	500	no rows	no rows
binary comparand, binary $in member	500	no rows	no rows
Map or function comparand	500	compiles; DB refuses	DATABASE_ERROR / 500
controls (9 well-formed scopes, including the null predicates and the live RLS composite)	the same rows on all three faces

The card named two shapes. The measurement found the gap is every shape the two shared faces refuse and this compiler's own gates did not: null list members and bounds, null ordering comparands, blank bounds, oversized bigints, binary, and non-scalar objects in a scalar position. That is the set that moves.

The mechanism hypotheses

• Hypothesis 1 is confirmed, and the set is wider than the card's two shapes. These are the compile arms that accepted them:
  • case '$eq': return val === null ? … : `${col} = ${bind(params, val)}`;. A plain object is bound, and no gate judges a $eq comparand's type. assertNoFieldReferenceComparand steps past an object that is not { $field: string }.
  • case '$in' → assertCompilableMembers(op, field, val) → isBindableComparand(member), which admits null (an accepted comparand type) and binary (a package-local extra), then IN (…) binds each member.
  • The ordering arms and $between bind whatever passes those gates. The implicit-equality arm binds any non-object.
• Hypothesis 2: measured. See the table above, and the producers section below.
• Hypothesis 3 is confirmed in verdict, but the placement is better than "at the entry". The two walks are pure functions of the scope, so the placement cannot change which scopes are refused, only which log sentence a doubly-refused scope carries.
  • Ablation A2 below put the call at the entry and turned 37 tests red in 7 files: the new ordering pin, plus 36 existing log-sentence pins, for example read-scope-eq-array-refusal (15) and read-scope-undefined-comparand (8).
  • After the lowering, every shape this compiler already refused keeps its own sentence (the analytics: read-scope-sql's ruled $not-over-$in-empty residue has no open card — and #13640 turned it into an echo-vs-execution disagreement on the ObjectQL strategy #13926 ordering), and the faces add only what would otherwise have been lowered.

Producers: who can emit these shapes today

All readings below are on 9d81af714f.

producer	emits a refused shape?	evidence
RLS using predicates, through compileCelToFilter → RLSCompiler.compileFilter	yes, from an authored predicate. f in ['a', null], f in [null], !(f in ['a', null]), f > null, f <= null and f != current_user each lower verbatim into a refused shape.	Scratch probe. All six pass isSupportedRlsExpression, and validateRlsPredicateEnforceability returns 0 findings for each.
authored policies in this repository	none	git grep of using / check predicates for a null list member or a null ordering comparand, over examples/** and packages/**/*.ts (tests excluded): 0 matches, exit 1. The control on the same tree and file set, predicates naming current_user, matched 77 lines in 7 files, exit 0.
a resolved membership variable with a null member	no	The #13496 guard. Measured: f in current_user.teams with a null member lowers to the deny sentinel.
plugin-sharing buildReadFilter	no	Owner ids are String(userId) or a resolver's string[]. grantedRecordIds filters out null and ''.
a host getReadScope option, or a direct caller of the export	anything	The door the card names.

Verdict: no producer both legitimately authors one of these shapes and relies on the native answer. Three facts support that:

• The in-repo producer emits these shapes only from predicates that the standing rulings, carried by the shared faces, refuse. No policy in this repository is one of them.
• The ObjectQL analytics face already refused every such scope.
• The native answer was not the predicate's meaning. It dropped the null member, admitted only the NULL rows the scope excludes, returned zero rows, or hit a database error.

So this is execution under the rulings, not a needs_decision. The authoring half is reported separately as a finding; see the Acceptance notes.

Tests

The new file is packages/services/service-analytics/src/__tests__/read-scope-comparand-three-faces.test.ts, with 32 cases. It uses one SqliteWasmDriver, a real ObjectQL behind the ObjectQL face, and the echo's SQL executed.

• 13 refusal classes. Each asserts on all three faces: code READ_SCOPE_COMPILE_FAILED, status 500, and the prose withheld. "Withheld" means serverFaultProvenance(resolveThrownHttpError(err, 500)) is 'declared' and declaredRefusalMessage(err) is undefined.
• The native face refuses before any statement reaches the database. Zero executeRawSql calls across all 13 classes.
• The joined hop. applyReadScope's per-hop lowering refuses a joined object's scope, named for that hop.
• The public export refuses every class, and a well-formed scope compiles to the same bound predicate as before.
• Log sentences. A shape the compiler already refused, a list under $eq, keeps its own sentence. A shape only the faces refuse carries their sentence, the same on all three faces.
• Controls:
  • with no scope, every face serves all four rows;
  • 9 well-formed scopes admit the same rows on every face. They include { region: null }, $ne: null, a non-empty $nin, the live RLS composite with an emptied $in beside an own-rows grant, and the spelling the null-member ruling prescribes ($or of $in and $null: true);
  • a well-formed caller where composes with a well-formed scope;
  • a caller where in a refused scope shape answers exactly as it does with no scope, and is never attributed to the read scope.

Existing pins that asserted the lowering binds a refused shape. Each is re-judged with a [#20018] note, and each keeps what it was there to pin:

• comparand-door-single-source.test.ts. Three matrix cells now read READ_SCOPE_COMPILE_FAILED/500: null under scopeIn, binary under scopeIn and scopeEq, and plain object under scopeEq. This PR moves no where-door cell.
  • After merging fix(service-analytics)!: the analytics where door runs every arm of the shared comparand-shape face on the object spelling (#20010) #20032 (service-analytics: object-form analytics where skips the shared comparand-shape face's other arms ($in null member, $gt: null, null/blank $between bound, scalar $in) that the FilterArray spelling refuses 400 #20010), the null row carries both re-judged cells: whereIn is INVALID_FILTER/400 (service-analytics: object-form analytics where skips the shared comparand-shape face's other arms ($in null member, $gt: null, null/blank $between bound, scalar $in) that the FilterArray spelling refuses 400 #20010) and scopeIn is READ_SCOPE_COMPILE_FAILED/500 (this PR).
  • The "six accepted types, every position" check names each moved cell with the change that moved it, and holds every other cell at accept.
• comparand-shape-refusal.test.ts. "Keeps binding every legitimate $in member" drops null, and a new case pins null refused with the face's sentence.
• cross-field-reference-refusal.test.ts. { $gt: { $field: 5 } } is refused as a plain object, with the type face's sentence and not the field-reference gate's.
• read-scope-boolean-flag-comparand.test.ts. An inherited $null still cannot trip the flag gate, because the refusal carries no flag sentence. The object is refused by the type face as a non-plain value.
• read-scope-undefined-comparand.test.ts. $in: [null], $nin: [null] and $between: [null, 5] leave the null control group for their own "refused by the shared face" block. Every null predicate stays in the group, unmoved.

Results:

run	tree	result
new file, measurement commit (test only)	8c80d9c3d2	Tests 17 failed | 15 passed (32). Every refusal class and the three pins built on them are red; every control is green. The first face to fail in each row is the echo; the ObjectQL face passed first.
whole package	0fbed27877 (final; origin/main adbbc5d01e merged)	Test Files 120 passed (120), Tests 2689 passed (2689)
pnpm --filter @objectstack/service-analytics typecheck	0fbed27877	exit 0; tsc --noEmit --listFiles includes the new test file

Ablations. Both ran through node scripts/ablation-replace.mjs in WRAP mode (the anchor must hit exactly once, the blob must change, and the tool's own trap restores). The subject resolves to src/ through relative imports, so no dist/ leg applies. Each restore was proven: blob 34705e267dba equals HEAD, and git diff HEAD is empty.

A1 was re-run on the final head 0fbed27877. A2 ran on f293340e94. read-scope-sql.ts is the same blob, 34705e267dba, on both trees (neither merge touched it), so A2's placement result stands.

leg	mutation	result (whole package)
A1 (at 0fbed27877)	delete the new assertReadScopeComparandsRunnable(filter, alias); call	26 failed | 2663 passed (2689): all 17 negative pins in the new file, plus the 9 re-judged pins (3, 1, 1, 1 and 3 across the five files). Every control stayed green. The same 17 + 9 went red at f293340e94 (26 failed | 2567 passed (2593)).
A2 (at f293340e94)	call it BEFORE compileNode instead of after	37 failed | 2556 passed (2593): the new ordering pin, plus 36 existing log-sentence and precedence pins in 6 files

The first attempt at A2 was a void run. Its replacement text contained its anchor, so the tool refused with "anchor count moved 1 -> 1" and restored. Nothing was measured. It was re-run with a three-line anchor, and that is the result above.

Gates

Derived gates. Derived at 0fbed27877 with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands: 61 families, not stale. They are a superset of the 48-line dispatch-time list.

• 59 exited 0.
• check:dual-build-cjs-loads and check:type-check-debt first answered PREREQUISITE NOT MET (exit 3), because the workspace had no dist/. After building every ./packages/** workspace package (VERDICT command-exit 0), both exited 0: check:dual-build-cjs-loads passes its floors, and check:type-check-debt re-measured 4 ledger entries (53 raw tsc errors) with none above its recorded number.
• The --ran reconciliation: 61 derived, 61 run, 0 NOT-MEASURED, 0 UNRUN. That zero is derived, not claimed: all 61 records carry an exit code and none is 3.

Other checks:

• GITHUB_TOKEN=… node scripts/check-issue-citations.mjs: exit 0; 8 citations judged, 8 resolve.
• node scripts/check-adr-0087-registration.mjs --base origin/main (one of the derived families, at 0fbed27877): exit 0. The changeset is BREAKING+bang+clause-②-narrowing, disposition not-required (no-migration-prescription).
• Lint, narrowed to the 8 touched TypeScript files with eslint --no-inline-config --format json at 0fbed27877: 8 files read, 0 errors, 0 warnings, none ignored.
  • The changeset is outside eslint's population ("no matching configuration").
  • eslint.config.mjs enables no type-aware linting: its parserOptions carry only ecmaVersion and sourceType, with no project and no projectService. So this diff cannot move a verdict on an untouched file.
  • pnpm lint itself is CI's.

Acceptance notes

• BREAKING, minor with !. Read scopes the native face and the echo used to serve are refused now; the table above lists what they served. The changeset carries the banner and the fix spellings: the null predicate beside a membership, $gte / $lte for a one-sided range, and a scalar comparand.
• The refusal set is the shared faces' set, not only the card's two shapes. Judging the scope with the same function the ObjectQL face uses is what makes the verdicts equal. A narrower hand-picked subset would have left the other rows of the table as "one scope, two answers".
• Binary. The package-local binary bindable (isBindableComparand) no longer reaches the read-scope door. The shared type face refuses binary, and the ObjectQL face already did. The where door and the predicate itself are unchanged.
• The joined-hop log sentence names the alias. compileScopedFilterToSql knows only the alias, so the operator's log reads read scope for "ALIAS": the object name on the base table, the join alias on a hop. The response withholds it either way.
• Precedence on the native face. When a scope carries both a shape this compiler refuses and one only the faces refuse, the compiler's sentence answers. The verdict is the same either way.
• The CRUD path is not an analytics face and was not measured here. The security middleware composes the same RLS filter into the engine's where after the engine's shared-face seam has run. What driver-sql answers for these shapes there is outside this card.
• Finding, class (c), not fixed here, for the seat to file. The RLS authoring surface admits predicates that lower into scope shapes the shared faces refuse: a literal null in an in list, an ordering comparison against null, or a comparison against the whole current_user object. All three pass isSupportedRlsExpression, and validateRlsPredicateEnforceability returns no finding for them. They are refused (500) on every analytics face after this PR. The evidence is in the producers table.
• origin/main was merged in twice.
  • At b3735968ba: three commits in packages/objectql and packages/plugins/plugin-security, sharing no path with this diff.
  • At adbbc5d01e: fix(service-analytics)!: the analytics where door runs every arm of the shared comparand-shape face on the object spelling (#20010) #20032 (service-analytics: object-form analytics where skips the shared comparand-shape face's other arms ($in null member, $gt: null, null/blank $between bound, scalar $in) that the FilterArray spelling refuses 400 #20010) and a driver-sql / driver-turso fix. That merge had one content conflict, in the comparand matrix, resolved as above.
  • comparand-shape-refusal.test.ts and cross-field-reference-refusal.test.ts auto-merged and were re-read on the merged tree. fix(service-analytics)!: the analytics where door runs every arm of the shared comparand-shape face on the object spelling (#20010) #20032's edits there are to the where-door pins and this PR's are to the read-scope pins, and their notes agree: the null member is refused by the same ruling at each door, each in its own envelope. One sentence in the non-string $field case ("it binds as JSON") is now scoped to the where door, where it still holds.
  • The dependency closure was rebuilt before the final runs.
• Files not touched: filter-normalizer.ts, preview-evaluator.ts, objectql-strategy.ts, native-sql-strategy.ts, packages/spec.
• security: the analytics ObjectQL execute face answers a row-level read scope it cannot run with INVALID_FILTER / 400 whose message echoes the policy's field name and comparands — the disclosure #5367 closed for the native / echo faces #19995 stays open behind engine + driver-sql: seven filter-refusal doors ignore the #8220 provenance mark, so a refusal inside a policy read-scope subtree relays the policy field (and some comparands) in its 400: the #7929 redaction is not applied #20020, for its engine- and driver-door residue. This PR does not address it.

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

1 anchor(s) derived from 1 changed package(s); no hand-written page names any of them. ⚠️ 1 changed file(s) yielded no anchor (packages/services/service-analytics/src/comparand-shape.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

What this run could not see

• 1 changed file(s) yielded no anchor (packages/services/service-analytics/src/comparand-shape.ts) — pages documenting those are invisible to this run
• the SDK route bridge reached 60 of 215 client-bound route-ledger rows — the other 155 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 155: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 100 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
• a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 7e6ca1787aa97a98dd88b1d6fe32ec1962655edc → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 3768208dc084c84372d8f59559bd986d30e5b612 — the merge of head 0fbed27877e5a70ec591352d32cc3250bf4fe64d into base 7e6ca1787aa97a98dd88b1d6fe32ec1962655edc, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 3768208dc084c84372d8f59559bd986d30e5b612 && git checkout 3768208dc084c84372d8f59559bd986d30e5b612
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 7e6ca1787aa97a98dd88b1d6fe32ec1962655edc 0fbed27877e5a70ec591352d32cc3250bf4fe64d && git checkout -B drift-repro 7e6ca1787aa97a98dd88b1d6fe32ec1962655edc && git merge --no-ff 0fbed27877e5a70ec591352d32cc3250bf4fe64d

node scripts/docs-audit/affected-docs.mjs --json 7e6ca1787aa97a98dd88b1d6fe32ec1962655edc

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.