fix(service-analytics)!: the analytics where door refuses a list in the equality slot instead of reading it as IN (#19888)

[objectstack-fleet]

Fixes #19888

Clause-②: no (narrowing)

What this changes

The analytics where door (lowerAnalyticsWhere in packages/services/service-analytics/src/strategies/filter-normalizer.ts) now refuses a list in the equality slot of an object-form filter, { f: [...] } and { f: { $eq: [...] } }, with INVALID_FILTER / 400. This applies ruling 乙 of #19757 (record 5793368540: 「an array in the implicit-equality slot is refused at the shared face, for every driver at once」) to the one analytics spelling that never reached the shared face.

The new gate, assertNoListInEqualitySlot, walks the object-form condition the way the shared face does: $and / $or, $not and field entries, from the same where path seed. It hands each equality-slot list to assertListComparandShapes (@objectstack/spec/data) as a one-entry node. That node holds nothing but the list, so only the face's equality arm can fire, and the refusal is the face's own: its envelope, wording, path and $in prescription. The FilterArray spelling of the same condition was already refused inside parseFilterAST. Both spellings now produce the same bytes, and a test pins this.

Every filter source of this door passes through the gate: the caller where, a dataset's scope filter and a measure's filter. That holds on the NativeSQL execute path, the /analytics/sql echo and the ObjectQL engine path. The draft-data preview (preview-evaluator.ts) calls the same exported gate, so a drafted chart refuses what the published chart refuses. The in reading is deleted from fieldLeaves. So are the two arms that existed only because of it: the bare-array member sweep in assertDefinedComparands and the bare-array guard in nullGuardForFieldSpec.

Two details:

• Nested relations. A nested-relation object ({ acct: { region: [...] } }) is descended too. The shared face leaves such an object alone, because a driver reads it as a deep-equality comparand or another object's condition. This compiler flattens it to the dotted member acct.region, and the list is in that member's equality slot.
• Precedence. The list is diagnosed before any member gate. { d: [1, undefined] } gets the list refusal, not the undefined-comparand one. The shared face and read-scope-sql.ts use the same order.

$ne with a list is not judged, because ruling 乙 names equality only. The list operators keep their lists, including $in: [] and $nin: []. Every scalar compiles as before, including null.

Measured first (recorded on this branch as cca9717240, before any source change)

In-repo stored filters. An AST scan (TypeScript compiler API) covered every git-tracked .ts / .js / .json file, plus the fenced ts/js/json blocks of md/mdx files. It used four detectors:

• D1: $eq with an array literal.
• D2: a field entry with an array literal under a filter-bearing key (filter, filters, where, runtimeFilter, relatedListFilter, having and five more). The object is walked as a FilterCondition, descending $and / $or / $not and nested relations.
• D3: a FilterArray triple on = / == / equals / eq carrying an array.
• D4: a filter rule { field, operator: EQUALITY_SPELLING, value: [...] }.

The positive controls ran on the same walkers:

• a synthetic fixture with one case per detector (4/4 hits, in both TS and JSON);
• a count of $in array literals (C1);
• a count of the filter-bearing objects walked (C2).

scope	files	C1	C2	hits
examples/**	228 (3 tsconfig JSONC unparsed)	1 (matches the text grep)	91	0
packages/**	6910	522	4636	5 in non-test files, all false positives: 2 realtime-subscription eventTypes lists, 3 MongoDB aggregation-expression $eq: [a, b] operands. 65 in tests, all deliberate refusal fixtures or pins.
md/mdx fenced blocks (content, skills, docs, examples, packages, .changeset)	1545 files, 3162 blocks	12	176	3, all false positives (a nin rule in a design note; a plugin-permission filter)

A text grep for a $eq list outside packages/ and examples/ found 7 hits, all in changesets that describe the refusal itself. The control ($in list) found 140. No in-repo artefact carries the shape, so nothing needed converting. Deployed sys_metadata rows: NOT MEASURED (there is no deployment data here).

The analytics faces at base, over a real sql.js engine. The rows are d1 won, d2 lost, d3 open, d4 NULL, d5 the text 'won,lost':

where	native execute / echo	ObjectQL engine path	draft preview
{ stage: ['won', 'lost'] }	stage IN (?, ?), rows d1, d2	the engine received { stage: { $in: [...] } }, so the engine's own shared-face check never saw the list	string-compared the row against 'won,lost': d5
{ stage: { $eq: ['won', 'lost'] } }	stage = ? bound to 'won': d1, and 'lost' was dropped without a word	{ stage: 'won' }	d5
{ stage: { $eq: [] } }	no WHERE at all: every row	{}	no row
{ stage: [] }	the FALSE constant
['stage', '=' / 'equals' / '==' / 'eq', [...]]	refused INVALID_FILTER / 400	refused	no row (the preview does not lower an array)
control { stage: { $in: ['won', 'lost'] } }	d1, d2	d1, d2	d1, d2

After this change, every object-form cell above is refused with INVALID_FILTER / 400, carrying the face's message (re-measured at 896e1e70f3).

Compile surfaces (a list in the equality slot)

surface	verdict
lowerAnalyticsWhere / normalizeAnalyticsFilterTree (analytics caller where, dataset scope filter, measure filter; NativeSQL execute, /analytics/sql echo, ObjectQL engine path)	changed. Both spellings are refused with INVALID_FILTER / 400 at any depth, measured over sql.js before and after.
evaluateAnalyticsQueryOverRows / matchesWhere (analytics draft-data preview)	changed, as a bounded in-place fix (see Deviations). It used to string-compare the row against the list and now runs the same gate.
assertListComparandShapes (spec shared face)	already compliant. This is ruling 乙's own face (#19882). This PR calls it and does not change it.
compileScopedFilterToSql (service-analytics read scope)	already compliant. Since #19975 (landed e8f163fc3a) it refuses both spellings with READ_SCOPE_COMPILE_FAILED / 500. Its matrix cells and read-scope-eq-array-refusal.test.ts are green in this PR's package run. Not touched.
matchesFilterCondition (formula)	already compliant, per the table in PR #19994 (400 for the implicit list, $eq and $not). Not re-measured here.
applyFilterCondition (driver-sql)	already compliant, per the table in the #19882 changeset. Not re-measured here.
buildWhereSQL (driver-turso RemoteTransport)	already compliant, per the table in PR #19994. Not re-measured here.
checkCondition (driver-memory)	already compliant: 400 at every depth, per the table in the #19882 changeset. Not re-measured here.
translateFieldOperators (driver-mongodb)	out of scope. The driver answers with MongoDB array equality. A platform door reaches it only through the shared face, which refuses the list (the declared scope of #19882).
applyHaving / matchesHaving / checkCondition (objectql HAVING)	out of scope. This is a caller-authored filter over aggregated rows, a different door from this card's. Its answers are recorded in the #19886 stage-2a report.

Tests and evidence (head 896e1e70f3)

• New src/__tests__/where-equality-slot-list-refusal.test.ts, 59 tests. Every refusal asserts code + status.
  • The $eq list at 9 positions: every depth, the empty list, and beside another operator in either key order.
  • The implicit list at 7 positions, including the empty list and a nested relation.
  • Byte identity (4): the object spelling equals the FilterArray spelling, which equals the face's own message.
  • List before member (4).
  • 12 neighbouring shapes that must compile as before: a scalar, a Date, the null predicate, the $in remedy, the $in: [] / $nin: [] constants, a nested scalar, a $field reference, $between, and $ne (not judged).
  • Four faces over a real sql.js engine (native execute, echo, ObjectQL engine path, draft preview) × 4 spellings, plus a control. Each is refused before any statement runs and before any engine.aggregate call.
  • A stored dataset through the service doors (6): the dashboard door and the draft preview, for a scope filter and a measure filter; the registered cube on the ObjectQL door; and a control.
• comparand-door-single-source.test.ts: the array row's whereEq cell is re-judged from accept to INVALID_FILTER/400, with a note. It had pinned qty = 'al' with 'be' dropped.
• Two existing pins re-judged, not rewritten by rote:
  • filter-normalizer-not-null-safe.test.ts: the bare [] is now refused, and $in: [] keeps its FALSE constant.
  • filter-normalizer-undefined-comparand.test.ts: the { d: [1, undefined] } row leaves the undefined table, and the { d: [1, null] } row leaves the null control group. Each carries a note: its enclosing shape is now refused whole.
• Package run. pnpm --filter @objectstack/service-analytics test: 117 files and 2541 tests passed (base: 116 files, 2484 tests). typecheck exited 0, and tsc --listFiles includes all four touched test files.
• Ablations. Each was run from the committed fix through scripts/ablation-replace.mjs, with the red/green count predicted before running. The tests import the source by relative path, so no build is on the path. Each restore was proven by the blob hash matching HEAD and by an empty git diff HEAD.
  • A: the in reading put back. The gate call in lowerAnalyticsWhere was deleted and the old bare-array arm restored in fieldLeaves. Predicted 41 red; measured 41 red / 161 green over the four touched test files:
    • 9 $eq, 7 implicit, 4 byte identity, 4 list-first;
    • 12 faces: native, echo and engine × 4 spellings. The preview cells stayed green, because the preview runs the gate itself;
    • 3 stored, 1 matrix array where row, 1 bare [].
    • Sample failures: native execute: expected a refusal, got rows: expected [ 'd1', 'd2' ] to be undefined and expected 'accept' to be 'INVALID_FILTER/400'.
  • B: the preview's gate call deleted. Predicted 6; measured 6 red / 53 green: the four preview face cells and the two stored-dataset preview-door cells. Sample: draft preview: expected a refusal, got rows: expected [ 'd5' ] to be undefined.
• Gates. dispatch-gates derived 60 at 896e1e70f3. The run also covered the dispatch-time list's check:dispatcher-error-vocabulary, for 61 in total. 59 exited 0.
  • NOT MEASURED (2): check:dual-build-cjs-loads and check:type-check-debt exited 3 (PREREQUISITE NOT MET). They need the whole workspace built, which CI does.
  • check:lean-entry-closure exited 3 until objectql's closure was built, then 0.
  • --ran reconciliation: 60 derived, 58 run, 2 NOT-MEASURED (derived from the recorded exit 3), 0 unrun.
  • Among the passes: check-adr-0087-registration --base origin/main (1 declared-breaking changeset, not-required (already-registered)), check-changeset-no-major, check:changeset-gate-self-tests, check:nul-bytes, and check-issue-citations in its board-probing mode (19 citations, all of which resolve).
• Lint, narrowed to the change. eslint --no-inline-config --format json over the six touched TypeScript files: 6 files, 0 errors, 0 warnings. eslint --print-config resolves a config for each of them. eslint.config.mjs never enables type-aware linting (its note near line 327), so this diff cannot change the verdict on any untouched file.
• Dependents. pnpm --filter '...@objectstack/service-analytics' names 19 downstream packages. The AST scan above found no filter carrying the shape in their sources or tests. Their suites were not run here; CI's affected set runs them.

Deviations from the dispatch, stated

1. File surface. The claim declared filter-normalizer.ts, the matrix's where* cells, new test files and the changeset. This PR also touches three more files:
   • src/preview-evaluator.ts: one import, one call and comments. This is a bounded in-place fix, and all four conditions hold: it is the same defect class; it is a mechanical call of the same gate, whose shape ruling 乙 pins; no open PR touches this package, per the claim's own reading; and it is the same gate family, with no new verification surface. The claim's file surface needs this path added.
   • filter-normalizer-not-null-safe.test.ts and filter-normalizer-undefined-comparand.test.ts: three pins asserted the reading this ruling removes. They are re-judged, with notes (above).
2. The changeset's ADR-0087 disposition is not-required (already-registered filter-equality-array-comparand-refused), not the dispatched not-required (no-migration-prescription). A stored dataset, widget or measure filter can carry this shape, because the authoring schema admits it (measured). So the changeset carries a FROM → TO table. The gate refuses no-migration-prescription on a body that carries one, and that disposition would also claim that no author has to rewrite anything. The transition itself has been on the ledger since [finding] the comparand-SHAPE face declares it closes the door "for every driver at once", but an array in the IMPLICIT-EQUALITY slot passes it — and driver-mongodb alone answers it, as an exact-array match #19757, and that entry's surface and prescription cover this door verbatim. PR fix(spec): refuse a blank $between endpoint at the runtime filter door #19374 used the same disposition for the same situation. The gate accepts it.

Acceptance notes

• A registry sentence this PR makes false. The registered entry filter-equality-array-comparand-refused (packages/spec/src/migrations/entries/semantic/18.filter-equality-array-comparand-refused.ts, and its copy in registry.ts) says that the analytics normalizer's OBJECT form "still reads as membership". That is no longer true. Editing it is a packages/spec change, outside this dispatch, so the carrier is the seat's call.
• $ne with a list is not judged (ruling A of [finding] $ne with an array comparand splits across backends: driver-sql and driver-memory refuse (400), driver-mongodb answers, formula matches every row — and both shared faces pass it #19886). Measured at 896e1e70f3: { stage: { $ne: ['won', 'lost'] } } compiles to stage IS NULL OR stage != 'won', so 'lost' rows are served. Reported to the seat.
• The shared face's other arms. The object-form door still runs none of them: the null list member, the null ordering comparand, the null or blank $between bound, and the scalar $in. The FilterArray spelling of each is refused on the same door. This package's own pins hold several of the object-form answers. Reported, not addressed.
• The authoring door. It still admits the list: DatasetSchema with filter: { stage: ['won', 'lost'] } parses. The [finding] the comparand-SHAPE face declares it closes the door "for every driver at once", but an array in the IMPLICIT-EQUALITY slot passes it — and driver-mongodb alone answers it, as an exact-array match #19757 changeset declared this. Reported.
• The draft preview does not lower a FilterArray where. It answered no row for ['stage', '=', 'won'] in the probe. Reachability through the dataset door is not established, so this is an observation only.

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

7 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see

• the SDK route bridge reached 60 of 215 client-bound route-ledger rows — the other 155 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 155: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 100 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
• a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 1f05ea4fb296357dedf86ed34a234a1e779b1381 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from cdbf68847a4b750c3e5aa024b6e243b1750366dc — the merge of head 896e1e70f3fd96102dac6a5e5a0f86dbf34217ff into base 1f05ea4fb296357dedf86ed34a234a1e779b1381, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin cdbf68847a4b750c3e5aa024b6e243b1750366dc && git checkout cdbf68847a4b750c3e5aa024b6e243b1750366dc
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 1f05ea4fb296357dedf86ed34a234a1e779b1381 896e1e70f3fd96102dac6a5e5a0f86dbf34217ff && git checkout -B drift-repro 1f05ea4fb296357dedf86ed34a234a1e779b1381 && git merge --no-ff 896e1e70f3fd96102dac6a5e5a0f86dbf34217ff

node scripts/docs-audit/affected-docs.mjs --json 1f05ea4fb296357dedf86ed34a234a1e779b1381

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.