Repository navigation
Commit 14add48
Fixes #19888
Clause-②: no (narrowing)
## What this changes
The analytics `where` door (`lowerAnalyticsWhere` in
`packages/services/service-analytics/src/strategies/filter-normalizer.ts`)
now refuses a list in the equality slot of an object-form filter, `{ f:
[...] }` and `{ f: { $eq: [...] } }`, with `INVALID_FILTER` / 400. This
applies ruling 乙 of #19757 (record `5793368540`: 「an array in the
implicit-equality slot is refused at the shared face, for every driver
at once」) to the one analytics spelling that never reached the shared
face.
The new gate, `assertNoListInEqualitySlot`, walks the object-form
condition the way the shared face does: `$and` / `$or`, `$not` and field
entries, from the same `where` path seed. It hands each equality-slot
list to `assertListComparandShapes` (`@objectstack/spec/data`) as a
one-entry node. That node holds nothing but the list, so only the face's
equality arm can fire, and the refusal is the face's own: its envelope,
wording, path and `$in` prescription. The `FilterArray` spelling of the
same condition was already refused inside `parseFilterAST`. Both
spellings now produce the same bytes, and a test pins this.
Every filter source of this door passes through the gate: the caller
`where`, a dataset's scope `filter` and a measure's `filter`. That holds
on the NativeSQL execute path, the `/analytics/sql` echo and the
ObjectQL engine path. The draft-data preview (`preview-evaluator.ts`)
calls the same exported gate, so a drafted chart refuses what the
published chart refuses. The `in` reading is deleted from `fieldLeaves`.
So are the two arms that existed only because of it: the bare-array
member sweep in `assertDefinedComparands` and the bare-array guard in
`nullGuardForFieldSpec`.
Two details:
- **Nested relations.** A nested-relation object (`{ acct: { region:
[...] } }`) is descended too. The shared face leaves such an object
alone, because a driver reads it as a deep-equality comparand or another
object's condition. This compiler flattens it to the dotted member
`acct.region`, and the list is in that member's equality slot.
- **Precedence.** The list is diagnosed before any member gate. `{ d:
[1, undefined] }` gets the list refusal, not the undefined-comparand
one. The shared face and `read-scope-sql.ts` use the same order.
`$ne` with a list is not judged, because ruling 乙 names equality only.
The list operators keep their lists, including `$in: []` and `$nin: []`.
Every scalar compiles as before, including `null`.
## Measured first (recorded on this branch as `cca9717240`, before any
source change)
**In-repo stored filters.** An AST scan (TypeScript compiler API)
covered every git-tracked `.ts` / `.js` / `.json` file, plus the fenced
ts/js/json blocks of md/mdx files. It used four detectors:
- D1: `$eq` with an array literal.
- D2: a field entry with an array literal under a filter-bearing key
(`filter`, `filters`, `where`, `runtimeFilter`, `relatedListFilter`,
`having` and five more). The object is walked as a FilterCondition,
descending `$and` / `$or` / `$not` and nested relations.
- D3: a FilterArray triple on `=` / `==` / `equals` / `eq` carrying an
array.
- D4: a filter rule `{ field, operator: EQUALITY_SPELLING, value: [...]
}`.
The positive controls ran on the same walkers:
- a synthetic fixture with one case per detector (4/4 hits, in both TS
and JSON);
- a count of `$in` array literals (C1);
- a count of the filter-bearing objects walked (C2).
| scope | files | C1 | C2 | hits |
|:--|--:|--:|--:|:--|
| `examples/**` | 228 (3 tsconfig JSONC unparsed) | 1 (matches the text
grep) | 91 | **0** |
| `packages/**` | 6910 | 522 | 4636 | 5 in non-test files, **all false
positives**: 2 realtime-subscription `eventTypes` lists, 3 MongoDB
aggregation-expression `$eq: [a, b]` operands. 65 in tests, all
deliberate refusal fixtures or pins. |
| md/mdx fenced blocks (content, skills, docs, examples, packages,
.changeset) | 1545 files, 3162 blocks | 12 | 176 | 3, **all false
positives** (a `nin` rule in a design note; a plugin-permission
`filter`) |
A text grep for a `$eq` list outside `packages/` and `examples/` found 7
hits, all in changesets that describe the refusal itself. The control
(`$in` list) found 140. **No in-repo artefact carries the shape, so
nothing needed converting.** Deployed `sys_metadata` rows: NOT MEASURED
(there is no deployment data here).
**The analytics faces at base**, over a real sql.js engine. The rows are
d1 `won`, d2 `lost`, d3 `open`, d4 NULL, d5 the text `'won,lost'`:
| `where` | native execute / echo | ObjectQL engine path | draft preview
|
|:--|:--|:--|:--|
| `{ stage: ['won', 'lost'] }` | `stage IN (?, ?)`, rows d1, d2 | the
engine received `{ stage: { $in: [...] } }`, so the engine's own
shared-face check never saw the list | string-compared the row against
`'won,lost'`: d5 |
| `{ stage: { $eq: ['won', 'lost'] } }` | `stage = ?` bound to `'won'`:
d1, and `'lost'` was dropped without a word | `{ stage: 'won' }` | d5 |
| `{ stage: { $eq: [] } }` | **no WHERE at all: every row** | `{}` | no
row |
| `{ stage: [] }` | the FALSE constant | | |
| `['stage', '=' / 'equals' / '==' / 'eq', [...]]` | refused
`INVALID_FILTER` / 400 | refused | no row (the preview does not lower an
array) |
| control `{ stage: { $in: ['won', 'lost'] } }` | d1, d2 | d1, d2 | d1,
d2 |
After this change, every object-form cell above is refused with
`INVALID_FILTER` / 400, carrying the face's message (re-measured at
`896e1e70f3`).
## Compile surfaces (a list in the equality slot)
| surface | verdict |
|:--|:--|
| `lowerAnalyticsWhere` / `normalizeAnalyticsFilterTree` (analytics
caller `where`, dataset scope `filter`, measure `filter`; NativeSQL
execute, `/analytics/sql` echo, ObjectQL engine path) | **changed.**
Both spellings are refused with `INVALID_FILTER` / 400 at any depth,
measured over sql.js before and after. |
| `evaluateAnalyticsQueryOverRows` / `matchesWhere` (analytics
draft-data preview) | **changed**, as a bounded in-place fix (see
Deviations). It used to string-compare the row against the list and now
runs the same gate. |
| `assertListComparandShapes` (spec shared face) | **already
compliant.** This is ruling 乙's own face (#19882). This PR calls it and
does not change it. |
| `compileScopedFilterToSql` (service-analytics read scope) | **already
compliant.** Since #19975 (landed `e8f163fc3a`) it refuses both
spellings with `READ_SCOPE_COMPILE_FAILED` / 500. Its matrix cells and
`read-scope-eq-array-refusal.test.ts` are green in this PR's package
run. Not touched. |
| `matchesFilterCondition` (formula) | **already compliant**, per the
table in PR #19994 (400 for the implicit list, `$eq` and `$not`). Not
re-measured here. |
| `applyFilterCondition` (driver-sql) | **already compliant**, per the
table in the #19882 changeset. Not re-measured here. |
| `buildWhereSQL` (driver-turso RemoteTransport) | **already
compliant**, per the table in PR #19994. Not re-measured here. |
| `checkCondition` (driver-memory) | **already compliant**: 400 at every
depth, per the table in the #19882 changeset. Not re-measured here. |
| `translateFieldOperators` (driver-mongodb) | **out of scope.** The
driver answers with MongoDB array equality. A platform door reaches it
only through the shared face, which refuses the list (the declared scope
of #19882). |
| `applyHaving` / `matchesHaving` / `checkCondition` (objectql HAVING) |
**out of scope.** This is a caller-authored filter over aggregated rows,
a different door from this card's. Its answers are recorded in the
#19886 stage-2a report. |
## Tests and evidence (head `896e1e70f3`)
- **New `src/__tests__/where-equality-slot-list-refusal.test.ts`, 59
tests.** Every refusal asserts `code` + `status`.
- The `$eq` list at 9 positions: every depth, the empty list, and beside
another operator in either key order.
- The implicit list at 7 positions, including the empty list and a
nested relation.
- Byte identity (4): the object spelling equals the FilterArray
spelling, which equals the face's own message.
- List before member (4).
- 12 neighbouring shapes that must compile as before: a scalar, a
`Date`, the null predicate, the `$in` remedy, the `$in: []` / `$nin: []`
constants, a nested scalar, a `$field` reference, `$between`, and `$ne`
(not judged).
- Four faces over a real sql.js engine (native execute, echo, ObjectQL
engine path, draft preview) × 4 spellings, plus a control. Each is
refused before any statement runs and before any `engine.aggregate`
call.
- A stored dataset through the service doors (6): the dashboard door and
the draft preview, for a scope filter and a measure filter; the
registered cube on the ObjectQL door; and a control.
- **`comparand-door-single-source.test.ts`:** the `array` row's
`whereEq` cell is re-judged from `accept` to `INVALID_FILTER/400`, with
a note. It had pinned `qty = 'al'` with `'be'` dropped.
- **Two existing pins re-judged, not rewritten by rote:**
- `filter-normalizer-not-null-safe.test.ts`: the bare `[]` is now
refused, and `$in: []` keeps its FALSE constant.
- `filter-normalizer-undefined-comparand.test.ts`: the `{ d: [1,
undefined] }` row leaves the undefined table, and the `{ d: [1, null] }`
row leaves the null control group. Each carries a note: its enclosing
shape is now refused whole.
- **Package run.** `pnpm --filter @objectstack/service-analytics test`:
117 files and 2541 tests passed (base: 116 files, 2484 tests).
`typecheck` exited 0, and `tsc --listFiles` includes all four touched
test files.
- **Ablations.** Each was run from the committed fix through
`scripts/ablation-replace.mjs`, with the red/green count predicted
before running. The tests import the source by relative path, so no
build is on the path. Each restore was proven by the blob hash matching
HEAD and by an empty `git diff HEAD`.
- **A: the `in` reading put back.** The gate call in
`lowerAnalyticsWhere` was deleted and the old bare-array arm restored in
`fieldLeaves`. Predicted 41 red; measured **41 red / 161 green** over
the four touched test files:
- 9 `$eq`, 7 implicit, 4 byte identity, 4 list-first;
- 12 faces: native, echo and engine × 4 spellings. The preview cells
stayed green, because the preview runs the gate itself;
- 3 stored, 1 matrix `array` where row, 1 bare `[]`.
- Sample failures: `native execute: expected a refusal, got rows:
expected [ 'd1', 'd2' ] to be undefined` and `expected 'accept' to be
'INVALID_FILTER/400'`.
- **B: the preview's gate call deleted.** Predicted 6; measured **6 red
/ 53 green**: the four preview face cells and the two stored-dataset
preview-door cells. Sample: `draft preview: expected a refusal, got
rows: expected [ 'd5' ] to be undefined`.
- **Gates.** `dispatch-gates` derived 60 at `896e1e70f3`. The run also
covered the dispatch-time list's `check:dispatcher-error-vocabulary`,
for 61 in total. 59 exited 0.
- **NOT MEASURED (2):** `check:dual-build-cjs-loads` and
`check:type-check-debt` exited 3 (PREREQUISITE NOT MET). They need the
whole workspace built, which CI does.
- `check:lean-entry-closure` exited 3 until objectql's closure was
built, then 0.
- `--ran` reconciliation: 60 derived, 58 run, 2 NOT-MEASURED (derived
from the recorded exit 3), 0 unrun.
- Among the passes: `check-adr-0087-registration --base origin/main` (1
declared-breaking changeset, `not-required (already-registered)`),
`check-changeset-no-major`, `check:changeset-gate-self-tests`,
`check:nul-bytes`, and `check-issue-citations` in its board-probing mode
(19 citations, all of which resolve).
- **Lint, narrowed to the change.** `eslint --no-inline-config --format
json` over the six touched TypeScript files: 6 files, 0 errors, 0
warnings. `eslint --print-config` resolves a config for each of them.
`eslint.config.mjs` never enables type-aware linting (its note near line
327), so this diff cannot change the verdict on any untouched file.
- **Dependents.** `pnpm --filter '...@objectstack/service-analytics'`
names 19 downstream packages. The AST scan above found no filter
carrying the shape in their sources or tests. Their suites were not run
here; CI's affected set runs them.
## Deviations from the dispatch, stated
1. **File surface.** The claim declared `filter-normalizer.ts`, the
matrix's `where*` cells, new test files and the changeset. This PR also
touches three more files:
- `src/preview-evaluator.ts`: one import, one call and comments. This is
a bounded in-place fix, and all four conditions hold: it is the same
defect class; it is a mechanical call of the same gate, whose shape
ruling 乙 pins; no open PR touches this package, per the claim's own
reading; and it is the same gate family, with no new verification
surface. The claim's file surface needs this path added.
- `filter-normalizer-not-null-safe.test.ts` and
`filter-normalizer-undefined-comparand.test.ts`: three pins asserted the
reading this ruling removes. They are re-judged, with notes (above).
2. **The changeset's ADR-0087 disposition** is `not-required
(already-registered filter-equality-array-comparand-refused)`, not the
dispatched `not-required (no-migration-prescription)`. A stored dataset,
widget or measure filter can carry this shape, because the authoring
schema admits it (measured). So the changeset carries a FROM → TO table.
The gate refuses `no-migration-prescription` on a body that carries one,
and that disposition would also claim that no author has to rewrite
anything. The transition itself has been on the ledger since #19757, and
that entry's surface and prescription cover this door verbatim. PR
#19374 used the same disposition for the same situation. The gate
accepts it.
## Acceptance notes
- **A registry sentence this PR makes false.** The registered entry
`filter-equality-array-comparand-refused`
(`packages/spec/src/migrations/entries/semantic/18.filter-equality-array-comparand-refused.ts`,
and its copy in `registry.ts`) says that the analytics normalizer's
OBJECT form "still reads as membership". That is no longer true. Editing
it is a `packages/spec` change, outside this dispatch, so the carrier is
the seat's call.
- **`$ne` with a list** is not judged (ruling A of #19886). Measured at
`896e1e70f3`: `{ stage: { $ne: ['won', 'lost'] } }` compiles to `stage
IS NULL OR stage != 'won'`, so `'lost'` rows are served. Reported to the
seat.
- **The shared face's other arms.** The object-form door still runs none
of them: the null list member, the null ordering comparand, the null or
blank `$between` bound, and the scalar `$in`. The FilterArray spelling
of each is refused on the same door. This package's own pins hold
several of the object-form answers. Reported, not addressed.
- **The authoring door.** It still admits the list: `DatasetSchema` with
`filter: { stage: ['won', 'lost'] }` parses. The #19757 changeset
declared this. Reported.
- **The draft preview does not lower a FilterArray `where`.** It
answered no row for `['stage', '=', 'won']` in the probe. Reachability
through the dataset door is not established, so this is an observation
only.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Evb5jFDZGKQE9KG4jbMfMF)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent b81da66 commit 14add48
7 files changed
Lines changed: 607 additions & 45 deletions
File tree
- .changeset
- packages/services/service-analytics/src
- __tests__
- strategies
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
Lines changed: 6 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
153 | 153 | | |
154 | 154 | | |
155 | 155 | | |
156 | | - | |
157 | | - | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
158 | 161 | | |
159 | 162 | | |
160 | | - | |
| 163 | + | |
161 | 164 | | |
162 | 165 | | |
163 | 166 | | |
| |||
Lines changed: 9 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
545 | 545 | | |
546 | 546 | | |
547 | 547 | | |
548 | | - | |
| 548 | + | |
549 | 549 | | |
550 | 550 | | |
551 | 551 | | |
| |||
556 | 556 | | |
557 | 557 | | |
558 | 558 | | |
559 | | - | |
560 | | - | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
561 | 567 | | |
562 | 568 | | |
563 | 569 | | |
| |||
Lines changed: 17 additions & 15 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
167 | 167 | | |
168 | 168 | | |
169 | 169 | | |
170 | | - | |
171 | | - | |
172 | | - | |
173 | | - | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
174 | 181 | | |
175 | 182 | | |
176 | | - | |
177 | | - | |
178 | | - | |
179 | | - | |
180 | | - | |
181 | | - | |
182 | 183 | | |
183 | 184 | | |
184 | 185 | | |
| |||
294 | 295 | | |
295 | 296 | | |
296 | 297 | | |
297 | | - | |
298 | | - | |
299 | | - | |
300 | | - | |
301 | | - | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
302 | 304 | | |
303 | 305 | | |
304 | 306 | | |
| |||
0 commit comments