|
| 1 | +--- |
| 2 | +"@objectstack/spec": minor |
| 3 | +--- |
| 4 | + |
| 5 | +fix(spec)!: a filter carrying a comparand the query faces refuse is refused when it is saved (#20116) |
| 6 | + |
| 7 | +**BREAKING** — an accept-set narrowing of published authoring schemas, shipped as `minor` under the repo's launch-window convention for accept-set narrowings. The save door narrows to exactly what the query faces already refuse. The hand-migration prescription is registered under protocol major 18 as `filter-query-face-comparands-refused-at-save`. |
| 8 | + |
| 9 | +## What changes |
| 10 | + |
| 11 | +`FilterConditionSchema` now refuses, at parse, every comparand slot the query faces refuse: |
| 12 | + |
| 13 | +- a `$null` or `$exists` flag that is not a boolean — `"false"`, `"x"`, `null`, `1`; |
| 14 | +- a `null` comparand of `$gt` / `$gte` / `$lt` / `$lte`; |
| 15 | +- an `$in` or `$nin` comparand that is not a list, or a list holding `null`; |
| 16 | +- a `$between` comparand that is not a two-element list, or whose endpoint is `null`, blank (`""`) or a `{ $field }` reference; |
| 17 | +- an array under `$ne`. |
| 18 | + |
| 19 | +It judges the field entries of a condition and of every `$and` / `$or` / `$not` member. The dataset `filter` and measure `filter` also judge the same slots INSIDE a nested-relation condition, through the nested-relation walk they already carry, because the analytics `where` door flattens a relation and judges its entries. The shared comparand-shape face (`assertListComparandShapes`) is called read-only as the judge for each slot, so the save door refuses exactly what that face refuses on every query. The two boolean flags, which that face does not judge, are refused on the predicate every flag face uses (`driver-sql`, `driver-memory`, `driver-mongodb`, the read-scope compiler and the analytics `where` door): the comparand is not a boolean. |
| 20 | + |
| 21 | +Measured on `origin/main` `af32cf9a` before the change: a dataset `filter`, a dataset measure `filter`, a dashboard widget `filter` and a report `runtimeFilter` each parsed with `success: true` for one instance of every shape above. The comparand-shape face refused each one but the flags with `INVALID_FILTER` / 400, and the analytics `where` door refused all of them, inside a nested relation too. So such a document published clean and then failed every chart built on it. |
| 22 | + |
| 23 | +Every schema that carries a `FilterCondition` refuses on parse. That covers the dataset `filter` and measure `filter`, the dashboard widget `filter` and options-source `filter`, the report and joined-report-block `runtimeFilter`, the field `relatedListFilter` and rollup `summaryOperations.filter`, the solution-blueprint summary `filter`, the analytics query `where`, the dataset selection `runtimeFilter`, the query `where` and `having`, the data-engine aggregate call's `having`, the aggregation `filter`, and the query-filter `where`. So `defineStack`, `os validate` and a save through the metadata protocol (`422 INVALID_METADATA`) refuse such a document at the slot's path, for example `filter.stage.$null` or `measures.0.filter.amount.$between.0`. |
| 24 | + |
| 25 | +The words are the query face's. For an array under `$ne`, a non-list `$in` / `$nin` and a malformed `$between`, the refusal is the face's sentence without its location clause (`at where.<field>.<op>`), because the issue's path carries the location. For a `null` ordering comparand, a `null` list member or endpoint, and a blank or `{ $field }` endpoint, it is the sentence the enforced operator slot (`FieldOperatorsSchema`) already prints for the same comparand. A non-boolean flag gets the query faces' sentence: `Operator "$null" on field "stage" requires a boolean comparand (true or false).`, then the received value and the prescription. |
| 26 | + |
| 27 | +Two request doors parse these carriers, and they now answer before the analytics compiler does. The REST dataset selection (its `runtimeFilter`, parsed against `DatasetSelectionSchema`) and the analytics query body (its `where`, parsed against `AnalyticsQueryRequestSchema`) answer `VALIDATION_FAILED` / 400 with the sentence at the field, for a top-level or combinator slot. Before, the compiler answered `INVALID_FILTER` / 400 for the same filter. |
| 28 | + |
| 29 | +This also changes the `$ne` note of the equality-slot change earlier in this release: an array under `$ne` is now refused on save too, in the sentence `FieldOperatorsSchema.$ne` and the face print. |
| 30 | + |
| 31 | +## What does NOT change |
| 32 | + |
| 33 | +- **Nothing stored is rewritten, and nothing is dropped.** The parse fails and strips nothing. The read path does not re-validate stored rows, so a stored document keeps loading, and its next save is refused. Such a filter has failed every query since the runtime refusal of its shape, so the refusal is a repair. |
| 34 | +- **The shared reach is the face's, and no wider.** On every carrier but the two dataset ones, a field spec with no `$` key, such as the nested-relation condition `{ account: { region: { $in: ["a", null] } } }`, is not judged, because neither the face nor the drivers' flag checks descend one. A dashboard widget `filter` and a report `runtimeFilter` therefore still save that shape, and the analytics `where` door refuses it when they are charted. |
| 35 | +- **What the face passes still passes:** `$eq: null` and `$ne: null` (the null predicate), a `{ $field }` reference as the whole comparand of a scalar comparison, `$in: []` and `$nin: []`, a whitespace-only `$between` endpoint, and falsy endpoints such as `[0, 0]`. |
| 36 | +- **The data-engine calls' `where` option still parses.** Its type is a union whose first arm is an open record. The face refuses the shape when the call runs. |
| 37 | +- **No key, export or JSON Schema changes.** The published JSON Schema cannot state a refinement, and `FilterCondition`'s already could not state the equality-slot one. |
| 38 | + |
| 39 | +## FROM → TO |
| 40 | + |
| 41 | +| you wrote | write instead | |
| 42 | +|:--|:--| |
| 43 | +| `{ stage: { $null: "true" } }`, `{ stage: { $null: 1 } }` | `{ stage: { $null: true } }` ("has no value") | |
| 44 | +| `{ stage: { $exists: "false" } }`, `{ stage: { $null: null } }` | the boolean you meant: `$exists: false` is "has no value", `$null: false` is "has a value" | |
| 45 | +| `{ amount: { $gt: null } }` | `{ amount: { $eq: null } }` ("has no value") or `{ amount: { $ne: null } }` ("has a value") | |
| 46 | +| `{ stage: { $in: "won" } }` | `{ stage: { $in: ["won"] } }` or `{ stage: "won" }` | |
| 47 | +| `{ stage: { $in: ["won", null] } }` | `{ $or: [{ stage: { $in: ["won"] } }, { stage: { $null: true } }] }` | |
| 48 | +| `{ stage: { $nin: ["lost", null] } }`, meaning "has a value and is not `lost`" | `{ stage: { $nin: ["lost"], $null: false } }` | |
| 49 | +| `{ stage: { $in: [null, ""] } }`, a filter builder's "is empty" | `{ $or: [{ stage: { $null: true } }, { stage: "" }] }` | |
| 50 | +| `{ stage: { $nin: [null, ""] } }`, a filter builder's "is not empty" | `{ stage: { $null: false, $ne: "" } }` | |
| 51 | +| `{ amount: { $between: [null, 5] } }`, `{ amount: { $between: ["", 5] } }` | `{ amount: { $lte: 5 } }`, or the bound you meant | |
| 52 | +| `{ amount: { $between: [{ $field: "floor" }, 5] } }` | `{ amount: { $gte: { $field: "floor" }, $lte: 5 } }` | |
| 53 | +| `{ amount: { $between: 5 } }`, `{ amount: { $between: [1] } }` | `{ amount: { $between: [1, 5] } }` | |
| 54 | +| `{ stage: { $ne: ["won", "lost"] } }` | `{ stage: { $nin: ["won", "lost"] } }` | |
| 55 | + |
| 56 | +### FROM → TO at the HTTP doors |
| 57 | + |
| 58 | +Same status, different code: the refusal now comes from the route's schema door, located on the member, instead of from the analytics filter normalizer. |
| 59 | + |
| 60 | +| request | before | after | |
| 61 | +|:--|:--|:--| |
| 62 | +| `POST /analytics/dataset/query` with `selection.runtimeFilter: { amount: { $between: [10] } }` | `400 INVALID_FILTER` from the analytics normalizer, in the comparand-shape face's sentence | `400 VALIDATION_FAILED`, `details.fields[]` entry `selection.runtimeFilter.amount.$between` with the sentence `Operator "$between" on field "amount" requires a [min, max] value array. Received array ([10]). …` | |
| 63 | +| the same route, any other slot above in `selection.runtimeFilter` (top level or in `$and` / `$or` / `$not`) | `400 INVALID_FILTER` | `400 VALIDATION_FAILED`, located on the slot, with that slot's sentence | |
| 64 | +| `POST /analytics/query` (`AnalyticsQueryRequestSchema`) with the same shape in `where` | `400 INVALID_FILTER` | refused by the request schema at `where.amount.$between`, answered `400 VALIDATION_FAILED` | |
| 65 | + |
| 66 | +A client that branches on `INVALID_FILTER` for these shapes reads `VALIDATION_FAILED` instead. Both are 400 and both name the field. |
| 67 | + |
| 68 | +## Who is affected, measured |
| 69 | + |
| 70 | +A literal-comparand scan of every member shape, with a lit control per shape, over `examples/**` and the non-test `packages/**` of this repository at `af32cf9a`, the console repository at its pinned commit `f8a9d0fb05`, and the cloud repository's `main` at `48d70663ab`, found authored filters carrying one in one place. The console's filter-condition widget writes "is empty" as `{ field: { $in: [null, ""] } }` and "is not empty" as `{ field: { $nin: [null, ""] } }`. That widget edits a field's `relatedListFilter` and a rollup's `summaryOperations.filter` in the Studio field designer, and a sharing rule's criteria. Both shapes carry a `null` list member, which the face has refused on every query since the 2026-08-31 ruling, so a filter saved that way has been failing its related list or rollup since then. After this change, the Studio save is refused instead, with the `$or` / `$null` prescription. Every other hit is prose, a type table or a test fixture. Deployed datasets, dashboards and reports were NOT measured. Validating each stack, or re-saving each document, finds every instance the surface above lists. |
| 71 | + |
| 72 | +Clause-②: no (narrowing) — nothing is widened. No key is added, removed or renamed, no exported symbol moves, and the operator vocabulary is unchanged. Comparand shapes that every query face already refused are now refused on save as well. |
| 73 | + |
| 74 | +<!-- adr-0087: registered filter-query-face-comparands-refused-at-save --> |
0 commit comments