Repository navigation
Commit a6a4361
fix(service-analytics): the draft preview refuses a filter operator it cannot evaluate instead of answering every row (#19833)
Fixes #19810
Clause-②: no
The analytics draft-data preview
(`packages/services/service-analytics/src/preview-evaluator.ts`, the
ADR-0037 P3 Live Canvas path) answered **true for every row** for any
`where` operator its switch had no case for. A drafted chart therefore
silently IGNORED those filters and CHANGED at publish, where the real
filter doors apply them.
## The enumeration, read at source before any edit
At `origin/main` `c1dfa5241b`, `matchOp`'s switch carried exactly TEN
cases and one default arm:
| | |
|---|---|
| Evaluated (10) | `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`,
`$between`, `$in`, `$nin`, `$contains` |
| Default arm (`:109`) | `default: return true; // unknown operator —
permissive (preview, reads only)` |
| Answered for EVERY row | `$notContains`, `$startsWith`, `$endsWith`,
`$icontains`, `$null`, `$exists` (the rest of `FILTER_OPERATORS`), the
staged `$like` / `$ilike`, and any typo |
The card's premise holds exactly as filed. The combinators `$and` /
`$or` / `$not` were and remain handled by `matchesWhere` itself.
## The repair, and which contract it matches
**Fail-closed by REFUSING** — `INVALID_FILTER` / `400`, through
`filter-normalizer.ts`'s already-exported `invalidFilterError`. **No new
error code and no new exported symbol** (the module's five exports are
byte-identical before and after).
Three candidate behaviours, and why refusal:
- **answer true** — the defect;
- **exclude the row** — makes the preview merely DIFFERENT from publish
(zero rows where publish draws numbers). That is precisely the silent
shape `lowerPreviewDateRange` abolished on this same evaluator (#16322),
so it trades one invisible divergence for another;
- **refuse** — the only one that makes the disagreement VISIBLE to the
author who can fix it.
The yardstick is what the real filter doors do, read rather than
invented:
- `driver-memory`'s `uncompilableFieldOperatorError` (#5345): "It is
refused rather than dropped: a predicate that compiles to nothing does
not narrow the query, it WIDENS it — the aggregate is then computed over
rows the filter excluded, and a chart drawn over them looks like a
working chart (#3948, #4286/ADR-0078)."
- `service-analytics` **already** refuses `$like` / `$ilike` this way.
From the `FILTER_OPERATORS` docblock's own face table:
"`driver-mongodb`, `objectql` `having`, `service-analytics` — REFUSE,
loudly, in the ADR-0112 `INVALID_FILTER` envelope". This change puts the
preview face on the posture its own package already holds.
- The triage note asked the preview to keep the refusal PR #19750 /
#19514 gave the two filter doors for an empty or non-string `$icontains`
comparand. It is kept the strong way round: the preview does not
evaluate `$icontains` at all, so there is no comparand for it to
disagree about — every spelling of it refuses.
Two structural points, both copied from how this defect class was closed
elsewhere:
1. **The vocabulary and the evaluator are ONE table.** The switch
becomes a `Map` whose keys ARE what this face accepts — the shape
`memory-analytics`' `MONGO_TO_CUBE_OPERATOR` took for the identical
defect: "adding a row here is the only way to widen what this face
accepts, and forgetting to add one is a loud refusal rather than a wrong
number." A `Map` and not an object literal, so a constraint key naming
an `Object.prototype` member cannot resolve to an inherited function and
be called as a predicate.
2. **The gate is row-independent.** A per-row refusal only fires if some
row reaches it, so a pending seed draft holding ZERO rows — the state a
draft is authored in — would have answered an empty chart for a filter
it cannot evaluate. The `where` tree is walked once before any row is
read, the way `driver-memory`'s `assertFilterConditionShape` runs ahead
of that driver's lowering.
**⛔ No case was added, deliberately.** The default arm is the defect;
adding `$icontains`, `$startsWith` and `$endsWith` would have left the
next unhandled operator in exactly the same state, which is why the
table and not a case list is the repair. Growing the arms is separate
work with its own ordering already ruled: the `FILTER_OPERATORS`
docblock's #6520 constraint — a name must not land ahead of its
evaluators — reads the same in this direction, so an arm joins the table
in the PR that measures it against the shared text/temporal conformance
kits. This face is not enrolled in `FILTER_TEXT_CASES` today, and its
one shipped text arm (`$contains`) is itself off that contract (see
Acceptance notes).
## Evidence — both directions
Command, identical in both states:
```
pnpm --filter @objectstack/service-analytics exec vitest run --maxWorkers=2 \
src/__tests__/preview-unevaluable-operator.test.ts
```
**BEFORE** — `preview-evaluator.ts` restored to `c1dfa5241b` on disk
(blob `227496af` confirmed on disk against `git rev-parse BASE:path`;
marker counts `default: return true` = 1, `PREVIEW_FIELD_OPERATORS` =
0), the test file and everything else at HEAD:
```
Test Files 1 failed (1)
Tests 14 failed | 17 passed (31)
FAIL ... > does NOT answer the row that `name $icontains "acme"` excludes
AssertionError: expected [ 'Acme Corp', 'Globex' ] to not include 'Globex'
FAIL ... > refuses it in the ADR-0112 `INVALID_FILTER` / 400 envelope
AssertionError: expected undefined to be an instance of Error
FAIL ... > refuses over an EMPTY seed draft too — the walk is not a function of the data
FAIL ... > refuses inside `$or`, `$and` and `$not` arms
FAIL ... > refuses a constraint key that names an Object.prototype member
FAIL ... > $notContains / $startsWith / $endsWith / $icontains / $null / $exists / $like / $ilike
is refused, never answered for every row (8 rows)
FAIL ... > refuses the drafted selection instead of charting every seed row
AssertionError: promise resolved "{ rows: [ { …(2) }, { …(2) } ], …(1) }" instead of rejecting
```
`expected [ 'Acme Corp', 'Globex' ] to not include 'Globex'` is the
card's claim measured: `Globex` does not match `name $icontains 'acme'`,
and the preview charted it anyway.
The restore leg was verified by hash, not by an exit code: on-disk blob
back to `0e1bf302` = `HEAD:path`, `git diff HEAD` empty.
**AFTER** — same command, tree at HEAD:
```
Test Files 1 passed (1)
Tests 31 passed (31)
```
**The unchanged direction.** The 17 tests that pass in BOTH states are
the regression guard, and they are meant to: a fail-closed default that
starts rejecting rows which used to match correctly is the mirror-image
defect. They assert both directions (a matching row still matches, a
non-matching row still does not) for every one of the ten evaluated
arms, plus the `$lte` bare-day rule (#3777), implicit equality, `$and`,
`$or`, `$not`, and an absent `where`. Every predicate body is
byte-for-byte the `case` it replaces.
## Checks run locally at `2deda8dab5`
- `pnpm --filter @objectstack/service-analytics test` — **114 files /
2442 tests passed**
- `pnpm --filter @objectstack/service-analytics run typecheck` — exit 0
- `pnpm --filter '@objectstack/service-analytics^...' build` — exit 0
(dependency closure)
- `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack
--ran ...` — **60 derived families accounted for: 57 run green, 3 NOT
MEASURED** (`check:dual-build-cjs-loads`, `check:lean-entry-closure`,
`check:type-check-debt` each exit 3, PREREQUISITE NOT MET — they read a
whole-workspace `pnpm build`, which is CI's Build Core job). Among the
57: `check:where-matcher` (417 matchers, 0 silently-wrong),
`check:nul-bytes`, `check:issue-citations`, `check:doc-authoring`,
`check:empty-changeset`, `check:test-source-alias`,
`check:undeclared-dep-imports`.
- The four roster gates whose ledger sits under a directory this diff
touches, read and run rather than assumed silent:
`check-changeset-fixed`, `check:authz-resolver`,
`check:error-code-casing`, `check:filter-alias-parity` — all exit 0.
- `eslint . --no-inline-config` — the whole repo, not a narrowing:
**7022 files, 0 errors, 0 warnings**, run at this PR's final commit.
- Control-character self-scan over all three changed files: no hits.
A changeset is included: `@objectstack/service-analytics` is published
and this changes its runtime behaviour.
## Acceptance notes — found in passing, NOT fixed here
1. **`$contains` in this same file folds case, and the contract says it
must not.** `matchOp`'s `$contains` arm is `String(value ??
'').toLowerCase().includes(String(expected ?? '').toLowerCase())`.
`filter-text-conformance.ts` records that `$contains` / `$notContains` /
`$startsWith` / `$endsWith` "compare CASE-SENSITIVELY" and that
`driver-memory` moved its two folding faces onto the case-exact answer
in #6682. The same line also coerces a non-string stored value, which
the #14079 ruling type-gates. So the preview answers `$contains`
differently from every published face — the same preview-vs-publish
divergence this card is about, one arm over. ⛔ Deliberately untouched:
this PR's second evidence direction is that the ten evaluated arms do
not change behaviour.
2. **An undeclared `$`-key in a NODE position is silently read as a
field name.** `matchesWhere` handles `$and` / `$or` / `$not` and falls
through everything else to implicit equality, so `{ $nor: [...] }`
compares `row['$nor']` and excludes every row without a word. The
published path refuses it (`unknownLogicalOperatorError`). Fails closed
rather than open, so it is not this card's harm — but it is silent.
3. **An empty field constraint `{ name: {} }` matches every row.** No
operator keys, so the inner loop never runs. `driver-memory` refuses
this shape (`emptyFieldConstraintError`, #5240): "`{ status: {} }` did
not mean 'no rows', it meant 'rows whose status is anything'". This one
IS answer-true-shaped, on the same evaluator, and is outside the
operator vocabulary this card closes.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01AhQASwqJr2Z7XfGWUdvnbF)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 4112752 commit a6a4361
3 files changed
Lines changed: 384 additions & 30 deletions
File tree
- .changeset
- packages/services/service-analytics/src
- __tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
Lines changed: 216 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
0 commit comments