Repository navigation
Commit dcd3309
fix(core,objectql)!: a relative-date placeholder resolved outside its field's years is refused INVALID_FILTER / 400, naming the placeholder and the year (#21065)
Fixes #20844
Clause-②: no (narrowing)
A relative-date placeholder is now judged by the year of the value it
resolved to. If that value falls outside its column's years (a `date`
0001..9999, a `datetime` 1000..9999, core's
`isOutsideTemporalYearRange`), the engine refuses it with
`INVALID_FILTER` / 400. The refusal names the placeholder as written and
the year it resolved to, in the temporal-comparand door's words. It runs
on `where` (`find`, `findOne`, `count`, `aggregate`, multi-row `update`
and `delete`), a per-aggregation `filter`, `having`, and `judgeFilter`,
before any driver read.
Dispatched by the PM loop (round 1, seat `domain:engine#2`), claim
comment 5923455068, session `session_01Ujdtvqs7ree7WyQmEDwEnG`.
## What changes
- **`packages/objectql/src/engine.ts`, the resolution stage.**
`resolveWhereFilterTokens` takes a required judge and hands it the
condition before and after resolution whenever something resolved.
Execution and the judge call the same stage function, so they cannot
drift: `resolveRelateThenLowerWhere` serves `where` on every verb,
`resolveThenLowerWhere` serves `having` and the per-aggregation
`filter`, and `judgeWhereAdmission` serves `judgeFilter`. The aggregate
region changes in two places: the per-aggregation resolution call passes
its judge, rooted at `aggregations[i].filter`, and one comment that this
change made false is corrected. Neither is the aggregate door.
- **`packages/objectql/src/temporal-comparand-door.ts`, beside the
door.** The door's one walk now carries a twin tree step for step, and
the door itself passes the tree as its own twin, so its verdicts are
unchanged (the door suites are green). The new
`assertResolvedTemporalTokensInRange` /
`assertHavingResolvedTemporalTokensInRange` walk the caller's condition
beside its resolution. They judge only a comparand written as a date
macro (`classifyFilterToken` kind `date-macro`), by the year class the
door asks of a literal of the same value:
- a `date` or a `datetime` asks core's `isOutsideTemporalYearRange`;
- a `time` column asks the door's `[#20480]` class, an instant whose UTC
year has no four-digit spelling. Year 0 (`0000-…`) still reads, as it
does for a literal.
This is ⛔ not a second pass of the door: every other comparand was
judged before resolution. There is ⛔ no second copy of the range, and ⛔
no new error code.
- **`packages/core/src/utils/filter-tokens.ts`, the producer.** A date
macro that lands on a day outside 0001..9999 now resolves to the
expanded-year form of ECMAScript's date time string format:
`+010026-10-01`, `-000001-10-01`, or `0000-10-01` for year 0. It used to
take the storage rule's unpadded spelling (`10026-10-01`, `-1-10-01`,
`0-10-01`). `Date.parse` reads that spelling through the host's legacy
parser, in the host's zone, so `-1-10-01` read as 2001-01-10. Every
reader read it that way, `isOutsideTemporalYearRange` included, for both
kinds (measured below). The resolver stays field-agnostic; the range is
asked of the day itself. A day inside 0001..9999 and a sub-day
placeholder's instant are spelled as before.
## Measured, before and after (scratch probes, not committed)
The card's object has two rows: `opened_at` (`datetime`) at
2026-03-01T10:00Z and 1500-03-01T10:00Z, `placed_on` (`date`) on the
same days, and `opens_at` (`time`) at 09:00 and 12:00. The engine column
is `engine.find` on InMemoryDriver. The REST column is `POST
/api/v1/data/:object/query` on SqlDriver over SQLite (better-sqlite3).
| `where` | resolved to (base) | base, engine on memory | base, REST on
SQLite | now, both |
|:--|:--|:--|:--|:--|
| `opened_at $gt {8000_years_from_now}` | `10026-10-01` | 200, both rows
| 200, both rows | 400 `INVALID_FILTER`, year 10026 |
| `opened_at $lt {2027_years_ago}` | `-1-10-01` (read as 2001-01-10) |
200, the 1500 row | 200, the 1500 row | 400, year -1 |
| `opened_at $lt {1977_years_ago}` | `0049-10-01` | 200, no row | 200,
no row | 400, year 49 (the `datetime` floor) |
| `placed_on $gt {8000_years_from_now}` | `10026-10-01` | 200, both rows
| 200, both rows | 400, year 10026 |
| `placed_on $lt {1977_years_ago}` | `0049-10-01` | 200, no row | 200,
no row | 200, no row (a `date` keeps 0001..0999) |
| `opens_at $gt {8000_years_from_now}` (see note) | `+010026-10-01` |
200, both rows | 200, both rows | 400 (the time class) |
| `having` `max(opened_at) $gt {8000_years_from_now}` | | 200, both
groups | | 400 |
| `judgeFilter` of the first two rows | | `{ ok: true }` | | `{ ok:
false, code: INVALID_FILTER, status: 400 }`, execution's message |
| control: `opened_at $lt {100_years_ago}` / `$gt` | `1926-10-01` | the
1500 row / the 2026 row | the same | unchanged |
`{2027_years_ago}` answered one row on the base, not two as the card
records: the base spelling is read by the legacy parser as a day in
2001.
Note on the `time` row: it was measured on this branch before the `time`
commit (`d5a8e100b`), with core's new spelling already in, through `POST
/api/v1/data/:object/query` on InMemoryDriver and on SQLite. On the base
the value is `10026-10-01`, which also compares as text above every
`HH:MM:SS`, so that row is read from the code, not measured on
`2f2fa11d7`.
## The PM's hypotheses (zone 2)
- **H1, reproduced** on `2f2fa11d7` (table above). One cell has moved
since the card was measured: `$lt {2027_years_ago}` answers the 1500
row, where the card records both rows.
- **H2, partly falsified.** The check lives where the PM expected: the
engine's resolution stage, which holds the field map. But judging "the
values resolution produced" with `isOutsideTemporalYearRange` could not
see years at or below 0. Core spelled them `-1-10-01` / `0-10-01`, and
the range read those as 2001 / 2000 for both kinds. Measured in UTC,
America/New_York and Asia/Shanghai, `isOutsideTemporalYearRange`
answered `false`. The defect is upstream, so it is fixed at the
producer: one field-agnostic spelling change in `filter-tokens.ts`,
which the claim lists as staying field-agnostic, not as read-only. A
consumer-side parse of the unpadded spelling would have been the lenient
fallback AGENTS.md rules out. Once the spelling is readable, triage's
two halves hold together: resolution refuses per kind, and the
`datetime` floor of 1000 applies to a resolved placeholder as it does to
a literal.
- **H3, both doors agree.** `judgeWhereAdmission` calls the same stage
function with the same judge. The pin asserts that `judgeFilter` returns
execution's exact `code`, `status` and `message`.
- **H4, `having` is in reach.** On the base, `max(opened_at) $gt
{8000_years_from_now}` kept both groups. It is now refused by the
aggregated column's class (`aggregatedRowColumnClasses`: `min` / `max`
keep the field's kind, a `day` bucket is a `date`). `count` / `sum` /
`avg` columns are not temporal and are not judged. Text operators are
stepped over on `having`, as the `having` door does.
- **H5, two output forms.** `{N_years_*}` and every other day-or-coarser
macro resolve to a calendar day: `YYYY-MM-DD`, or now `±YYYYYY-MM-DD`
outside 0001..9999. `{now}` / `{N_hours_*}` / `{N_minutes_*}` resolve to
an instant (`toISOString`). The message's year is pinned for both forms:
days `+010026-09-30` (10026), `-000001-09-30` (-1) and `0049-09-30`
(49), and the instant `{80000000_hours_from_now}` (11153).
## Faces that resolve `{tokens}` outside the engine
- `packages/services/service-analytics/src/analytics-service.ts` and
`dataset-executor.ts` are **already refused for a time-dimension member
(code evidence plus a predicate measurement, not measured end to end);
out of scope otherwise (`domain:services`)**. Both resolve the query's
positions first and then pick a strategy. The ObjectQL strategy hands
the resolved literal to `engine.aggregate`, where the temporal-comparand
door refuses it as a literal. The native-SQL strategy declines a
time-dimension member whose comparand core's
`isUninterpretableTemporalComparand` refuses (`comparand-shape.ts`
`findUninterpretableTemporalMember`). That predicate answers `true` for
both the base spelling and the new one (measured: `-1-10-01`,
`10026-10-01`, `-000001-10-01`, `+010026-10-01`, both kinds), so the
declined query reaches the engine door. The residual is the hole that
package already records for a temporal column not declared as a time
dimension.
- `packages/services/service-analytics/src/read-scope-sql.ts` is
**changed on the ObjectQL face, out of scope on the native-SQL face**.
On the ObjectQL face the engine resolves the original scope inside
`where`, so this PR's judge applies. On the native-SQL face
(`compileScopedFilterToSql`) the resolved tree is lowered straight to
SQL. A read scope is platform-authored (CEL / stored policy), not caller
input, and this is `domain:services`. Not measured.
-
`packages/services/service-analytics/src/strategies/filter-normalizer.ts`
is **already compliant**: it has no resolver call, only a doc comment
naming `resolveFilterTokens`' `FILTER_TOKEN_UNRESOLVED`.
- `packages/plugins/plugin-security/src/position-catalog-refusal.ts` is
**already compliant**: it never resolves. A placeholder-shaped position
name is compared in JavaScript with `===` against catalog names
(`catalogCarries`), so no resolved instant reaches a comparison.
- `packages/services/service-automation/src/builtin/template.ts` is
**changed, through the engine**. `interpolateFilter` passes a recognised
filter placeholder through verbatim to the engine, whose resolution
stage now judges it.
- `packages/core/src/utils/analytics-date-range.ts` is **unaffected** (a
resolver caller the dispatch list did not name). It asks only fixed
tokens (`today`, `week_start`, `7_days_ago`, …), which never land
outside 0001..9999.
## Pins
- `packages/objectql/src/engine-resolved-token-year-range.test.ts`
(recording driver, `Date` pinned to 2026-09-30T12:00Z). It pins every
position for both kinds; `code` + `status` + the placeholder + `resolved
to "…" (the year N)` + the kind's years in the message; and the door's
words per position (`at aggregations[1].filter…`, the `having` column
phrase). It also covers a list member, a `$between` bound, implicit
equality, a `$not` branch and the array sugar. Further cases: the
`datetime` floor (`{1977_years_ago}`, `{1027_years_ago}`) with the
`date` control reaching the driver as its day, and the in-range control
on every position, reaching the driver as the day it names. The rest are
`judgeFilter` equal to execution, the `time` class (year 0 still read),
and a text column / context placeholder not judged.
- `packages/rest/src/data-resolved-token-year-range.test.ts` (SqlDriver
on SQLite, the card's two rows plus a `time` column) checks 400
`INVALID_FILTER` naming the placeholder and the year at `where`, the
per-aggregation `filter` and `having`, with no read of the object. The
controls answer the right rows, filter counts and `having` groups.
- `packages/core/src/utils/filter-tokens-year-outside-range.test.ts`
checks the expanded-year spelling on both sides of 0001..9999 and year
0. It runs in UTC and Asia/Shanghai, with the range reading the year it
names, and covers the edges inside, the `datetime` floor's edge and the
sub-day instant.
## Verification
Tests ran on `fc3fc97f3`. The final head `4c5f259e5` adds only a merge
of `origin/main` that touches no package source (`pr-automation.yml`,
`packages/spec/CHANGELOG.md`, `scripts/check-empty-changeset.mjs`).
- `@objectstack/core`: `vitest run --project local`: 62 files / 1822
tests passed; `typecheck` passed.
- `@objectstack/objectql` (dist rebuilt): `vitest run --project local`:
350 files / 6860 tests passed; `--project repo` 1 / 5 passed;
`typecheck` (incl. `check:test-typecheck`) passed.
- `@objectstack/rest`: `typecheck` passed. The new pin plus
`aggregation-filter-array-membership`, `data-temporal-year-range` and
`rest-aggregate-positions` passed (21 passed, 26 named skips for
unprovisioned live PG / MySQL cells). Earlier, at `574bcbb51`, the 27
rest suites that mention tokens, temporal values or years: 755 passed,
65 skipped. The full rest suite is declared to CI.
- **Gates**, derived with `node scripts/pm/dispatch-gates.mjs --commands
--repo objectstack-ai/objectstack` at `4c5f259e5` (67 commands, all run
at that head, exits recorded). 65 exited 0 and `--ran` reconciles 67 /
67. **NOT MEASURED: `pnpm check:dual-build-cjs-loads` and `pnpm
check:type-check-debt`, reason: exit 3 PREREQUISITE NOT MET.** Both read
every workspace package's built `dist/`, and only the core / objectql /
rest closure was built here. This diff changes no build config, no
`exports` and no published type. Declared to CI.
- `node scripts/check-driver-conformance.mjs`: before "50 covered
cell(s), 0 in the DEBT ledger, 0 exempt"; after the same.
## Reverse verification (ablations, one-off, not committed)
Each leg went through `scripts/ablation-replace.mjs` in wrap mode, with
a trap restore. A suite resolving through `dist/` got a rebuild and
`scripts/ablation-dist-preflight.mjs` on both legs. Each restore was
proven by blob equality and an empty `git diff HEAD`, and the tree was
clean after. All three went red, the usual direction.
1. **The stage never calls its judge** (`engine.ts`, at `d5a8e100b`).
The objectql pin went 7 / 9 red and the REST pin 1 / 2 red, with the
marker in objectql's `dist/` (4 files). Restored: 9 / 9 and 2 / 2 green,
marker absent.
2. **Core's expanded-year spelling reverted** (`filter-tokens.ts`, at
`4dcb910f1`). The core pin went 12 / 24 red (every outside-range row,
both hosts). The objectql pin went 6 / 8 red through core's rebuilt
`dist/`, including `$in` with `{2027_years_ago}` answered rather than
refused. Restored: 24 / 24 and 8 / 8 green, with the original marker
back in `dist/` (2 files).
3. **The `time` branch answers "inside"** (`temporal-comparand-door.ts`,
at `d5a8e100b`). The first attempt was a no-op that `ablation-replace`
refused: its replacement contained the anchor, so the anchor count did
not drop and no mutation ran. Redone with a disjoint replacement, the
objectql pin went 1 / 9 red (exactly the `time` test). Restored: 9 / 9.
## Acceptance notes
- **Memory cell.** The refusal is pinned by the engine suite's recording
driver, because it answers before a driver is resolved. That follows the
lane convention stated in `data-no-operator-object-door.test.ts`'s
header: neither `objectql` nor `rest` depends on `driver-memory`. The
memory measurements in the table are from the scratch probe through
`engine.find` on InMemoryDriver.
- **PostgreSQL / MySQL** were not run (no live servers here). The
refusal precedes the driver, so a dialect adds nothing to it.
- **A placeholder inside a nested-relation condition** (`{ owner: {
opened_at: … } }`) is resolved by the parent's stage before the related
read. The related object's own door then refuses the resolved literal,
naming the value rather than the placeholder. This PR does not change
that.
- **Scope beyond the card's table.** The `time` class joins on the
bounded in-place rule: the same defect class (a resolved placeholder
bypasses the door's year class), the door's own pinned words, the same
file and the same gates.
- **Not addressed here, and reported for the seat to file (same
family):** a date macro whose offset lands past the instants a
JavaScript `Date` holds. A day macro resolves to the text `Invalid
Date`, so `opened_at $lt {300000_years_ago}` answers 200 with both rows
on memory and SQLite through `POST /api/v1/data/:object/query`. A
sub-day macro throws an uncoded `RangeError` inside the resolver, and
the same door answers 500 `INTERNAL_ERROR` for
`{99999999999999999999_minutes_ago}`. That mechanism is not the card's
extended-year text, and no shape for its refusal is pinned, so this PR
leaves it alone.
- `content/docs/releases/` and every `CHANGELOG.md` are untouched. The
changeset is `.changeset/20844-resolved-token-year-range.md`
(`@objectstack/core` minor, `@objectstack/objectql` minor, BREAKING).
`check:adr-0087-registration` reads its disposition as `not-required
(no-migration-prescription)`. The `05a7547c9` precedent registered no
ADR-0087 id, so `already-registered` has nothing to name.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 11d28c1 commit dcd3309
7 files changed
Lines changed: 912 additions & 52 deletions
File tree
- .changeset
- packages
- core/src/utils
- objectql/src
- rest/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
Lines changed: 84 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
34 | 40 | | |
35 | 41 | | |
36 | 42 | | |
| |||
92 | 98 | | |
93 | 99 | | |
94 | 100 | | |
95 | | - | |
| 101 | + | |
96 | 102 | | |
97 | 103 | | |
98 | 104 | | |
| |||
202 | 208 | | |
203 | 209 | | |
204 | 210 | | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
205 | 224 | | |
206 | | - | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
207 | 230 | | |
208 | 231 | | |
209 | 232 | | |
| |||
0 commit comments