Skip to content

Commit 89f87f2

Browse files
fix(core,objectql)!: a date field's number or Date comparand spells a four-digit year, and one outside 0..9999 is refused INVALID_FILTER / 400 (#20240) (#20261)
Fixes #20240 Clause-②: no (narrowing) A number or `Date` compared against a `date` field now spells its year with four digits, the spelling its ISO string already had. One whose UTC calendar day falls in a year below 0 or above 9999 has no `YYYY-MM-DD` form, and it is now refused `INVALID_FILTER` / 400 at the temporal-comparand door, as its ISO string already was. Both edits are in the one rule and its one predicate, in `@objectstack/core`. No driver source changes. ## What changed - `packages/core/src/utils/temporal-storage-form.ts`, `canonicalCalendarDay`: a non-negative year of a `Date` or a number is padded to four digits (`0999-06-15`, not `999-06-15`). A year below 0 or above 9999 keeps the spelling it had (`10000-01-01`, `-1-01-01`). The rule is shared with the write and read paths, and no ordered form is invented. - `packages/core/src/utils/temporal-comparand.ts`, `isUninterpretableTemporalComparand`: on `date`, a finite number or a valid `Date` whose UTC year is below 0 or above 9999 is uninterpretable. That includes a finite number past the `Date` range (±8.64e15). `NaN`, ±Infinity and an Invalid Date name no year, so they stay unjudged. The `datetime` and `time` branches are untouched. The module note about non-string comparands was rewritten so it stays true (the stop valve, below). - `packages/objectql/src/temporal-comparand-door.ts`: the door already calls that predicate on `where` (every verb, both spellings) and on a per-aggregation `filter`. It now words the new class in its own sentence, because such a comparand answers the wrong rows or a database error, not "compare false for every row". The result's `value` is widened from `string` to `unknown`. It is internal and not exported from the package. - Tests and `.changeset/20240-date-year-four-digits.md` (`minor`: core and objectql; `patch`: driver-sql and driver-memory). There is one one-clause DELIBERATE CORRECTION to `.changeset/20203-epoch-ms-date-comparand.md` (see Deviations). ## Stop valve: the ruling scoped #8690's own change; it is not a standing rule `temporal-comparand.ts` said: "Non-string comparands. A number is epoch milliseconds, a `Date` is an instant, `null` is a null test. The refusal scopes to strings by ruling." Every scope sentence on #8690, read in full: - Triage 5294745926: "Refusing a non-interpretable bare string on a temporal target is the queued work." - Maintainer ruling 5299879288, its only scope clause: "The **empty-string cell stays its own card** — B and C scope to non-empty strings and must not decide it in passing". - PR #8808's body lists "Non-string comparands untouched — a number is epoch milliseconds, a `Date` is an instant" under "Scope boundaries, each by ruling". The ruling itself contains no clause about non-strings. Reading: the ruling scoped #8690's own change (strings, the card's defect, and the empty-string boundary). The door's own reason for leaving non-strings alone was a fact ("both are read correctly today"). That fact is false for a `date` year outside 0..9999, which this card measured. The valve does not trip, so triage's refuse direction applies. The module note now says exactly this. ## The rows, base `0d3ec47137` → head The fixture has seven rows on a `date` field (six 2026 days plus `0999-06-15`), live on InMemoryDriver, SqlDriver on better-sqlite3, and PostgreSQL 16.13. The process zone is `America/New_York` and the server zone `Asia/Shanghai`. Each cell went through `engine.find` / `engine.aggregate` and through `POST /api/v1/data/:object/query` with a JSON round-tripped body. The two doors agree on all 288 comparable cells at base and at head. Over REST a `Date` is its ISO string (all 144 REST `Date` cells equal their ISO twins). Cells are `$gt` / `$lt` / `$eq`, given as memory · SQLite · PostgreSQL. | comparand | position | base | head | |:--|:--|:--|:--| | number for 0999-06-15 (`-30627504000000`), and its `Date` (engine) | `where` | 0/7/0 · 0/7/0 · 6/0/1 | 6/0/1 on all three | | the same | per-aggregation `filter` | 0/7/0 on all three | 6/0/1 on all three | | the same | `having` on `max(date)` | none / all four groups / none, on all three | c1,c2,c3 / none / c4, on all three | | ISO `0999-06-15T00:00:00.000Z` | all three positions | 6/0/1, and c1,c2,c3 / none / c4 | unchanged | | number for 10000-01-01 (`253402300800000`), and its `Date` | `where` | 6/1/0 · 6/1/0 · 0/7/0 | `INVALID_FILTER` / 400, no read | | the same | per-aggregation `filter` | 6/1/0 on all three | `INVALID_FILTER` / 400, no read | | number for -1-01-01 (`-62198755200000`), and its `Date` | `where` | 7/0/0 · 7/0/0 · `DATABASE_ERROR` 500 | `INVALID_FILTER` / 400, no read | | the same | per-aggregation `filter` | 7/0/0 on all three | `INVALID_FILTER` / 400, no read | | ISO `+010000-01-01T00:00:00.000Z` / `-000001-01-01T00:00:00.000Z` | `where`, per-aggregation `filter` | `INVALID_FILTER` / 400 | unchanged | | control, number / `Date` / ISO for 2026-02-01T10:00Z | all three positions | 1/4/2, and c2 / c1,c4 / c3 | unchanged | | the same numbers on a `datetime` field | `where` | memory, SQLite 7/0/0 on every year; PostgreSQL 500 for 10000 and -1 | unchanged (see Out-of-scope 2) | "No read" was measured with a counter on the object's `find` / `findOne` / `count` / `aggregate`: zero reads through the engine and REST, on all three drivers. Moved cells, classified mechanically: 180 of 867. Every one is a number or `Date` on the `date` field: 0999 at `where`, the per-aggregation `filter` and `having`, and 10000 / -1 at `where` and the per-aggregation `filter`. Zero moved among ISO strings, the 2026 control, the `datetime` control and `having` out-of-range. ## H2, the PostgreSQL raw number: not reproduced at base At `0d3ec47137` (and at the card's `615c468746`, whose query path is byte-identical: `git diff --stat` on core, both drivers, objectql, rest and metadata-protocol sources shows only four plugin-contract / plugin-loader files), no layer hands PostgreSQL the raw number for year -1. `SqlDriver.coerceFilterValue` → `toDateOnly` → `temporalStorageForm` binds the rule's `"-1-01-01"`, and the server refuses it with `22007 invalid input syntax for type date: "-1-01-01"`. That was captured from knex's query-error bindings on every one of the 9 statements. The card's message, `22009 time zone displacement out of range: "-62198755200000"`, is exactly what `select '-62198755200000'::date` answers. That is the pre-#20224 path, where `toDateOnly` returned a number unchanged. At head the refusal precedes the driver, with zero reads. ## H4's `having` leg: falsified, `having` never reaches the door `having` takes its own entry doors in `engine.aggregate` (`assertHavingIsFilterCondition`, `assertListComparandShapes`, `normalizeFilterComparandTypes`, `assertHavingIsEvaluable`). None of them is the temporal-comparand door, for any comparand. Measured at base on all three drivers, through both doors, on `max(date)`: - `having: { last_placed: { $lt: 'not-a-date' } }` keeps every group (200), and its `where` twin is refused 400; - the ISO string `+010000-01-01T00:00:00.000Z` under `$gt` keeps every group, and its `where` twin is refused 400. So a year-outside-range number or `Date` on `having` still compares as written: the number for 10000-01-01 under `$gt` keeps c1, c2, c3 at head, as at base. Wiring the door into `having` would also newly refuse the strings above. That is a second narrowing on a file (`engine.ts` / `having-filter.ts`) outside the claimed surface, so it is reported (Out-of-scope 1) and not done. The padding does reach `having`, and its 0999 cells are now right. ## H5, no collateral - **The rule and the predicate:** 63 shapes × 3 kinds (189 cells), base → head. 23 moved, all `date` × a number or `Date`: the 0..999 years now pad, and the predicate turns true outside 0..9999, the Date-range edges and past-range numbers included. Every `datetime` and `time` cell is byte-identical, and so is every string, `null`, bigint, boxed Number, boolean, object and array cell. - **End to end, a second fixture** (`date`, `datetime` and `time` fields; rows in 1000, 2026, 9999 and 0999): 3099 cells over `where`, the per-aggregation `filter` and `having` on all three fields, three drivers, both doors, and the read paths (`find`, `max` / `min` per group, a `groupBy` key). 72 moved, all a finite number past the `Date` range (±9e15) on the `date` field at `where` or the per-aggregation `filter`, now 400. Before, memory answered 0/0/0, SQLite 7/0/0 and PostgreSQL 500. Every cell for years 1000..9999 (number, `Date`, ISO, bare day) is byte-identical, as is every `datetime` and `time` cell, year 0, `'not-a-date'`, a wall clock and every read-path cell. - **Write paths:** 156 cells, 14 moved, all a year-0..999 number or `Date` on memory or SQLite, now stored padded (`0999-06-15`, `0000-06-15`). That covers `driver.create` / `driver.update`, plus `engine.insert` of a `Date`, which the engine accepts. PostgreSQL already stored a three-digit year's day (this sweep's short years were 0000 and 0999), but not a shorter one: under its default `DateStyle` (`ISO, MDY`) it stored the unpadded `9-03-04` as 2004-09-03 and refused `99-03-04` (`22008`). That was measured under ablation A (below) and with `select '9-03-04'::date`, and it is pinned by the 0009 / 0099 cells. MySQL 8.0, measured at the driver door in round 2, stored `99-03-04` as 1999-03-04. All three dialects now store the day. The engine and REST write doors still refuse a number on `date` (`VALIDATION_FAILED` / 400), unchanged, and every out-of-range write cell is unchanged. - **`service-analytics`' raw-SQL decline** (`NativeSQLStrategy.canHandle`, not edited): 0 of 7 moved. It reads a time dimension with the `datetime` rule (`lookupMember(...)?.type === 'time' ? 'datetime' : null`), so the new `date` branch is unreachable from it. ## Tests New pins: - `packages/core/src/utils/temporal-storage-form.test.ts`: years 0000..0999 pad for the number, its `Date` and its ISO string; the spellings sort chronologically; out-of-range years keep their spelling. - `packages/core/src/utils/temporal-comparand.test.ts` (new): the `date` year class, the datetime / time controls, `NaN` / Infinity / Invalid Date unjudged, and an invariant that for a finite number the predicate refuses exactly when the rule cannot spell `YYYY-MM-DD`. - `packages/objectql/src/engine-date-year-range-door.test.ts`: `code` AND `status` plus zero driver reads, with its positive control (years 0000, 0999, 2026, 9999 reach the driver) in the same `it()`. Also covered: every comparand position and both spellings, all six verbs, the per-aggregation `filter`, and the datetime / time controls. - `packages/drivers/driver-memory/src/memory-20240-date-year-spelling.test.ts` and `packages/drivers/driver-sql/src/sql-driver-20240-date-year-spelling.test.ts` (the live-dialect matrix): years 0009, 0099 and 0999 answer one count for the number, its `Date` and its ISO string, plus the 2026 control, `$in` / `$between`, and the create / update write path. - `packages/rest/src/data-query-date-year-range.test.ts`: engine and REST over SqlDriver, with `where`, the per-aggregation `filter` and `having` for 0999 and 2026; 10000 and -1 refused with no read of the object; a datetime control. Whole suites, after merging `main` (`e46218674b`) and rebuilding the closure of rest, driver-memory and driver-sql at `2f219baa71`: - `@objectstack/core`: 58 files / 1522 passed. - `@objectstack/driver-memory`: 57 / 1374 passed. - `@objectstack/objectql`: 320 / 5777 passed. - `@objectstack/driver-sql` with live PostgreSQL (`TZ=America/New_York`, server `Asia/Shanghai`): 199 files passed, 3 skipped / 3888 passed, 88 skipped; the reporter says sqlite RAN, live postgres RAN, live mysql NOT RUN. In round 2 (below), at `e83b248a39`, the whole driver-sql suite ran against both live servers: PostgreSQL 16.13 and a private MySQL 8.0.46 (server zone `+08:00`), with `OS_EXPECT_LIVE_DIALECT_MATRIX=1`. Result: 202 files / 4644 passed, 1 skipped; the reporter says all 3 dialects were exercised. - `@objectstack/rest` (every query-route test): 203 / 3592 passed, 1 skipped. Typecheck at `2f219baa71`: core, objectql, driver-memory, driver-sql and rest each exit 0. Every new test file is in a compiled program (`--listFiles`: `tsconfig.test.json` for core, objectql and rest; `tsconfig.json` for the two drivers). The test-typecheck ledgers held: core 4, objectql 234 errors / 65 signatures, rest 0. ## Reverse verification Both ablations ran after the fix was committed, through `ablation-replace` (wrap mode, restore trap on the absolute path). Core was rebuilt in every leg and `ablation-dist-preflight` passed (present on the mutate leg, `--absent` on the restore leg), because objectql and rest resolve `@objectstack/core` through its `dist`. The direction was predicted before each run. - **A, padding off.** The whole line `const yyyy = y >= 0 ? String(y).padStart(4, '0') : String(y);` became `const yyyy = String(y).padStart(0, '0');`; anchor 1 → 0, blob `123c85a117` → `685087daf5`. The contract review's literal `padStart(4, '0')` → `padStart(0, '0')` substitution gives `d803edfcb6`. Both ids reproduce from the source, and both unpad a non-negative year the same way, so the counts agree: - core 8 failed / 82 passed; - memory 9 failed / 3 passed; - driver-sql 12 failed / 12 passed / 1 skipped; - REST 3 failed / 5 passed; - objectql 6 passed. Predicted red, observed red, with one more red than first predicted. On the first run PostgreSQL went red on the write path, which I had predicted green. Its default `DateStyle` (`ISO, MDY`) reads the unpadded `9-03-04` as **2004-09-03**, a different day, silently, and refuses `99-03-04` (`22008`). The padded forms read right. So I added 0009 and 0099 cells to both driver pins and re-ran; the counts above are that run. The base defect is in this card's class and the padding closes it. - **B, refusal off** (`return isOutsideCalendarDayYears(value);` → `return isOutsideCalendarDayYears(value) ? false : false;`; blob `912a0609ab` → `f150d2d782`): - core 3 failed / 87 passed; - objectql 4 failed / 2 passed (the datetime/time control and the unjudged `NaN` stay green); - REST 2 failed / 6 passed; - memory 12 passed and driver-sql 24 passed, as predicted, since the drivers are not the door. - **Restore legs:** after both, the blob equals HEAD, `git diff HEAD` is empty and `git status --porcelain` is clean. Core was rebuilt and the preflight `--absent` exited 0. Pins: core 90, objectql 6, memory 12, driver-sql 24 + 1 skipped, REST 8, all passed. ## Gates `dispatch-gates --commands --repo objectstack-ai/objectstack` at `2f219baa71` derives 65 commands (the dispatch clue's 50, plus the changeset families and more). Every one ran at `2f219baa71`: 62 exit 0, 1 exit 1, 2 exit 3. `--ran` with the recorded exit codes reads: 65 derived, 63 run, 2 NOT MEASURED (derived from exit 3), 0 unrun. - `check-empty-changeset --base origin/main`: **exit 1, on purpose.** It names exactly `.changeset/20203-epoch-ms-date-comparand.md`, which is the DELIBERATE CORRECTION class. Please confirm it here; it must not be restored from the base. - `pnpm check:dual-build-cjs-loads` and `pnpm check:type-check-debt`: **NOT MEASURED**, `PREREQUISITE NOT MET`. Both read every package's built output, and CI builds the whole `./packages/*` before them. - Also run, all exit 0: the six roster gates whose roster sits under a path of mine (`check-changeset-fixed`, `check:authz-resolver`, `check:error-code-casing`, `check:filter-alias-parity`, `check:object-def-param-keys`, `check:tenant-chokepoint`), and `check-issue-citations --base e462186` (11 citations, all resolve). `check-adr-0087-registration` reads the `not-required (no-migration-prescription)` disposition, and `check-changeset-no-major` finds no `major`. - Lint is a declared narrowing: `eslint --no-inline-config --format json` on the 9 changed `.ts` files at `2f219baa71` reports 9 files, 0 errors / 0 warnings / 0 fatal, and `--print-config` resolves a config for each. The two `.md` files resolve none, so they are outside eslint's population. `eslint.config.mjs` enables no type-aware linting (no `parserOptions.project`, no `projectService`, no `TypeChecked` preset), so no untouched file's verdict can move. ## Round 2: contract review 5857959126 **Must-change 1: the live MySQL cell** (CI job 108661118941, `Temporal Conformance`). It failed with `expected '1909-03-04' to be '0009-03-04'` on the write-path assertion only; the SQLite and PostgreSQL cells passed, and so did MySQL's `where` cells, which read ids and not dates. The cause is the read, not the write: - mysql2 3.23.1, `lib/packets/packet.js` `parseDate`, rebuilds a `DATE` as `new Date(y, m - 1, d)`, or as `new Date(Date.UTC(y, m - 1, d))` under `timezone: 'Z'`, which driver-sql sets (`sql-driver.ts`, the `timezone: 'Z'` connection override). Both constructors map a year 0..99 to 1900..1999. - On a private MySQL 8.0.46 (server zone `+08:00`, the CI setting), the bound `'0009-03-04'` is stored as `0009-03-04` (`CAST(... AS CHAR)` and `DATE_FORMAT` agree). mysql2 hands back 1909-03-04 for it, 1999-03-04 for `0099-03-04` and 1900-06-15 for `0000-06-15`, while year 999 survives. - The failure reproduced locally before the fix, byte for byte (1 failed / 23 passed). The fix is in the test only; there is no driver `src` change and no skip. The write-path cell now reads the STORED text through a raw per-dialect cast (`::text` on PostgreSQL, `cast(... as char)` on MySQL, `cast(... as text)` on SQLite) on every dialect, the pattern `sql-driver-12380-json-roundtrip.test.ts` uses. It adds the year-99 write, the one MySQL stored wrong at base, and checks `$eq` on both short-year days. The year-999 row still reads back through the driver too. Result: 36 passed across SQLite, PostgreSQL and MySQL. Reverse verification of the new cell, with the fix committed and ablation A re-run through `ablation-replace` (driver-sql reads core from source, so no rebuild was owed). Direction predicted before the run: 14 failed / 22 passed, split SQLite 9, PostgreSQL 3 (0009, 0099, the write path) and MySQL 2 (0099 `$lt`, the write path). MySQL reads `9-03-04` and `999-06-15` literally, so its other cells stay green. Observed exactly that. Restored: blob equals HEAD, `git diff HEAD` empty, porcelain clean. MySQL at the driver door, base `e46218674b` (a detached worktree) against head, create and update alike: | written | base: stored / read back | head: stored / read back | |:--|:--|:--| | number or `Date` for 0009-03-04 | `0009-03-04` / `1909-03-04` | `0009-03-04` / `1909-03-04` | | number or `Date` for 0099-03-04 | **`1999-03-04`** / `1999-03-04` | `0099-03-04` / `1999-03-04` | | bare day `'0099-03-04'` | `0099-03-04` / `1999-03-04` | unchanged | | number or bare day for 0999-06-15 | `0999-06-15` / `999-06-15` | `0999-06-15` / `0999-06-15` | | number or bare day for 0000-06-15 | `0000-06-15` / `1900-06-15` | unchanged | | number for 2026-02-01T10:00Z | `2026-02-01` / `2026-02-01` | unchanged | `$eq` found every row it wrote, at base and at head. **Must-change 2, the prose.** The changeset's "PostgreSQL already stored the day." now says what was measured: PostgreSQL and MySQL stored a three-digit year's day, and PostgreSQL misread (`9-03-04` as 2004-09-03) or refused (`99-03-04`, `22008`) a shorter one, while MySQL stored `99-03-04` as 1999-03-04. Its "Unchanged … every read-path presentation" is now scoped to the three dialects it was measured on, and it names the MySQL read cells above. The H5 write-path sentence in this body is aligned the same way. Gates for this round, at `e83b248a39`: the 59 commands derived for the two changed paths ran, plus the six roster gates and `check-issue-citations`. Result: 63 exit 0; `check-empty-changeset` exit 1 on the same DELIBERATE CORRECTION; `check:dual-build-cjs-loads` and `check:type-check-debt` exit 3 (PREREQUISITE NOT MET). `--ran` against the full derivation at `e83b248a39` reads 65 derived, 63 run, 2 NOT MEASURED, 0 unrun. The six families this round's paths do not reach carry their `2f219baa71` records. Also clean: eslint on the changed test (0 / 0) and driver-sql typecheck (exit 0, file in the program). The core pins passed (90). ## Deviations - **`.changeset/20203-epoch-ms-date-comparand.md`, a one-clause DELIBERATE CORRECTION.** Its "Not changed" list named "every `Date` … on a `date` field" and "a number outside the `Date` range", and this card moves both in the same release. It now reads "… every `Date` and every string on a `date` field (#20240, in the same release, then pads a `Date`'s or a number's year 0..999 to four digits and refuses one whose year falls outside 0..9999, a number past the `Date` range included), and every `datetime` and `time` reading." Nothing else in the note changed. `.changeset/20176-*` and `.changeset/20212-*` were read and left alone: 20176's "Not changed" list is scoped before and after its own change (the reading #20224's accepted review applied to that sentence's "every `where` answer"), and 20212 is RLS. - **H4's `having` leg is not delivered** (above). `having` never reached the door, and wiring it in is outside the claimed surface and a second narrowing. - **Where each driver is pinned** follows PR #20224 / #20202. `packages/rest` has no driver-memory dependency, and a live PostgreSQL is reachable only through driver-sql's testkit (its isolation test forbids reading the env var elsewhere). So memory and PostgreSQL are pinned at their driver doors, and their engine and REST cells are the measured table above. - `main` was merged once (`e46218674b`) before opening, per AGENTS.md §10. The whole suites, typecheck and gate union above were re-run on the merged head. `main` has moved again since (`443b2f4fdc`), and the queue rebuilds on it. ## Acceptance notes - `driver-mongodb`'s `storageDateValue` (`mongodb-temporal.ts`) is its own copy of the `date` rule. It spells a `Date`'s year unpadded (`${y}-${m}-${d}`), as core did before this PR, and returns a number unchanged (#20203's note). After this PR it disagrees with core on both, as a code reading only: no `mongod` is reachable here, so it is NOT MEASURED. Carrier none, beside the same note on PR #20202 and PR #20224. - `IObjectQLEngine.judgeFilter` runs the same `where` admission, so the analytics read-scope judgement (#19995) now refuses a scope carrying such a number or `Date` on a `date` field. This is a code reading; it was not measured. - The rule keeps `10000-01-01` / `-1-01-01` for an out-of-range `Date` on the write path, so `engine.insert` of such a `Date` still stores that text on memory and SQLite, unchanged. ## Out-of-scope findings (reported for the seat; nothing filed from here) 1. **class (a) · reach: public door, a measured wrong answer.** `having` on a temporal aggregated column does not reach the temporal-comparand door for any comparand. `POST /api/v1/data/:object/query` with `having: { last_placed: { $lt: 'not-a-date' } }` over `max(placed_on)` answers 200 and keeps every group on memory, SQLite and PostgreSQL, while the `where` twin answers `INVALID_FILTER` / 400. The number for 10000-01-01 under `$gt` keeps three groups where a refusal is due. Seam: runtime `packages/objectql/src/engine.ts` (`aggregate`, the `having` entry doors) → `having-filter.ts`. Family: the temporal-comparand door's positions (#8690 `where`, #20148 per-aggregation `filter`; `having` missing). Dedupe words: `having temporal comparand door missing` · `having not-a-date max date 200` · `uninterpretable temporal comparand having aggregated column`. 2. **class (a) · reach: public door, a measured wrong answer.** The `datetime` rule spells a year outside 0..9999 in extended ISO (`+010000-01-01T00:00:00.000Z`, `-000001-01-01T00:00:00.000Z`), which sorts as no instant does. On memory and SQLite, `where: { opened_at: { $gt / $lt / $eq: X } }` for 10000-01-01 answers 7 / 0 / 0 (the instant's answer is 0 / 7 / 0), in number, `Date` and ISO form alike, and PostgreSQL answers 500 (`22009` / `22007`) for 10000 and -1. The ISO string passes the `datetime` door, because `Date.parse` reads extended years. Seam: runtime `packages/core/src/utils/temporal-storage-form.ts` `canonicalUtcDatetime` and `temporal-comparand.ts` `readsAsInstant` → both drivers. Family: this card's (a year outside the four-digit range), on the arm the claim keeps byte-identical. Dedupe words: `datetime comparand year 10000 extended ISO text order` · `+010000 datetime filter postgres 22009` · `datetime storage form year outside 0 9999`. 3. **class (a) · reach: public door, a measured wrong answer.** PostgreSQL has no year 0000. A `date` comparand in year 0 answers `DATABASE_ERROR` 500 (`22008`) on PostgreSQL in every spelling, bare day `'0000-06-15'` included, while memory and SQLite answer 7 / 0 / 0. That is at base and at head (at base the number spelled `0-06-15`, also `22008`). A direct driver write of year 0 is refused `22008` too. Triage's pad range (0..999) includes year 0, so this PR pads it, and whether year 0 is in the `date` range is not decided here. Family: this card's. Dedupe words: `postgres date year 0000 out of range 22008` · `date comparand year zero postgres 500` · `ISO year 0000 1 BC postgres date`. 4. **class (a) · reach: public door, measured.** The REST create door accepts an extended-year ISO string on a `date` field and stores it verbatim. `POST /api/v1/data/:object` with `placed_on: '+010000-01-01T00:00:00.000Z'` answers 201 and stores that text (not a `YYYY-MM-DD` day) on memory and SQLite, and `DATABASE_ERROR` 500 on PostgreSQL; this is unchanged by this PR. Family: this card's, on the write door. Dedupe words: `date field write extended year ISO stored verbatim` · `create date +010000 stored not YYYY-MM-DD` · `date write door year 10000 postgres 500`. Findings 2, 3 and 4 are one family with this card (a temporal value outside the years a fixed-width text or a backend can hold), at three more positions, so they are named for one closing card rather than three. 5. **class (a) · reach: public door, a measured wrong answer.** On MySQL, a stored date in a year below 100 reads back a century late. `POST /api/v1/data/:object` with `placed_on: '0009-03-04'` answers 201 and the server stores `0009-03-04`, but `POST /api/v1/data/:object/query` returns `placed_on: '1909-03-04'`; `'0099-03-04'` returns `'1999-03-04'`, year 0 returns 1900, and `find` / `findOne` at the driver door agree. A `datetime` written as `'0009-03-04T10:00:00.000Z'` is stored as `0009-03-04 10:00:00.000` and returned as `'2004-09-03T10:00:00.000Z'`. Measured at the head's REST door, and at the driver door at base `e46218674b` and head, with identical read-back: the write went in as a bare string, which is unchanged base → head. Cause, for DATE: mysql2 `parseDate` rebuilds the column with `Date.UTC(y, m - 1, d)` under driver-sql's `timezone: 'Z'`, and `Date.UTC` maps years 0..99 to 1900..1999. The `datetime` read was not traced. Seam: runtime `packages/drivers/driver-sql/src/sql-driver.ts` (the mysql2 connection options and the read presentation) → mysql2 `parseDate` / `parseDateTime`. Family: driver-sql's MySQL read path for a year below 100; separate from findings 2 to 4. Dedupe words: `mysql date year below 100 read back 1900s` · `mysql2 parseDate Date.UTC year 0009 1909` · `mysql datetime year 9 presented 2004`. --- _Generated by [Claude Code](https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 80c29a1 commit 89f87f2

11 files changed

Lines changed: 985 additions & 26 deletions

‎.changeset/20203-epoch-ms-date-comparand.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,4 +28,4 @@ The rule is shared by the drivers' write and read paths too:
2828
- `create()` / `update()` on either driver, given a number for a `date` field, stores its UTC day. Before, `driver-memory` and SQLite stored the number, and PostgreSQL refused the statement. The engine and REST write doors refuse a number on a `date` field before a driver sees it (`VALIDATION_FAILED`), as before.
2929
- A number already stored in a SQLite `date` column is read back as its UTC day by `find()`, a `groupBy` key and `distinct()`. Only a direct driver write could have put one there.
3030

31-
Not changed, measured identical before and after: `NaN`, ±Infinity, a number outside the `Date` range (past ±8.64e15), a bigint, an epoch-millisecond string, every `Date` and every string on a `date` field, and every `datetime` and `time` reading. `driver-mongodb` keeps its own copy of the `date` rule and is not changed here.
31+
Not changed, measured identical before and after: `NaN`, ±Infinity, a number outside the `Date` range (past ±8.64e15), a bigint, an epoch-millisecond string, every `Date` and every string on a `date` field (#20240, in the same release, then pads a `Date`'s or a number's year 0..999 to four digits and refuses one whose year falls outside 0..9999, a number past the `Date` range included), and every `datetime` and `time` reading. `driver-mongodb` keeps its own copy of the `date` rule and is not changed here.
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
"@objectstack/core": minor
3+
"@objectstack/objectql": minor
4+
"@objectstack/driver-sql": patch
5+
"@objectstack/driver-memory": patch
6+
---
7+
8+
fix(core,objectql)!: a number or `Date` compared against a `date` field spells its year with four digits, and one whose UTC year falls outside 0..9999 is refused `INVALID_FILTER` / 400, as its ISO string already was (#20240)
9+
10+
Clause-②: no (narrowing)
11+
12+
<!-- adr-0087: not-required (no-migration-prescription) a query-time refusal of a comparand VALUE on a `date` field: no authorable key, spelling or stored shape moves, `packages/spec` is untouched, and a stored row keeps the form it has. What is refused is a number or `Date` whose day has no `YYYY-MM-DD` form, and which in-range day the caller meant is not something a ledger entry can decide. The other categories are closed on facts: the packages publish (not `unpublished`); no ADR-0087 id covers a comparand value (not `registered` / `already-registered`); and the change is runtime behaviour, not a declaration (not `runtime-interface-only` / `type-surface-only`). -->
13+
14+
**BREAKING**: this narrows what a filter on a `date` field accepts. A number or `Date` whose UTC calendar day falls in a year below 0 or above 9999 used to answer 200 with the wrong rows, or a 500 on PostgreSQL; it now answers `INVALID_FILTER` / 400. It ships as `minor` under the launch-window convention for accept-set narrowings.
15+
16+
`temporalStorageForm(value, 'date')` in `@objectstack/core` spelled the year of a `Date` or an epoch-millisecond number unpadded: `999-06-15`, `10000-01-01`, `-1-01-01`. The ISO string and the bare day of the same instant spelled `0999-06-15`, and as text an unpadded year sorts as no day does. Measured through `engine.find` / `engine.aggregate` and `POST /data/:object/query` (the two doors agree), on a `date` field holding six 2026 days and 0999-06-15, `$gt` / `$lt` / `$eq`:
17+
18+
| position | comparand | before: memory · SQLite · PostgreSQL | now, on all three |
19+
|:--|:--|:--|:--|
20+
| `where` | a number (or, in-process, a `Date`) for 0999-06-15 | 0/7/0 · 0/7/0 · 6/0/1 | 6/0/1 |
21+
| per-aggregation `filter` | the same | 0/7/0 on all three | 6/0/1 |
22+
| `having` on `max(date)` | the same | no group / every group / no group | the three 2026 groups / none / the 0999 group |
23+
| `where` | a number (or `Date`) for 10000-01-01 | 6/1/0 · 6/1/0 · 0/7/0 | `INVALID_FILTER` / 400 |
24+
| `where` | a number (or `Date`) for -1-01-01 | 7/0/0 · 7/0/0 · `DATABASE_ERROR` (500) | `INVALID_FILTER` / 400 |
25+
| per-aggregation `filter` | either of those two | 6/1/0 and 7/0/0 on all three | `INVALID_FILTER` / 400 |
26+
| `where`, per-aggregation `filter` | the ISO string of either | `INVALID_FILTER` / 400 | unchanged |
27+
28+
What changes:
29+
30+
- The rule pads a year from 0 to 999 to four digits, for a `Date` and a number alike, so a number, its `Date` and its ISO string spell one day. `driver-sql` (`toDateOnly`, `temporalFilterValue`), `driver-memory` (`coerceTemporalValue`) and the engine's per-aggregation `filter` and `having` all call it.
31+
- `isUninterpretableTemporalComparand('date', value)` is now also true for a finite number or a valid `Date` whose UTC year is below 0 or above 9999, a finite number past the `Date` range (±8.64e15) included. The engine's temporal-comparand door refuses such a comparand on `where` for every verb (`find`, `findOne`, `count`, `aggregate`, `update`, `delete`), in both the object and the array spelling, and in a per-aggregation `filter`, before any driver read. `IObjectQLEngine.judgeFilter` runs the same door.
32+
- The write path: `create()` / `update()` on `driver-memory` or SQLite, given a year-0..999 number or `Date` for a `date` field, now stores `0999-06-15` where it stored `999-06-15`; `engine.insert` of such a `Date` does the same. PostgreSQL and MySQL already stored a three-digit year's day, but not a shorter one: under its default `DateStyle` (`ISO, MDY`) PostgreSQL stored the unpadded `9-03-04` as 2004-09-03 and refused `99-03-04` (`22008`), and MySQL 8.0 stored `99-03-04` as 1999-03-04. All three dialects now store the day. A year outside 0..9999 keeps the spelling it had on the write and read paths; no ordered form is invented for it.
33+
34+
**Who is affected.** A caller that compares a `date` field with an epoch-millisecond number or a `Date` in a year below 0 or above 9999. No writer that stores or queries such a day has been measured; the reach is the public query door.
35+
36+
**Fix.** Compare against a `YYYY-MM-DD` day, or a number or `Date` whose UTC calendar day falls in a four-digit year.
37+
38+
**Unchanged**, measured identical before and after on memory, SQLite and PostgreSQL through the engine and REST: every `datetime` and `time` cell, the same numbers included; every string comparand on a `date` field; every number and `Date` in the years 1000 to 9999; `NaN`, ±Infinity and an Invalid Date, which name no year and are not judged; and every read-path presentation on those three. On MySQL, measured at the driver door, a stored year from 100 to 999 now reads back padded (`0999-06-15`, where it read `999-06-15`); a stored year below 100 still reads back a century late (`0009-03-04` as `1909-03-04`, mysql2's `Date.UTC` reading of a `DATE`), which this change does not touch. `having` does not reach the temporal-comparand door for any comparand, so a number or `Date` outside 0..9999 there is still compared as written. `service-analytics`' raw-SQL decline reads a time dimension by the `datetime` rule, so its answer does not move. `driver-mongodb` keeps its own copy of the `date` rule and is not changed here.
Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
//
3+
// [#20240] `isUninterpretableTemporalComparand` on a `date` column judges one
4+
// non-string class: a finite number or a `Date` whose UTC calendar day falls in
5+
// a year below 0 or above 9999. Such a day has no `YYYY-MM-DD` spelling, so the
6+
// storage rule's text for it (`10000-01-01`, `-1-01-01`) orders as no day does.
7+
// Measured over REST before this, the number for 10000-01-01 counted `$gt` 6 /
8+
// `$lt` 1 on driver-memory and SQLite (the day's answer is 0 / 7) and the
9+
// number for -1-01-01 answered 500 on PostgreSQL, while the ISO string of either
10+
// instant was already refused `INVALID_FILTER` / 400. The door that calls this
11+
// predicate (`@objectstack/objectql`, `temporal-comparand-door.ts`) now refuses
12+
// the number and the `Date` too; its own suite pins the envelope.
13+
14+
import { describe, it, expect } from 'vitest';
15+
import { isUninterpretableTemporalComparand } from './temporal-comparand.js';
16+
import { temporalStorageForm } from './temporal-storage-form.js';
17+
18+
const at = (iso: string) => Date.parse(iso);
19+
20+
/** Finite numbers whose UTC calendar day falls outside the four-digit years. */
21+
const OUT_OF_RANGE: ReadonlyArray<readonly [string, number]> = [
22+
['the first day of year 10000', 253402300800000],
23+
['year -1 (the card\'s number)', -62198755200000],
24+
['the last millisecond of year -1', at('-000001-12-31T23:59:59.999Z')],
25+
['the Date range maximum', 8.64e15],
26+
['the Date range minimum', -8.64e15],
27+
];
28+
29+
/** Numbers past the `Date` range — an instant with a year past ±271821. */
30+
const PAST_THE_DATE_RANGE: ReadonlyArray<readonly [string, number]> = [
31+
['one past the maximum', 8.64e15 + 1],
32+
['one before the minimum', -8.64e15 - 1],
33+
['9e15', 9e15],
34+
['MAX_SAFE_INTEGER', Number.MAX_SAFE_INTEGER],
35+
];
36+
37+
/** Finite numbers whose UTC calendar day has a four-digit year — read as before. */
38+
const IN_RANGE: ReadonlyArray<readonly [string, number]> = [
39+
['the first millisecond of year 0', at('0000-01-01T00:00:00.000Z')],
40+
['0999-06-15', -30627504000000],
41+
['1000-01-01', at('1000-01-01T00:00:00.000Z')],
42+
['2026-02-01T10:00Z', 1769940000000],
43+
['the epoch', 0],
44+
['the last millisecond of year 9999', at('9999-12-31T23:59:59.999Z')],
45+
];
46+
47+
describe('[#20240] isUninterpretableTemporalComparand — a date column\'s number or Date outside the four-digit years', () => {
48+
it('refuses a number and the Date of the same value, for a year below 0 or above 9999', () => {
49+
for (const [name, ms] of OUT_OF_RANGE) {
50+
expect(isUninterpretableTemporalComparand('date', ms), `number, ${name}`).toBe(true);
51+
expect(isUninterpretableTemporalComparand('date', new Date(ms)), `Date, ${name}`).toBe(true);
52+
}
53+
});
54+
55+
it('refuses a finite number past the Date range — its year is past ±271821', () => {
56+
for (const [name, ms] of PAST_THE_DATE_RANGE) {
57+
expect(isUninterpretableTemporalComparand('date', ms), name).toBe(true);
58+
}
59+
});
60+
61+
it('reads every number and Date whose year has four digits, as before — the discriminating half', () => {
62+
for (const [name, ms] of IN_RANGE) {
63+
expect(isUninterpretableTemporalComparand('date', ms), `number, ${name}`).toBe(false);
64+
expect(isUninterpretableTemporalComparand('date', new Date(ms)), `Date, ${name}`).toBe(false);
65+
}
66+
});
67+
68+
it('agrees with the rule: a finite number is refused exactly when the rule cannot spell it YYYY-MM-DD', () => {
69+
for (const [name, ms] of [...OUT_OF_RANGE, ...PAST_THE_DATE_RANGE, ...IN_RANGE]) {
70+
const spelled = temporalStorageForm(ms, 'date');
71+
const isDay = typeof spelled === 'string' && /^\d{4}-\d{2}-\d{2}$/.test(spelled);
72+
expect(isUninterpretableTemporalComparand('date', ms), name).toBe(!isDay);
73+
}
74+
});
75+
76+
it('joins the verdict the ISO string of the same instant already had', () => {
77+
// The string arm refused these before this card; the number and the Date
78+
// were the two spellings of the same instant it let through.
79+
expect(isUninterpretableTemporalComparand('date', '+010000-01-01T00:00:00.000Z')).toBe(true);
80+
expect(isUninterpretableTemporalComparand('date', '-000001-01-01T00:00:00.000Z')).toBe(true);
81+
expect(isUninterpretableTemporalComparand('date', '0999-06-15T00:00:00.000Z')).toBe(false);
82+
expect(isUninterpretableTemporalComparand('date', '0999-06-15')).toBe(false);
83+
});
84+
85+
it('leaves the datetime and time rules alone — they read an instant', () => {
86+
for (const kind of ['datetime', 'time'] as const) {
87+
for (const [name, ms] of [...OUT_OF_RANGE, ...PAST_THE_DATE_RANGE, ...IN_RANGE]) {
88+
expect(isUninterpretableTemporalComparand(kind, ms), `${kind}, number, ${name}`).toBe(false);
89+
expect(isUninterpretableTemporalComparand(kind, new Date(ms)), `${kind}, Date, ${name}`).toBe(false);
90+
}
91+
}
92+
});
93+
94+
it('does not judge NaN, ±Infinity or an Invalid Date — they name no instant and no year', () => {
95+
for (const value of [Number.NaN, Number.POSITIVE_INFINITY, Number.NEGATIVE_INFINITY, new Date(Number.NaN)]) {
96+
expect(isUninterpretableTemporalComparand('date', value), String(value)).toBe(false);
97+
}
98+
});
99+
100+
it('judges nothing else that is not a string, as before', () => {
101+
for (const value of [null, undefined, true, 1769940000000n, Object(1769940000000), {}, ['2026-02-01']]) {
102+
expect(isUninterpretableTemporalComparand('date', value), String(value)).toBe(false);
103+
}
104+
});
105+
});

‎packages/core/src/utils/temporal-comparand.ts‎

Lines changed: 54 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -44,9 +44,19 @@
4444
* if the door let it through — `temporalStorageForm` (`temporal-storage-form.ts`,
4545
* the one rule both drivers read since #20176; before it, a copy in each).
4646
* That function is total on purpose: an input it cannot interpret is returned
47-
* UNCHANGED rather than becoming an invented instant. So "the driver would
48-
* return it unchanged" IS the definition of uninterpretable, and defining it
49-
* any other way would refuse comparands that work today.
47+
* UNCHANGED rather than becoming an invented instant. So, for a STRING, "the
48+
* driver would return it unchanged" IS the definition of uninterpretable, and
49+
* defining it any other way would refuse comparands that work today.
50+
*
51+
* [#20240] One non-string class is uninterpretable by the same test's other
52+
* half — the rule cannot put it in the column's form. On a `date` column a
53+
* finite number or a `Date` becomes its UTC calendar day, and a year below 0 or
54+
* above 9999 has no `YYYY-MM-DD` spelling: the rule keeps writing
55+
* `10000-01-01` / `-1-01-01` for the write and read paths, but as a comparand
56+
* that text orders as no day does (`10000-01-01` sorts below `2026-…`), and
57+
* PostgreSQL refuses `-1-01-01` outright. So such a comparand is judged
58+
* uninterpretable here, exactly as an unparseable string is, and refused by
59+
* the same doors.
5060
*
5161
* That is why this is not `utcInstantMs` (`@objectstack/spec/data`), which is
5262
* the stricter canonical reader: it rejects a bare epoch-millisecond string and
@@ -56,8 +66,18 @@
5666
*
5767
* ## Two things it deliberately does NOT judge
5868
*
59-
* - **Non-string comparands.** A number is epoch milliseconds, a `Date` is an
60-
* instant, `null` is a null test. The refusal scopes to strings by ruling.
69+
* - **Non-string comparands, save the one `date` class above.** A number is
70+
* epoch milliseconds, a `Date` is an instant, `null` is a null test, and the
71+
* `datetime` and `time` rules read every finite one; so does the `date` rule
72+
* for a year from 0 to 9999. The #8690 refusal was scoped to strings by that
73+
* card's own ruling — its triage queued "a non-interpretable bare string", and
74+
* the maintainer ruling scoped its two options "to non-empty strings" so the
75+
* empty-string cell stayed its own card. That scoped that change; it is not a
76+
* standing rule that a non-string is never refused. [#20240] extends the
77+
* refusal to the `date` class above by the triage direction on that card.
78+
* `NaN`, ±Infinity and an Invalid Date name no instant and no year, so they
79+
* are not that class and stay unjudged, as before; no JSON body can carry
80+
* one (JSON spells them `null`).
6181
* - **The EMPTY string.** Measured, `$gte ""` binds as `''` and every canonical
6282
* UTC text sorts at or above it, so it returns every non-null row — a third
6383
* behaviour again, and one the maintainer ruled stays its own card: "B and C
@@ -130,13 +150,35 @@ function readsAsWallClock(s: string): boolean {
130150
return Number(m[1]) <= 23 && Number(m[2]) <= 59 && Number(m[3] ?? '0') <= 59;
131151
}
132152

153+
/**
154+
* [#20240] `temporalStorageForm`'s `date` reading of a NUMBER or a `Date` lands
155+
* outside the four-digit years `YYYY-MM-DD` can spell.
156+
*
157+
* Both name an instant, and the rule takes that instant's UTC calendar day.
158+
* Its year must fall from 0 to 9999: `0999-06-15` is a day, `10000-01-01` and
159+
* `-1-01-01` are not. A finite number past ±8.64e15 names an instant the
160+
* `Date` type cannot hold at all — a year past ±271821 — so it is outside the
161+
* range too, and the rule hands it back unchanged. `NaN`, ±Infinity and an
162+
* Invalid Date name no instant and no year, and are not judged here.
163+
*/
164+
function isOutsideCalendarDayYears(value: number | Date): boolean {
165+
if (typeof value === 'number' && !Number.isFinite(value)) return false;
166+
const instant = typeof value === 'number' ? new Date(value) : value;
167+
if (Number.isNaN(instant.getTime())) return typeof value === 'number';
168+
const year = instant.getUTCFullYear();
169+
return year < 0 || year > 9999;
170+
}
171+
133172
/**
134173
* Is `value` a comparand that a `kind` column's storage rule cannot read?
135174
*
136-
* `true` ONLY for a non-empty, non-placeholder STRING that the kind's rule
137-
* would hand back unchanged. Everything else — a number, a `Date`, `null`, a
138-
* `{ $field }` reference, filter structure, the empty string, a `{token}` —
139-
* answers `false`, each for a reason recorded in the module note or below.
175+
* `true` for a non-empty, non-placeholder STRING that the kind's rule would
176+
* hand back unchanged, and — on a `date` column only — for a finite number or
177+
* a `Date` whose UTC calendar day falls in a year below 0 or above 9999
178+
* ({@link isOutsideCalendarDayYears}). Everything else — any other number or
179+
* `Date`, `null`, a `{ $field }` reference, filter structure, the empty
180+
* string, a `{token}` — answers `false`, each for a reason recorded in the
181+
* module note or below.
140182
*
141183
* A `{placeholder}` is stepped around rather than judged because it is another
142184
* layer's vocabulary and that layer already refuses the unknown ones loudly
@@ -149,6 +191,9 @@ export function isUninterpretableTemporalComparand(
149191
kind: TemporalComparandKind,
150192
value: unknown,
151193
): boolean {
194+
if (kind === 'date' && (typeof value === 'number' || value instanceof Date)) {
195+
return isOutsideCalendarDayYears(value);
196+
}
152197
if (typeof value !== 'string') return false;
153198
const s = value.trim();
154199
// The empty-string cell is its own card — see the module note.

0 commit comments

Comments
 (0)