Repository navigation
Commit 615c468
fix(core): read an epoch-ms number on a
Fixes #20203
Clause-②: no
An epoch-millisecond number compared against a `date` field is now read
as the UTC calendar day of its instant. The change is in
`temporalStorageForm` (`@objectstack/core`), the one storage rule that
both drivers and the engine's per-aggregation `filter` / `having` call.
One REST request used to get three answers: InMemoryDriver 0 rows,
SQLite 6 of 6, and PostgreSQL `DATABASE_ERROR` / 500. All three now give
the date's own count, the count a `Date` of the same value already got.
## What changed
- `packages/core/src/utils/temporal-storage-form.ts`, the `date` arm: a
finite number becomes `new Date(value)` and goes through the same
UTC-calendar-day conversion a `Date` takes. `NaN`, ±Infinity and a
number outside the `Date` range (past ±8.64e15) come back unchanged. The
`datetime` and `time` arms, and every non-number `date` input, are
untouched; they measured byte-identical (below).
- No driver source changes. `driver-sql` (`toDateOnly`,
`temporalFilterValue`) and `driver-memory` (`coerceTemporalValue`)
already call this rule, so there is no third copy.
- Tests (below), and `.changeset/20203-epoch-ms-date-comparand.md`
(patch: core, driver-sql, driver-memory, objectql).
- `.changeset/20176-aggregation-temporal-storage-rule.md`: a one-clause
deliberate correction (see Deviations).
## Stop valve (ADR-0053): nothing requires a calendar-day string, so the
rule reads the number
Triage's valve: refuse instead if ADR-0053 states that a `date`
comparand must be a calendar-day string. Every sentence found:
- ADR-0053, TL;DR: "`date` = a timezone-naive calendar day, represented
as a `YYYY-MM-DD` string end-to-end (storage, query, CEL). It is
**never** converted to an instant and **never** timezone-shifted." This
describes how a date is represented. It does not limit which comparand
inputs are accepted.
- ADR-0053, Phase 1 decision 1: "A `Date`, a full ISO string, or a
`YYYY-MM-DD` string all collapse to the calendar day (UTC calendar day
for a `Date`, matching the existing filter coercion)." Inputs that are
not a day string are collapsed to one. They are not refused.
- ADR-0053, D-C1, the sibling `time` rule: "A `Date` / epoch-ms /
full-timestamp value folds to its **UTC** time-of-day". Epoch ms is
already read on a temporal column.
- `packages/spec/src/data/filter.zod.ts`, the ordering-comparand
description: "Comparand is a number, a Date, a string, or a { $field }
reference".
- `packages/core/src/utils/temporal-comparand.ts` (the door): "A number
is epoch milliseconds, a `Date` is an instant, `null` is a null test.
The refusal scopes to strings by ruling."
The read direction applies. Separately, the engine and REST WRITE doors
refuse a number written to a `date` field (`VALIDATION_FAILED`, "must be
a valid date (ISO-8601)"). That is the stored-value contract, not a
comparand sentence in ADR-0053. It is unchanged here and listed under
Acceptance notes.
## The rows
Real `InMemoryDriver`, real `SqlDriver` on better-sqlite3 `:memory:`,
and live PostgreSQL 16.13 (a private server, server zone
`Asia/Shanghai`), schema-synced, over six rows. Each cell was measured
through `engine.find` and through `POST /api/v1/data/:object/query`
(JSON round-tripped); the two doors agreed in every cell. N =
`1769940000000` = 2026-02-01T10:00:00.000Z (a time of day, not a UTC
midnight). The base is `3875ae6773`. The head readings were taken at
`ba5c0de774` (the fix), and the new pins re-confirm every row at
`1c8c5ce756`.
| `date` field, operator | memory base → head | SQLite base → head |
PostgreSQL base → head | `Date` twin at head | `datetime` control (base
= head, all three) |
|:--|:--|:--|:--|:--|:--|
| `$gt` N | 0 → 1 | 6 → 1 | 500 → 1 | 1 | 3 |
| `$lt` N | 0 → 3 | 0 → 3 | 500 → 3 | 3 | 2 |
| `$eq` N | 0 → 2 | 0 → 2 | 500 → 2 | 2 | 1 |
| `$in` [N, Jan 10 15:00Z] | 0 → 3 | 0 → 3 | 500 → 3 | 3 | 1 |
| `$between` [Jan 2 23:00Z, N] | 0 → 5 | 0 → 5 | 500 → 5 | 5 | 1 |
| `$gte` N | 0 → 3 | 6 → 3 | 500 → 3 | 3 | 4 |
| `$lte` N | 0 → 5 | 0 → 5 | 500 → 5 | 5 | 3 |
| `$ne` N | 6 → 4 | 6 → 4 | 500 → 4 | 4 | 5 |
| `$nin` [N, Jan 10 15:00Z] | 6 → 3 | 6 → 3 | 500 → 3 | 3 | 5 |
| implicit equality | 0 → 2 | 0 → 2 | 500 → 2 | 2 | 1 |
| `$eq` on a UTC midnight | 0 → 2 | 0 → 2 | 500 → 2 | 2 | 0 |
- **Empty table:** memory and SQLite answered 0 before and after.
PostgreSQL answered 500 on every operator before (the bind is refused
whether or not rows exist) and 0 after.
- **PostgreSQL's 500, root cause.** `toDateOnly` handed the number
through unchanged. The statement was `select * from "ledger_order" where
"placed_on" > $1` with bindings `[1769940000000]`, and node-pg sent that
as text. The PostgreSQL server's own date input parser refused it:
SQLSTATE `22008`, routine `DateTimeParseError`, "date/time field value
out of range". driver-sql's read-error mapper wrapped that as an
unattributed `DATABASE_ERROR`, and REST answered 500. The raw `knex.raw`
bind reproduced the same error.
- **H3, which day.** The number reads as the UTC day of `new
Date(value)`, which is the `toDateOnly` convention. The `Date` twin
column above equals the number column on every cell. It was measured
through the engine on all three drivers, and on the driver door for all
11 operators in the new pins. Rule-level edge cases: a negative number
is a day before 1970 (`-1` → `1969-12-31`); a fraction truncates toward
zero, as the `Date` constructor does (`-0.5` → `1970-01-01`); `-0` →
`1970-01-01`; the `Date` range edges read `275760-09-13` /
`-271821-04-20`. `NaN`, ±Infinity and values past the range are
unchanged.
## H4, no collateral
- **The rule, over 34 shapes × 3 kinds (102 cells):** 12 moved. All 12
are `date` × a finite number inside the `Date` range. The other 90 are
byte-identical base → head: every `datetime` and `time` cell, and on
`date` every `Date`, Invalid Date, `NaN`, ±Infinity, past-range number,
bigint, boxed `Number`, epoch-ms string, ISO / bare-day / zone-naive /
offset string, wall clock, junk, empty string, `null`, `undefined`,
boolean, object and array.
- **End to end, 466 cells** (`where`, per-aggregation `filter` and
`having` through both doors, write and read paths, the rule sweep): 113
moved, and none of them is outside "a number on a `date` field".
- **`where` = per-aggregation `filter` at head,** on all 132 keys (3
drivers × populated / empty × `date` / `datetime` × 11 operators),
through both doors. At base the per-aggregation `filter` gave yet
another answer: 0 on every operator except `$between`, `$ne` and `$nin`
(6), on all three drivers.
- **`having`, both doors (native and rows):** only `max(date)` moved.
`$gt` N: no group → c2. `$eq` N: no group → c3. `$in` [N, Jan 10]: no
group → c1, c3. The `min(datetime)` `$gte` N control kept c2, c3 before
and after.
- **Write paths:** `engine.insert` and REST create of a number on a
`date` field answer `VALIDATION_FAILED` / 400 at base and at head.
Direct `driver.create` / `driver.update` of a number on a `date` field:
memory and SQLite stored the number, PostgreSQL refused (`22008`); all
three now store `2026-02-01`. The same number written to a `datetime` or
`time` field is unchanged (`2026-02-01T10:00:00.000Z` / `10:00:00`).
- **Read path (SQLite, a raw INTEGER already in a `date` column):**
`find`, `findOne`, a `groupBy` key and `distinct` presented
`1769940000000` before and now present `2026-02-01`. This is ADR-0053
Phase 1's read-repair reading. `max` over a mixed group is unchanged,
and so is a `where` day comparand against that raw row (see Acceptance
notes).
## Tests
New or changed:
- `packages/core/src/utils/temporal-storage-form.test.ts`: the old pin
"epoch milliseconds are not a calendar day, returned as they came"
pinned exactly the branch this changes, so it is replaced whole. 11
number cases, the number = its `Date` agreement, the number's day = the
`datetime` rule's instant's day, and 8 unchanged inputs.
-
`packages/drivers/driver-sql/src/sql-driver-20203-epoch-ms-date-comparand.test.ts`:
a live-dialect matrix. SQLite always runs; PostgreSQL and MySQL run when
provisioned, as in CI's Temporal Conformance job. It covers 11 operators
× (number, `Date` twin, `datetime` control), an empty table, and the
create / update write path.
-
`packages/drivers/driver-memory/src/memory-20203-epoch-ms-date-comparand.test.ts`:
the same rows at this driver's door.
- `packages/rest/src/data-query-epoch-ms-date-comparand.test.ts`: engine
and REST over `SqlDriver`, at `where` (7 operators, with the `datetime`
control), a per-aggregation `filter` equal to its `where` twin, and
`having` on `max(date)`, populated and empty.
Whole suites, after merging `main` (`4df101c383`) and rebuilding the
closure. Every package ran with `vitest run` and no `--project`, so both
`local` and `repo` projects ran where a package declares them:
- `@objectstack/core`: 57 files / 1488 passed.
- `@objectstack/driver-memory`: 55 / 1351 passed.
- `@objectstack/driver-sql` with live PostgreSQL attached
(`TZ=America/New_York`, server `Asia/Shanghai`): 197 files passed, 3
skipped / 3852 tests passed, 87 skipped. The package's own reporter
says: sqlite RAN, live postgres RAN, live mysql NOT RUN.
- `@objectstack/objectql`: 319 / 5767 passed.
- `@objectstack/rest` (includes every query-route test): 201 files /
3582 passed, 1 skipped.
Typecheck: `core`, `driver-memory`, `driver-sql` and `rest` exit 0. Each
new or changed test file is in a compiled program (`--listFiles`:
`tsconfig.json` for the two drivers, `tsconfig.test.json` for core and
rest). The test-typecheck ledgers are unchanged (core 4 held, rest 0).
## Reverse verification
The fix was committed first. `ablation-replace` (wrap mode, restore trap
on the absolute path) replaced the one number line with `const instant =
value;`: anchor 1 → 0, blob `61a6c0861a` → `2a9bd5d997`. Core was
rebuilt, and `ablation-dist-preflight --absent` passed. Pins in the
mutated state:
- core 13 failed / 59 passed (the 11 number cases and the 2 agreement
tests);
- driver-memory 12 failed / 11 passed (the 11 greens are the
empty-object rows, 0 either way);
- driver-sql on SQLite + live PostgreSQL 35 failed / 11 passed / 1
skipped (the 11 greens are SQLite's empty tables; PostgreSQL's empty
tables went red because the 500 came back);
- REST 17 failed / 1 passed (the `min(datetime)` control).
The tool restored the file: blob `61a6c0861a` = HEAD, `git diff HEAD`
empty, whole-tree `git status --porcelain` empty. Core was rebuilt, and
the preflight found the marker present again. Pins: core 72, memory 23,
driver-sql 46 + 1 (MySQL declared unprovisioned), REST 18, all passed.
## Gates
`dispatch-gates --commands --repo objectstack-ai/objectstack` at
`1c8c5ce756` derives 62 commands (the dispatch clue's 47 plus the
changeset families this diff adds, and more). Every command was run at
`1c8c5ce756`: 59 exit 0, 1 exit 1, 2 exit 3.
- `check-empty-changeset --base origin/main` **exit 1, on purpose.**
This is the deliberate correction below. It needs a person's
confirmation, and the file must not be restored from the base.
- `pnpm check:dual-build-cjs-loads` and `pnpm check:type-check-debt`:
**NOT MEASURED**, `PREREQUISITE NOT MET`. Both read every package's
built output, which needs the whole `./packages/*` build; CI builds that
before these steps.
`--ran` reconciles: 62 derived, 60 run, 2 NOT-MEASURED, 0 unrun. Also
run:
- the three roster gates whose roster sits under `packages/`
(`check:authz-resolver`, `check:error-code-casing`,
`check:filter-alias-parity`): exit 0 each;
- `check-issue-citations --base 4df101c`: exit 0;
- a control-byte scan of the 7 changed files: none found.
Lint, a declared narrowing: `eslint --no-inline-config --format json` on
the 5 changed `.ts` files at `1c8c5ce756` reports 5 files, 0 errors / 0
warnings / 0 fatal, and `--print-config` resolves a config for each.
`eslint.config.mjs` enables no type-aware linting (no
`parserOptions.project`), so no untouched file's verdict can move.
## Deviations
- **`.changeset/20176-aggregation-temporal-storage-rule.md`, a
one-clause DELIBERATE CORRECTION.** The pending note says "The copies
agreed on every shape measured when they were lifted, so no `where`,
write or read answer of either driver changes." This PR changes the
number-on-`date` answers in the same release, so that sentence would
ship false. It now reads "…, so the lift itself changes no `where`,
write or read answer of either driver (#20203, in the same release, then
reads an epoch-millisecond number on a `date` field as its UTC calendar
day)." Nothing else in that note changed. `check-empty-changeset` is red
for exactly this; please confirm it here.
- **Where each driver is pinned.** `packages/rest` has no
`driver-memory` dependency, and `driver-memory` cannot reach the engine,
so memory's engine and REST cells are measured (the table above) but not
committed as pins; its `where` is pinned at its own driver door.
PostgreSQL is pinned at the driver door, in driver-sql's live matrix,
and its engine and REST cells are measured. This follows PR #20202's
precedent. The claim's file surface ("tests in core, the two drivers and
REST") has no package that holds all three.
## Acceptance notes
- **PostgreSQL after the change: the 500 is gone.** On a private live
PostgreSQL 16.13 (server zone `Asia/Shanghai`), REST `where: {
placed_on: { $gt: 1769940000000 } }` answers 200 with 1 row (`o3`). All
11 operators give the date's count on a populated table and 0 on an
empty one, through the engine and REST, and none answers
`DATABASE_ERROR`. The live cell PR #20202's review used was not
reachable from this container.
- MySQL: NOT MEASURED here (no server). The new driver-sql file runs in
CI's Temporal Conformance MySQL cell.
- `SqlDriver.toDateOnly`'s docstring still says "Anything else (and
`null`/`undefined`) passes through unchanged", and a number no longer
does. That is driver source outside the claim, so it was not edited.
- A number already stored in a SQLite `date` column is now presented as
its day on read. It still does not match a day comparand in `where`,
because the column side has no legacy repair for `date` (one exists for
`datetime`). Only a raw write can put such a row there. This is
unchanged by this PR.
- The write doors refuse a number on a `date` field, while the read
doors now read one. The two have different contracts (stored value vs
comparand), and neither moved beyond what is listed above.
- `driver-turso` inherits `temporalFilterValue` from `SqlDriver`, and
service-analytics binds through the driver's `temporalFilterValue`, so
both follow the rule. Neither was measured separately.
## Out-of-scope findings (reported for the seat; nothing filed from
here)
1. **class (a).** `driver-mongodb`'s `mongodb-temporal.ts`
`storageDateValue`, its own copy of the `date` rule, still returns a
finite number unchanged. Probe: `storageDateValue(1769940000000)` →
`1769940000000`, where core's `temporalStorageForm(…, 'date')` now
answers `2026-02-01`. A `Date` and an ISO string agree on both. On a
live Mongo, a number against the `YYYY-MM-DD` text meets BSON type
bracketing, which is what gave driver-memory its 0: NOT MEASURED, no
server here. Dedupe words: `driver-mongodb epoch ms number date field` ·
`mongodb storageDateValue number unchanged` · `mongodb-temporal date
rule copy`.
2. **class (a).** The `date` rule's `Date` conversion (`${y}-${m}-${d}`
from `getUTCFullYear`) spells a year outside 1000..9999 unpadded
(`999-06-15`, `10000-01-01`, `-1-01-01`), so the text no longer orders
as a `YYYY-MM-DD` day. Measured at head with the same fixture: for year
999, memory and SQLite answer `$gt` 0 and `$lt` 6 (PostgreSQL 6 / 0,
correct); for year 10000, memory and SQLite answer 6 / 0 (PostgreSQL 0 /
6, correct); for year -1, PostgreSQL answers 500. A `Date` does the same
in-process. This PR routes a number through that one conversion, so a
REST number now reaches it too. Fixing it moves `Date` cells, which this
card's ruling keeps byte-identical, so it is left to its own card.
Dedupe words: `date rule year below 1000 unpadded` · `calendar day year
outside 4 digits text order` · `toDateOnly getUTCFullYear padding`.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx)_
---------
Co-authored-by: Claude <noreply@anthropic.com>date field as its UTC calendar day, so driver-memory, SQLite and PostgreSQL give one answer (#20224)1 parent 93cfc3f commit 615c468
7 files changed
Lines changed: 499 additions & 9 deletions
File tree
- .changeset
- packages
- core/src/utils
- drivers
- driver-memory/src
- driver-sql/src
- rest/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
31 | | - | |
| 31 | + | |
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
42 | | - | |
43 | | - | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
44 | 72 | | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
45 | 94 | | |
46 | 95 | | |
47 | 96 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
73 | | - | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
74 | 76 | | |
75 | 77 | | |
76 | 78 | | |
| |||
114 | 116 | | |
115 | 117 | | |
116 | 118 | | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
122 | 130 | | |
123 | 131 | | |
124 | 132 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
0 commit comments