Repository navigation
fix(driver-sql): aggregate count / count_distinct / sum / avg answer numbers on PostgreSQL and MySQL - #20372
Conversation
…s on every dialect node-postgres hands bigint (count) and numeric (sum, avg) back as strings and mysql2 does the same for DECIMAL, so the native aggregate path answered "2" where SQLite and the engine's rows path answer 2. Key the 'number' read presentation on the aggregate function the query asked for, with one precision policy: a JS double, the loss beyond a double's precision declared. Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
… dialect cell Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
…g across both paths Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
… with the precision policy Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
…ialect The docblock still said the numeric coercion is SQLite-only, which the numeric-representation change had already made false and the aggregate presentation makes false a second time. Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 1 package(s): 13 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 2 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 11 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 03e73ce74f8534883ed7fbfd940b711d1eafa48c && git checkout 03e73ce74f8534883ed7fbfd940b711d1eafa48c
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin a88a1bb399ea1ba6d717b8f6fefa0b13ec3e66a8 f3b9e1390c0bcea6aa49033485960a41647a48db && git checkout -B drift-repro a88a1bb399ea1ba6d717b8f6fefa0b13ec3e66a8 && git merge --no-ff f3b9e1390c0bcea6aa49033485960a41647a48db
node scripts/docs-audit/affected-docs.mjs --json a88a1bb399ea1ba6d717b8f6fefa0b13ec3e66a8
|
Contract reviewServed-tier: Inputs read, and nothing else: card #20335 (body and all four comments — triage 5861208644, claim 5861869486, os-dev-report 5862967111, takeover claim 5863887111); PR #20372 body and file list (4 files, equal to ① Derived judgments
② Semver level
③ Boundary flags
Dev flags (os-dev-report 5862967111,
Implemented-by: VERDICT: PASS |
…o fields of different comparison classes when it is authored (objectstack-ai#20347) (objectstack-ai#20403) Fixes objectstack-ai#20347 Clause-②: yes (narrowing) The spec half of the objectstack-ai#20347 triage split (`5862073027`), dispatched on claim `5863797885`. Base `eee09742`, head `bef47d1d`. The engine half is objectstack-ai#20355, which stays open and reads the export this PR adds. The changeset declares `Clause-②: yes (narrowing)`, BREAKING, `minor` on both `@objectstack/spec` (new exports, a widening) and `@objectstack/lint` (a new authoring refusal, a narrowing). ## What changes - **One classification, exported once** (`packages/spec/src/data/filter-cross-field-comparison-class.ts`, re-exported from `@objectstack/spec/data`, beside `filter-text-operator-declared-type.ts`). - Six classes (`CROSS_FIELD_COMPARISON_CLASSES`: `numeric`, `text`, `boolean`, `date`, `datetime`, `time`) and three families with none (`CROSS_FIELD_NO_CLASS_REASONS`: `list-or-object`, `file`, `formula`). - `CROSS_FIELD_COMPARISON_TYPE_CLASSES` classifies every `FieldType` member exactly once, by reference to the existing `field-value.zod.ts` sets. Nothing is re-listed. - Two pure verdicts. `crossFieldColumnVerdict(field)` answers one declared column; `multiple: true` on a multi-capable type holds a list. `crossFieldComparisonVerdict(left, right)` answers two: `comparable`, `cross-class`, `no-class`, or `unjudged` for a type outside `FieldType`. - It is lifted case for case from driver-sql's module-private `crossFieldComparisonClass` (the objectstack-ai#5222 boundary). `sql-driver.ts` is untouched: objectstack-ai#20355 rewires it, and PR objectstack-ai#20372 holds that file. - **Parity with driver-sql, run against both** (`packages/drivers/driver-sql/src/sql-driver-20347-cross-field-class-parity.test.ts`). One object declares every `FieldType` member (49), plus the 6 multi-capable members flagged `multiple: true`. Every ordered pair (55 × 55 = 3,025) is compiled as `{ a: { $eq: { $field: b } } }` on a real `:memory:` SQLite driver. The driver's admit or refuse must equal `crossFieldComparisonVerdict(a, b) === 'comparable'` on every pair. A refusal counts only in the cross-field boundary's own withheld `INVALID_FILTER` / 400 form (`withheldFilterDiagnosticOf` non-null), never by prose. - **The authoring door** (`packages/lint`). - `validateRlsPredicateEnforceability` gains a cross-class arm. `crossClassComparisons` reads the lowered filter's `{ $field }` sites against the declared field map. It reports `rls-predicate-unenforceable` for every comparison whose two columns are not `comparable`: `==`, `!=`, `>`, `>=`, `<`, `<=`, either side, under `!` too. - It covers `using` and `check` on every operation. - `validateSharingRuleEnforceability` reads the same function and reports `sharing-rule-unlowerable-condition` on a sharing rule's lowered `condition`. - A comparison against a list or an object stays the existing objectstack-ai#19886 arm's finding, so no comparison is reported twice. The new arm runs ahead of the engine-judge pass, like the list arm: one defect, one finding. - The finding names each comparison, each column's declared type and class (or why it has none), and the clause's measured run-time consequence. The hint lists every class with the declared types it holds, derived from the spec table. ## Measured before (lint as on `main`), then after Real `os validate` (`packages/cli/bin/run-dev.js validate` on a probe stack), plus the real plugin-security + ObjectQL on driver-sql (`better-sqlite3` `:memory:`, one RLS policy on a `text` / `number` / `image` / `formula` object). | predicate | `os validate` before | `find` (`using`) | insert (`check`) | insert (`using` as check) | by-id update / delete (`using`) | `os validate` after | |:--|:--|:--|:--|:--|:--|:--| | `record.status != record.amount` (text vs number) | valid, exit 0 | `INVALID_FILTER` / 400 | admitted, stored | admitted, stored | 403 / 403 | `rls-predicate-unenforceable`, exit 1 | | `record.status != record.photo` (text vs image) | valid, exit 0 | 400 | admitted, stored | admitted, stored | 403 / 403 | refused, exit 1 | | `record.status != record.is_open` (text vs formula; the card's NOT MEASURED cell) | valid, exit 0 | 400 | admitted, stored | admitted, stored | 403 / 403 | refused, exit 1 | | `record.amount > record.status` (number vs text) | — | 400 | 403 (JS `5 > 'open'` is false) | 403 | 403 / 403 | refused (lint unit and door pins) | | control `record.status != record.note` (text vs text) | valid, exit 0 | rows `[r1]` | admitted | admitted | updated / deleted | valid, exit 0 | The `check` rows on `insert` read the same at `os validate`: valid before, `rls-predicate-unenforceable` after. Sharing-rule conditions, measured at the real `os validate`, first with the arm ablated (the before-state) and then restored: `record.status != record.amount` and `record.status != record.photo` went from valid (exit 0) to `sharing-rule-unlowerable-condition` (exit 1). The control `record.status != record.note` stayed valid. At run time the seeded rule's criteria query meets the same driver-sql refusal the list-holding class meets (objectstack-ai#20375 measured that path). The write-check answer is whatever JavaScript's comparison of the two raw values gives, so the permissive side of the policy is the write. That half is objectstack-ai#20355's. ## Census (expected 0): 0 A script over `git ls-files examples packages` (tests, fixtures, docs, generated bundles excluded; 3,140 files at `bef47d1d`) extracts every `using` / `check` / `condition` string literal: 163. It lowers each through the real `compileCelToFilter` (RLS through `sqlPredicateToCel` first); 105 lower. It then lists every `{ $field }` comparison: 2. - `examples/app-showcase/src/data/hooks/index.ts:88`: `record.spent > record.budget`, a hook condition, both `number`. - `packages/lint/scripts/check-doc-formula-expressions.mjs:1396`: `record.a > record.b`, a gate fixture. Neither is an RLS predicate or a sharing-rule condition, and both are same-class. The only programmatic predicate constant is `OWNERSHIP_FLOOR_PREDICATE` (`created_by == current_user.id`), which is not field-to-field. So no shipped policy or sharing condition moves, and nothing re-grades to p1. The cloud repository was not in this session: NOT MEASURED. ## Ablation (one-time proof, committed state `bef47d1d`) Two ablations, both run from the committed state `bef47d1d`, each through `scripts/ablation-replace.mjs`. That tool landed each mutation (anchor count 1 to 0, blob changed) and restored it (the blob equals `HEAD`, and `git diff HEAD` is empty). A shell `trap` re-checked each restore by hash. The direction observed is the normal one: red. 1. **The lint arm.** The guard line in `crossClassComparisons` was replaced with an unconditional `continue`, so the arm reports nothing. `ablation-dist-preflight` found the marker in 4 built `@objectstack/lint` files, so the mutation reached the `dist/` the CLI consumes. - lint unit, the four cross-class and list-holding files: **525 failed / 485 passed** of 1,010. Restored: **1,010 / 1,010 passed**. - CLI integration `rls-policy-authoring-admission.test.ts`: **6 failed / 33 passed**. The 6 are exactly the new REFUSED rows. Restored: **39 / 39 passed**. - Real `os validate`, 9 cells. Ablated: all nine exit 0 with no finding, which is the before-state, sharing cells included. Restored: the 3 RLS `using` cells, the 2 RLS `check` cells and the 2 sharing cells exit 1, each with exactly one finding; both controls exit 0. - On restore, `ablation-dist-preflight --absent` passed its `dist/` reading (the marker is absent from all 14 built files). Its tree reading exited 3 only because two untracked scratch files were present at that moment; both are deleted now. 2. **The driver half of the parity pin.** Temporarily, never committed: in `sql-driver.ts`'s `crossFieldComparisonClass`, `if (type === 'time') return 'time'` was changed to return `'datetime'`. The parity test imports driver source, so no build was needed. Result: **2 failed / 54 passed**. The two are `f_datetime` and `f_time`, naming exactly `f_datetime vs f_time: spec says cross-class, driver admitted` and its mirror. Restored: **56 / 56 passed**, blob equal to `HEAD`. ## Tests (at `bef47d1d`) All at `bef47d1d`, after the last commit, on a shared box. - `@objectstack/spec` - `vitest run --project local src/data`: 103 files, **3,458 passed**, 1 todo. The new classification test contributes 19. - `typecheck` (tsc, scripts and the test layer): exit 0. - `@objectstack/lint` - `pnpm test`: 115 files, **5,314 passed**. - `typecheck` (with the test layer): exit 0. - `@objectstack/driver-sql` - The parity test plus the two existing cross-field suites (`sql-driver-cross-field-reference`, `sql-driver-cross-field-conformance`): **221 passed**, 2 skipped. The parity test alone: 56 passed, one test per probe column (55 × 55 pairs), plus the coverage pin. - `typecheck`: exit 0. - `@objectstack/cli` - `--project integration test/rls-policy-authoring-admission.test.ts`, the only CLI file touched (integration tier): **39 passed**, 9 of them new. - `typecheck`: exit 0. - The unit tier is declared to CI: no CLI source file and no unit-tier file changed. - Real `os validate` over the examples: `app-crm`, `app-multi-package` and `app-todo` exit 0, with 0 `rls-predicate-*` / `sharing-rule-*` findings. `app-showcase` is NOT MEASURED this way: its config imports `@objectstack/connector-mcp`, which is outside this worktree's build closure. Its security files are in the text census above. - Spec generated artifacts: `check:generated` named `api-surface/` and `export-origins/` stale, both additive only. Both were regenerated with their generators, and `check:api-surface` and `check:export-origins` are green. - Gates: `dispatch-gates --ran` accounts for 88 of 88 derived families. 86 exited 0. Two are NOT MEASURED, and CI owns both: - `check:dual-build-cjs-loads` answered PREREQUISITE NOT MET: it needs a full `pnpm build`. - `check:type-check-debt`: its `--re-measure` passed the 400 s local timeout. The kill left `packages/spec/dist` without declarations, so the spec was rebuilt (64 `.d.ts`) before every lint, driver-sql and cli reading above. - The derivation warned that the tree is behind `origin/main` by one family file (`scripts/cross-package-test-inputs.mjs`). `check:cross-package-test-inputs` was run from this tree and is green. ## Decisions - **Formula has no class, whatever its `returnType`.** That is driver-sql's answer: a formula is virtual, with no column to reference. The text-operator door reads `returnType`, but a column-to-column comparison needs a column on both sides. The measured runtime agrees (400 on the read). - **The file family is refused by name.** That is driver-sql's answer too (the ADR-0104 dual-encoding window), so `image == image` is refused as well. - **A type outside `FieldType` is `unjudged`.** A driver's aliases (`integer`, `object`, the absent-type `string` default) stay layered in the driver, as `field-value.zod.ts`'s header says every alias does. objectstack-ai#20355's rewire keeps those aliases above the export. At the door, an out-of-vocabulary type is Zod's to refuse, and the arm reports nothing. - **Registry-injected columns are judged** by the definition the registry provisions. `record.status != record.created_at` is refused (text vs datetime), because the driver sees the same column. `id` has no definition in the graph, so it is not judged. - **Same rule ids as the list arm.** The author's edit is the same kind: rewrite which two columns are compared. - **Two existing pins changed**, one in each objectstack-ai#19886 list-holding test. "A single-valued `file` field is one value" asserted *no finding at all* for `record.status != record.subject` with `subject` a single `file`. driver-sql refuses that comparison (the file family has no class), so the no-finding reading was never the runtime's. Each pin now asserts that the list arm stays silent and the class arm refuses once. `select` / `lookup` / `user` keep the no-finding pin. - **File surface beyond the claim, both required by the dispatch.** The driver-sql parity test: the classification can only be run "against both" there, and it adds no line to `sql-driver.ts`. And `validate-sharing-rule-enforceability.ts` plus its tests: the direction covers sharing conditions, and that rule is where they are judged. ## Acceptance notes - `listHoldingComparisons` still reads `STRUCTURED_JSON_TYPES` + `isMultiValueField` directly. That is the same family as the export's `list-or-object` reason, and the two agree by construction (pinned in the spec test), but it is two spellings. Converging it onto `crossFieldColumnVerdict` is the natural edit for whoever next touches that function (carrier: objectstack-ai#20355 or the next objectstack-ai#19886-family change). Noted, not filed. - The metadata save door for a `sharing_rule` does not run `validateSharingRuleEnforceability`, as objectstack-ai#20375 recorded. The new sharing arm therefore shows at `os validate` / `os build` / `os lint` only, like the list arm. Noted, not filed. - The `check` consequence sentence describes today's write check, which admits by raw comparison. When objectstack-ai#20355 moves the write check onto this classification, that sentence changes in the same change (a code comment at `crossClassConsequence` says so). --- _Generated by [Claude Code](https://claude.ai/code/session_01QcAS3qiYYZNezaxZxaUdMV)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…QL, as on SQLite and the rows path (objectstack-ai#20486) Fixes objectstack-ai#20387 Clause-②: no ## What changed `SqlDriver.aggregate` (`packages/drivers/driver-sql/src/sql-driver.ts`) now makes PostgreSQL and MySQL accumulate `sum` / `avg` in double. That is the arithmetic SQLite and the engine's rows path (`objectql`'s `in-memory-aggregation.ts`) already use. This is route (a) of triage 5865067775, under the one-double policy that PR objectstack-ai#20372's changeset stated for the answer's type. - `AGGREGATE_ACCUMULATION` is a `Record` over the spec's `AggregationFunction`, a sibling of `AGGREGATE_ANSWER_KIND`. Its entries are `avg: 'double'`, `sum: 'double-over-fractional'`, and `count` / `count_distinct` / `min` / `max`: `'as-stored'`. A function added to the enum without an entry fails `tsc`. - `fractionalNumericFields` is a registry filled beside `numericFields` in both fills and aliased for shards. It holds the declared columns whose type stores fractions: `numericColumnFor(type).kind === 'exact'` (`number`, `currency`, `percent`, `slider`, `progress`, `summary`) and the driver's `float` alias. - In `aggregate()`, on PostgreSQL and MySQL only, two cases take a double operand: `avg` over a declared numeric or boolean column, and `sum` over a fractional column. The operand is `cast(cast(x as text) as double precision)` on PostgreSQL and `cast(cast(x as char) as double)` on MySQL. The column stays one `??` binding. - A `sum` over an integer-valued column (`rating`, the `integer` / `int` aliases, a boolean) keeps the database's exact total. A column with no numeric or boolean declaration keeps the database's own arithmetic. SQLite is untouched. **Why the operand is the column's text, not a plain cast.** The text is what the SQL client hands `find()`, so it is what the rows path adds. - On the exact-decimal columns the two are identical. Both servers turn a decimal into a double through its text. Measured on 20,000 random `numeric(65,30)` / `DECIMAL(65,30)` values per server: 0 differ. - On a binary32 column they are not identical. A table created before the exact-decimal columns (landed in `9cdffbe36`) still has `real` (PostgreSQL) / `FLOAT(8,2)` (MySQL) under a `number` declaration. There a plain cast adds the widened binary value, `0.30000000447034836` for `0.1 + 0.2`, where `find()` reads `0.1` and `0.2`. - Ablation 2 below pins this difference. ## Route picked by measurement: (a) **H1: reproduced at base `75b216924`.** Measured through `engine.aggregate` and `POST /api/v1/data/:object/query` on SQLite, a private live PostgreSQL 16.13 (`Asia/Shanghai`) and a private live MySQL 8.0.46 (`+08:00`). The rows path was forced with a filtered sibling aggregation. The two doors agree cell for cell. | `having` (object with `number` w, `rating` st, `boolean` flag) | SQLite native | SQLite rows | PG native | PG rows | MySQL native | MySQL rows | |:--|:--|:--|:--|:--|:--|:--| | `s $eq 0.3`, `s $in [0.3]` (sum w of 0.1, 0.2) | none | none | **kept** | none | **kept** | none | | `a $eq 0.15`, `a $in [0.15]` (avg w) | none | none | **kept** | none | **kept** | none | | `s $eq 0.1 + 0.2` | kept | kept | **none** | kept | **none** | kept | | `sa $eq 5 / 3` (avg st of 1, 2, 2) | kept | kept | kept | kept | **none** (`1.6667`) | kept | | `fa $eq 1 / 3` (avg flag of 1, 0, 0) | kept | kept | kept | kept | **none** (`0.3333`) | kept | Base values: PG and MySQL native `sum` = `0.3`, `avg` = `0.15`; every other face `0.30000000000000004` / `0.15000000000000002`. **At head** (driver-sql source identical from `07f81ea19` to `4caf9e6ef`), same doors: - `s $eq 0.3`, `s $in [0.3]`, `a $eq 0.15` and `a $in [0.15]` keep no group on any face. - `s $eq 0.1 + 0.2` and `a $eq (0.1 + 0.2) / 2` keep the group on all six. - `sa $eq 5 / 3` and `fa $eq 1 / 3` keep it on all six. - PG and MySQL native answer `0.30000000000000004` / `0.15000000000000002`. **H2: confirmed, with the order question answered.** - `sum(float8)` on PostgreSQL and `SUM(DOUBLE)` on MySQL add in scan order, one value after another, as the rows path's `reduce` does. Raw SQL over 0.1, 0.2, 0.3 answered `0.6000000000000001`, and `avg` answered `0.20000000000000004`, on both. The JS left fold gives the same. - SQLite 3.53.4, as bundled by better-sqlite3, answered `0.6` / `0.19999999999999998`. It uses compensated (Kahan-Babuska-Neumaier) summation, which SQLite has done since 3.43. - With two addends, no order and no compensation scheme can move the last place. The pin `0.1 + 0.2` is therefore deterministic across all four faces. For three or more fractions it is not deterministic on SQLite's native face: see the residual below. - `count` / `min` / `max` are untouched (pinned). - An integer column's `sum` stays exact (pinned). A `bigint` column holding 2^53 + 1 and 1 answers `9007199254740994` on all three cells, where adding doubles would give `9007199254740992`. **H3: (b) does not hold, on SQLite, for the pin itself.** SQLite's native `sum` over a REAL column adds the stored doubles, `0.30000000000000004`. An exact rows path (`0.3`) would disagree with it for exactly `0.1 + 0.2`, and changing SQLite's native face is outside route (b). The scale half of H3 also holds: in `examples/` at `4caf9e6ef`, 2 of the 70 fractional-family declarations carry a `scale` (app-todo `estimated_hours` and `actual_hours`). ### In-place fix, declared: `avg` over an integer-valued column Triage's route (a) names "a non-integer column". `avg` over an integer column is the same defect class: a native decimal quotient against the rows path's double. It sits in the same function and is closed by the same operand. No other claim holds the file, and it runs in the same gate family. So `avg` is `'double'` whatever the column holds. Evidence: - MySQL rounds a `DECIMAL` average to `div_precision_increment` (4) places: `1.6667` and `0.3333` above. - PostgreSQL's `numeric` average rounds to 16 places before the presenter rounds again. Over sums 1 to 3000 and counts 3 to 13 (27,962 non-integral pairs), 10 differ from JS division. Example: `11 / 9` answered `1.2222222222222222`, JS `1.2222222222222223`. - Pinned by the `nine` and `three` groups of the new test. ### Residual, stated and not fixed: SQLite's compensated sum For three or more fractional addends, SQLite's native face can still differ from every other face in the last place. `0.1 + 0.2 + 0.3` answers `0.6` on SQLite native and `0.6000000000000001` everywhere else, so `having { s: { $eq: 0.6 } }` keeps the group on SQLite native only, at head. - This was already true between SQLite's own two paths at base. - Neither route reaches it. SQLite's `sum` cannot be made to add naively in SQL, and PostgreSQL / MySQL cannot add with compensation in SQL. - It is reported to the seat as a separate finding. ## Tests - New: `packages/drivers/driver-sql/src/sql-driver-20387-aggregate-double-accumulation.test.ts`. It has 5 cases for each cell of the live matrix (`declareDialectCell`), so the PostgreSQL and MySQL cells run in `Temporal Conformance (live PG + MySQL)`, which runs the whole driver-sql suite with both live URLs and `OS_EXPECT_LIVE_DIALECT_MATRIX=1`: - **The pin.** `sum` / `avg` over `number` and `currency` columns holding 0.1 and 0.2 equal the rows path's arithmetic over `find()`'s own rows, and the literals `0.30000000000000004` / `0.15000000000000002`. So `having` `$eq 0.3` / `$in [0.3]` / `$eq 0.15` keep no group and `$eq 0.1 + 0.2` keeps it: `having-filter.ts` compares the value through `@objectstack/formula`'s `looseEq`, which is strict `===` for numbers (seat edit after review 5875498653). - **Binary32.** A `number` column retyped to `real` / `FLOAT(8,2)` adds what `find()` reads. - **Integer average.** `avg` over `rating` and `boolean` columns is the double quotient (5 / 3, 11 / 9, 1 / 3). - **Integer sum.** `sum` over an integer column keeps the exact total (the 2^53 + 2 case). - **Unchanged functions.** `count` / `min` / `max` are unchanged. - Result: 15 passed (5 cases × SQLite, live PostgreSQL, live MySQL), every cell exercised (verbose reporter). - **Ablations.** The fix was committed first, and each mutation went through `scripts/ablation-replace.mjs` in wrap mode. The test imports `./sql-driver.js` (source), so no `dist/` leg exists. Each run was the new file on all three cells: 1. The operand was disabled (`false &&`). Anchor 1 → 0, blob `63e5dac455` → `5e33daf1c6`. **6 failed | 9 passed:** the PG and MySQL pin (`expected 0.3 to be 0.30000000000000004`), binary32 (`0.3`) and integer average (PG `nine avg(rating): expected 1.222222222222222…`, MySQL `three avg(rating): expected 1.6667`). Every SQLite case, the integer sum and count/min/max stayed green. 2. A plain cast replaced the text operand. Blob → `4929128349`. **2 failed | 13 passed:** the binary32 case on PG and MySQL only (`expected 0.30000000447034836 to be 0.30000000000000004`). The pin stayed green, so the plain cast equals the text cast on exact decimals. 3. `sum: 'double'` dropped the integer exception. Blob → `958360a033`. **2 failed | 13 passed:** the integer sum on PG and MySQL (`expected 9007199254740992 to be 9007199254740994`). - All three failed in the predicted direction, on the predicted cells. Each restore reported blob == HEAD (`63e5dac455`) and an empty `git diff HEAD`, and `git status --porcelain` was empty afterwards. - **Whole `@objectstack/driver-sql` suite** at `4caf9e6ef`, with live PostgreSQL and MySQL, `TZ=America/New_York` and `OS_EXPECT_LIVE_DIALECT_MATRIX=1`: 207 files passed, 4720 passed | 1 skipped. The reporter confirmed that all 3 dialects were exercised; the 1 skip is for a reason other than a missing backend. - `packages/rest/src/rest-aggregate-numeric-having.test.ts`, the pins from PR objectstack-ai#20372, ran with both live URLs against this driver: 36 passed (12 cases × 3 cells). - **Typecheck** at `4caf9e6ef`: `@objectstack/driver-sql`, and its two subclasses `@objectstack/driver-turso` and `@objectstack/driver-sqlite-wasm`, all exit 0. The new test file is in driver-sql's `tsc` program (`--listFiles`). ## Gates Measured at head `4caf9e6ef`. That head is a true merge of `origin/main` (`8e0285918`), after a full workspace build (`turbo run build`, 72 of 72 tasks). - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 63 commands. All 63 ran, each exit code captured before any pipe, and all 63 exited 0. `--ran` reconciliation: 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN. - `origin/main` moved during the run, so the three `--base` gates were also run with `--base 8e02859`, each exiting 0. `check-changeset-no-major` has no PR payload locally; its level axis reads the PR in CI. - `check:driver-conformance` before and after: 50 covered, 0 DEBT, 0 exempt; dialect axis 8 suites, 0 in the DIALECT ledger. The ledger did not move. - **Lint, a proved narrowing.** `eslint --no-inline-config --format json` over the two touched TypeScript files: 2 files, 0 errors, 0 warnings. `eslint --print-config` resolves both files with `parserOptions` `ecmaVersion` / `sourceType` only, with no `project` and no type-aware rules. So this diff cannot move the verdict on an untouched file. The repo-wide `pnpm lint` is CI's. - NOT MEASURED locally, owned by CI: - the workspace type-check lanes, the `Test Core` shards, `Dogfood`, `Build Core`, and `Temporal Conformance` as CI spells it; - the five CI-variable families dispatch-gates names: `check-issue-citations --census`, three `check-shard-attestation --emit`, and `check-test-completeness`. ## Acceptance notes - **Legacy binary32 columns change answer.** On a table created before the exact-decimal columns, the native `sum` / `avg` over a `real` / `FLOAT` column used to be computed as follows. PostgreSQL's `sum(real)` accumulated in float4 (0.1, 0.2, 1234567.9 answered `1.2345681e+06`). MySQL's `SUM(FLOAT(8,2))` answered a double shown to 2 decimals (`123457.20` for 0.1, 0.2, 123456.9). Both now answer the rows path's double, the sum of what `find()` reads: `1234568.2` on the PostgreSQL example, where a plain cast would have answered `1234568.1750000045`. - **MySQL floor.** `CAST(… AS DOUBLE)` needs MySQL 8.0.17 or later. CI's `mysql:8.0` image is newer than that. An older server refuses `sum` / `avg` over a declared numeric column with a syntax error, which is loud, not a wrong number. - **Analytics face, read and not measured.** service-analytics' `NativeSQLStrategy` (`AGGREGATE_SQL` in `native-sql-strategy.ts`) emits a raw `SUM(col)` / `AVG(col)`. `AnalyticsServicePlugin` auto-bridges it when the data engine exposes `execute()`. By reading, a cube measure on PostgreSQL / MySQL still adds exact decimals there. No door was measured, so there is no `reach:`. Carrier: none. - **Where the `having` pin lives.** The `having` pin at the engine and REST doors across dialects is this PR's local measurement (above). In CI the value pins sit in driver-sql's matrix, and those values decide `having`. The REST test's PostgreSQL / MySQL cells are still provisioned by no CI job, as PR objectstack-ai#20372 recorded. --- _Generated by [Claude Code](https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Fixes #20335
Clause-②: no
What changed
SqlDriver.aggregatehanded the SQL client's answer straight through. node-postgres parsesbigint(OID 20) andnumeric(OID 1700) to strings, and mysql2 does the same forDECIMAL. Socount/count_distinct/sum/avgreached the engine'shavingand the REST response as strings on the native path of PostgreSQL (all four) and MySQL (sum/avg), while SQLite and the rows path of every dialect answered numbers.The fix is one presentation step at the end of the native path, in
packages/drivers/driver-sql/src/sql-driver.ts:AGGREGATE_ANSWER_KINDis aRecordover the spec'sAggregationFunction.count,count_distinct,sumandavgare'number';minandmaxare'column'. A function added to the enum without an entry failstsc.aggregate(), an aliased aggregation whose function is'number'registers its alias with the existing'number'presenter (presentReadValue, the oneformatOutputapplies to a numeric field on afind()row).min/maxkeep the column's own presentation, unchanged.COUNT) pass through untouched.find(),distinct()and thewherecomparand path are unchanged.packages/objectqlis untouched.Precision policy (H3)
The answer is one JS number, an IEEE-754 double, on every dialect. A
sum/avgwhose exact value needs more than a double's 15 to 17 significant digits, or an integer total at or above 2^53, is rounded to the nearest double. The policy is stated onAGGREGATE_ANSWER_KINDand in the changeset.Options weighed:
having$in, or a chart reading the column, would silently stop matching for exactly those groups. That is also the harder shape for an AI author to get right.find()already puts on a read of the same exact-decimal column, sinceformatOutput's numeric pass (valueSchemaForgives the numeric classz.number().finite(), ADR-0104 D1). It is also the bound the rows path has always had:in-memory-aggregation.tssums JS doubles.What the spec says an aggregate's value type is:
packages/spec/src/data/aggregation-conformance.tstypesAggregationExpectation.valueasnumberand states it "stays anumberfor every case", across every enrolled face.service-analytics'measure-result-type.tspublishesnumberas the result type ofcount/count_distinct/sum/avgmeasures. Before this change, PostgreSQL delivered strings under that declaration.Measured (H1, H2)
Base
26daf0b036and headf3b9e1390c, on SQLite (better-sqlite3), live PostgreSQL 16.13 (server zone Asia/Shanghai) and live MySQL 8.0.46 (+08:00). Each cell went throughSqlDriver.aggregate,engine.aggregateandPOST /api/v1/data/:object/query(JSON round-trip). The fixture isgroupBycustomer, four groups, overnumberfields (one integer-valued, one decimal-valued),currency,percentandrating. The three native doors answered identically at base and at head, on every dialect.count,count_distinct"2"2sum/avgover number, currency, percent"500.000…"(30-digit scale)500sum/avgover rating (int4)"7"/"3.5000000000000000"7/3.5count,count_distinctsum/avgover number, currency, percent, ratingDECIMAL)min/maxover any of the five, native, PG and MySQLH2: a raw query through
pganswers field OIDsn:20 total:1700 mean:1700 mn:1700 mx:1700 ss:20 sa:1700 smin:23. Every value is a string exceptsmin(int4). The same query throughmysql2answers column typesn:8(LONGLONG, a number),246(NEWDECIMAL, a string) forsum/avg/min/maxof the decimal column and forsum/avgof the int column, and3forminof the int column. Somin/maxover anumericcolumn is a string at the client too. It already left the driver as a number, because a declared numeric field takes the'number'column presentation since the numeric-representation change. The rows path yields numbers becausefind()presents the numeric column as a number andin-memory-aggregation.tscomputescountasrows.lengthandsum/avgin JS arithmetic.The card's
havingtable, engine and REST doors (havingis evaluated by the engine on both paths):having{ n: { $in: [2] } }{ total: { $in: [500, 20] } }{ mean: { $in: [250, 600] } }{ avg_rate: { $in: [0.375] } }{ n: { $lt: 'not-a-date' } }{ total: { $lt: 'not-a-date' } }{ n: { $gt: '+010000-01-01T00:00:00.000Z' } }$eqon count / sum / avg,$gt/$gtenumbersAt base, SQLite (both paths) and the rows path of PostgreSQL and MySQL already answered the head column.
Collateral (H4)
find()anddistinct()over the five numeric columns are byte-identical base to head, on all three dialects (compared as typed JSON).min/maxpresentation is untouched: the'column'arm is the pre-existing branch.sql-driver-aggregate-temporal-output.test.tsandsql-driver-13973-canonical-iso-read-door.test.tspass in the live suite below.Consumers (H5)
These read the changed values and now receive a number:
@objectstack/objectqlaggregateSummaryValue: roll-up summaries write the value into the parent'ssummaryfield. That value is now a number where PostgreSQL (count,sum,avg) and MySQL (sum,avg) handed back a numeric string.summary-backfill.tscountsnonEmptyfor acount/sumroll-up only whentypeof computed === 'number', so by reading, its report under-counted those roll-ups on PostgreSQL (count,sum) and MySQL (sum) before; it counts them now.service-analyticsObjectQLStrategy: passes values through into responses that declarenumber, and now delivers one.cross-object-rebucket.tsanddataset-executor.tscoerce withNumber(...), which is the identity on a number.metadata-protocol(the REST query route),runtimeaction-executionaggregate,mcpstdio-data-bridge, and the client SDK: pass-through, no coercion.ObjectChartreadValue,ObjectMetricWidgetandMetricWidgetread these values throughNumber(...), which is the identity on a number.None of the consumers above compares these values as strings or branches on
typeof value === 'string'; the onetypeoftest (summary-backfill.ts) asks for'number'.Tests
packages/drivers/driver-sql/src/sql-driver-20335-aggregate-numeric-presentation.test.ts. Seven cases per dialect cell of the live matrix (declareDialectCell, so the PostgreSQL and MySQL cells run inTemporal Conformance (live PG + MySQL)):count/count_distinct/sum/avgover number, currency, percent and rating, andsum/avgover a boolean, asserted withtoBeagainst the rows' own JS arithmetic;min/maxpresentation unchanged;9007199254740993and12345678901234567.123456789answerNumber(literal), never a string, and equal whatfind()reads for the same row.packages/rest/src/rest-aggregate-numeric-having.test.ts. Engine and REST doors, native and rows paths:having$in/$eqoncount/count_distinct/sum/avg, the string-comparand rows of the card, and the response's JSON numbers. The SQLite cell always runs. The PostgreSQL and MySQL cells run whenOS_TEST_POSTGRES_URL/OS_TEST_MYSQL_URLare set and are otherwise a named skip.presentedOutput.set(agg.alias, 'number')line was deleted throughscripts/ablation-replace.mjs(anchor 1 → 0, blob8dc157d535→c247bcff26). driver-sql was rebuilt, andablation-dist-preflight --absentconfirmed the marker absent from all six built files. Result:c1 count(*): expected '2' to be 2). Every SQLite case and MySQLcountstayed green.git diff HEADempty. After a rebuild the marker is present in 2 built files, andgit status --porcelainis empty.06349dc973(identical code tof3b9e1390c, which only corrects a docblock). PostgreSQL and MySQL were live, withTZ=America/New_YorkandOS_EXPECT_LIVE_DIALECT_MATRIX=1.@objectstack/driver-sql: 204 files passed, 4690 tests passed, 1 skipped. The reporter confirmed that all 3 dialects were exercised.@objectstack/restlocalproject, with no live URL, as in CI: 210 files passed, 3817 tests passed, 26 skipped (24 of them this file's PostgreSQL and MySQL cells).@objectstack/objectqlaggregate,having, in-memory aggregation and summary files: 19 files, 612 tests passed.@objectstack/driver-sqlexit 0 and@objectstack/restexit 0 (includingcheck:test-typecheck). Both new files are in a typecheck program (--listFiles).Gates
At head
f3b9e1390c, after a full workspace build (turbo run buildover./packages/*and./packages/*/*, 71 of 71 tasks):node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackderived 63 commands, the same 63 as the dispatch list. All 63 ran and exited 0.--ranreconciliation: 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN.origin/mainmoved during the run, so the three--basegates (check-adr-0087-registration,check-changeset-no-major,check-empty-changeset) andcheck-issue-citationswere also run with--base 26daf0b036(the merge base). Each exited 0.check-changeset-no-majorreads the level from the PR, so its local run has no PR payload to judge.eslint --no-inline-config --format jsonover the three touched TypeScript files reports 3 files, 0 errors, 0 warnings.eslint --print-configresolves a config for each, so none is ignored. The config enables no type-aware linting (parserOptionsisecmaVersion/sourceTypeonly, with noproject), so this diff cannot move the verdict on an untouched file. The repo-widepnpm lintis CI's.Test Coreshards,Dogfood,Build Core, andTemporal Conformanceas CI spells it. The driver-sql suite was run locally against live PostgreSQL and MySQL as above.Acceptance notes
numbercolumn holding 0.1 and 0.2 sums to0.3on PostgreSQL and MySQL native (exactnumeric/DECIMALarithmetic) and to0.30000000000000004on the rows path and on SQLite (double arithmetic). Sohaving { s: { $eq: 0.3 } }keeps that group on PG / MySQL native and on no other face. Measured identical at base and at head: this PR moves the type, not the arithmetic. Reported to the seat as a separate finding.readPresentationKind's'number'kind (used bymin/maxanddistinct()) readsnumericFields, which includes the driver-internalinteger/int/floataliases, on every dialect.formatOutputnarrows its row pass tonumericValueFieldson PostgreSQL and MySQL, to keep an introspected externalbigintabove 2^53 as a string. Not measured, not changed here.driver-sql's suite. CI runs their SQLite cell. The PostgreSQL and MySQL value pins run in CI through the driver-sql file. Carrier: none.Generated by Claude Code