Skip to content

fix(driver-sql): aggregate count / count_distinct / sum / avg answer numbers on PostgreSQL and MySQL - #20372

Merged
objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-20335-pg-aggregate-numbers
Sep 28, 2026
Merged

objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-20335-pg-aggregate-numbers

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #20335
Clause-②: no

What changed

SqlDriver.aggregate handed the SQL client's answer straight through. node-postgres parses bigint (OID 20) and numeric (OID 1700) to strings, and mysql2 does the same for DECIMAL. So count / count_distinct / sum / avg reached the engine's having and the REST response as strings on the native path of PostgreSQL (all four) and MySQL (sum / avg), while SQLite and the rows path of every dialect answered numbers.

The fix is one presentation step at the end of the native path, in packages/drivers/driver-sql/src/sql-driver.ts:

  • AGGREGATE_ANSWER_KIND is a Record over the spec's AggregationFunction. count, count_distinct, sum and avg are 'number'; min and max are 'column'. A function added to the enum without an entry fails tsc.
  • In aggregate(), an aliased aggregation whose function is 'number' registers its alias with the existing 'number' presenter (presentReadValue, the one formatOutput applies to a numeric field on a find() row). min / max keep the column's own presentation, unchanged.
  • It is keyed on the function the query asked for, never on what a value looks like. It is not gated by dialect: the presenter rewrites only a string, so SQLite's answers (and mysql2's COUNT) pass through untouched.
  • No connection-level type parser is touched. find(), distinct() and the where comparand path are unchanged. packages/objectql is untouched.

Precision policy (H3)

The answer is one JS number, an IEEE-754 double, on every dialect. A sum / avg whose exact value needs more than a double's 15 to 17 significant digits, or an integer total at or above 2^53, is rounded to the nearest double. The policy is stated on AGGREGATE_ANSWER_KIND and in the changeset.

Options weighed:

  • A. A JS number, with the loss beyond double precision undeclared. Rejected, because the loss must be written down.
  • B. A string only when the value is unsafe as a double, a number otherwise. Rejected. The value's type would depend on its size, so the defect would come back for large totals only: a having $in, or a chart reading the column, would silently stop matching for exactly those groups. That is also the harder shape for an AI author to get right.
  • C. Chosen: always a number, with the loss declared. It is the same bound find() already puts on a read of the same exact-decimal column, since formatOutput's numeric pass (valueSchemaFor gives the numeric class z.number().finite(), ADR-0104 D1). It is also the bound the rows path has always had: in-memory-aggregation.ts sums JS doubles.

What the spec says an aggregate's value type is: packages/spec/src/data/aggregation-conformance.ts types AggregationExpectation.value as number and states it "stays a number for every case", across every enrolled face. service-analytics' measure-result-type.ts publishes number as the result type of count / count_distinct / sum / avg measures. Before this change, PostgreSQL delivered strings under that declaration.

Measured (H1, H2)

Base 26daf0b036 and head f3b9e1390c, on SQLite (better-sqlite3), live PostgreSQL 16.13 (server zone Asia/Shanghai) and live MySQL 8.0.46 (+08:00). Each cell went through SqlDriver.aggregate, engine.aggregate and POST /api/v1/data/:object/query (JSON round-trip). The fixture is groupBy customer, four groups, over number fields (one integer-valued, one decimal-valued), currency, percent and rating. The three native doors answered identically at base and at head, on every dialect.

face base head
PostgreSQL native: count, count_distinct string "2" number 2
PostgreSQL native: sum / avg over number, currency, percent string "500.000…" (30-digit scale) number 500
PostgreSQL native: sum / avg over rating (int4) string "7" / "3.5000000000000000" number 7 / 3.5
MySQL native: count, count_distinct number number, unchanged
MySQL native: sum / avg over number, currency, percent, rating string (DECIMAL) number
min / max over any of the five, native, PG and MySQL number number, unchanged
SQLite native, every cell number byte-identical
rows path, every dialect, every cell number byte-identical

H2: a raw query through pg answers field OIDs n:20 total:1700 mean:1700 mn:1700 mx:1700 ss:20 sa:1700 smin:23. Every value is a string except smin (int4). The same query through mysql2 answers column types n:8 (LONGLONG, a number), 246 (NEWDECIMAL, a string) for sum / avg / min / max of the decimal column and for sum / avg of the int column, and 3 for min of the int column. So min / max over a numeric column is a string at the client too. It already left the driver as a number, because a declared numeric field takes the 'number' column presentation since the numeric-representation change. The rows path yields numbers because find() presents the numeric column as a number and in-memory-aggregation.ts computes count as rows.length and sum / avg in JS arithmetic.

The card's having table, engine and REST doors (having is evaluated by the engine on both paths):

having base: PG native base: MySQL native head: every dialect × path
{ n: { $in: [2] } } no group c1, c2 c1, c2
{ total: { $in: [500, 20] } } no group no group c1, c4
{ mean: { $in: [250, 600] } } no group no group c1, c2
{ avg_rate: { $in: [0.375] } } no group no group c1
{ n: { $lt: 'not-a-date' } } c1–c4 no group no group
{ total: { $lt: 'not-a-date' } } c1–c4 c1–c4 no group
{ n: { $gt: '+010000-01-01T00:00:00.000Z' } } c1–c4 no group no group
$eq on count / sum / avg, $gt / $gte numbers as elsewhere as elsewhere unchanged

At base, SQLite (both paths) and the rows path of PostgreSQL and MySQL already answered the head column.

Collateral (H4)

  • find() and distinct() over the five numeric columns are byte-identical base to head, on all three dialects (compared as typed JSON).
  • SQLite: every aggregate cell is byte-identical base to head, on every door.
  • No connection-level parser changed. The temporal min / max presentation is untouched: the 'column' arm is the pre-existing branch. sql-driver-aggregate-temporal-output.test.ts and sql-driver-13973-canonical-iso-read-door.test.ts pass in the live suite below.

Consumers (H5)

These read the changed values and now receive a number:

  • @objectstack/objectql aggregateSummaryValue: roll-up summaries write the value into the parent's summary field. That value is now a number where PostgreSQL (count, sum, avg) and MySQL (sum, avg) handed back a numeric string. summary-backfill.ts counts nonEmpty for a count / sum roll-up only when typeof computed === 'number', so by reading, its report under-counted those roll-ups on PostgreSQL (count, sum) and MySQL (sum) before; it counts them now.
  • service-analytics ObjectQLStrategy: passes values through into responses that declare number, and now delivers one. cross-object-rebucket.ts and dataset-executor.ts coerce with Number(...), which is the identity on a number.
  • metadata-protocol (the REST query route), runtime action-execution aggregate, mcp stdio-data-bridge, and the client SDK: pass-through, no coercion.
  • objectui, read by code search: ObjectChart readValue, ObjectMetricWidget and MetricWidget read these values through Number(...), which is the identity on a number.

None of the consumers above compares these values as strings or branches on typeof value === 'string'; the one typeof test (summary-backfill.ts) asks for 'number'.

Tests

  • New: packages/drivers/driver-sql/src/sql-driver-20335-aggregate-numeric-presentation.test.ts. Seven cases per dialect cell of the live matrix (declareDialectCell, so the PostgreSQL and MySQL cells run in Temporal Conformance (live PG + MySQL)):
    • count / count_distinct / sum / avg over number, currency, percent and rating, and sum / avg over a boolean, asserted with toBe against the rows' own JS arithmetic;
    • the all-NULL fold;
    • min / max presentation unchanged;
    • the precision policy: SQL-written 9007199254740993 and 12345678901234567.123456789 answer Number(literal), never a string, and equal what find() reads for the same row.
  • New: packages/rest/src/rest-aggregate-numeric-having.test.ts. Engine and REST doors, native and rows paths: having $in / $eq on count / count_distinct / sum / avg, the string-comparand rows of the card, and the response's JSON numbers. The SQLite cell always runs. The PostgreSQL and MySQL cells run when OS_TEST_POSTGRES_URL / OS_TEST_MYSQL_URL are set and are otherwise a named skip.
  • Reverse verification: the fix was committed first, then the one presentedOutput.set(agg.alias, 'number') line was deleted through scripts/ablation-replace.mjs (anchor 1 → 0, blob 8dc157d535 → c247bcff26). driver-sql was rebuilt, and ablation-dist-preflight --absent confirmed the marker absent from all six built files. Result:
    • driver-sql test: 7 failed, 14 passed. The failures are every PG and MySQL string cell (for example c1 count(*): expected '2' to be 2). Every SQLite case and MySQL count stayed green.
    • REST test: 13 failed, 23 passed. The failures are the PG and MySQL cells the base table marks. SQLite stayed green.
    • The ablated native answers were byte-identical to the base on all three dialects.
    • Restore: blob equals HEAD, git diff HEAD empty. After a rebuild the marker is present in 2 built files, and git status --porcelain is empty.
  • Whole suites, at 06349dc973 (identical code to f3b9e1390c, which only corrects a docblock). PostgreSQL and MySQL were live, with TZ=America/New_York and OS_EXPECT_LIVE_DIALECT_MATRIX=1.
    • @objectstack/driver-sql: 204 files passed, 4690 tests passed, 1 skipped. The reporter confirmed that all 3 dialects were exercised.
    • @objectstack/rest local project, with no live URL, as in CI: 210 files passed, 3817 tests passed, 26 skipped (24 of them this file's PostgreSQL and MySQL cells).
    • @objectstack/objectql aggregate, having, in-memory aggregation and summary files: 19 files, 612 tests passed.
  • Typecheck: @objectstack/driver-sql exit 0 and @objectstack/rest exit 0 (including check:test-typecheck). Both new files are in a typecheck program (--listFiles).

Gates

At head f3b9e1390c, after a full workspace build (turbo run build over ./packages/* and ./packages/*/*, 71 of 71 tasks):

  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 63 commands, the same 63 as the dispatch list. All 63 ran and exited 0. --ran reconciliation: 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN.
  • origin/main moved during the run, so the three --base gates (check-adr-0087-registration, check-changeset-no-major, check-empty-changeset) and check-issue-citations were also run with --base 26daf0b036 (the merge base). Each exited 0. check-changeset-no-major reads the level from the PR, so its local run has no PR payload to judge.
  • Lint, a proved narrowing: eslint --no-inline-config --format json over the three touched TypeScript files reports 3 files, 0 errors, 0 warnings. eslint --print-config resolves a config for each, so none is ignored. The config enables no type-aware linting (parserOptions is ecmaVersion / sourceType only, with no project), so this diff cannot move the verdict on an untouched file. The repo-wide pnpm lint is CI's.
  • NOT MEASURED locally, and owned by CI: the type-check lanes over the whole workspace, Test Core shards, Dogfood, Build Core, and Temporal Conformance as CI spells it. The driver-sql suite was run locally against live PostgreSQL and MySQL as above.

Acceptance notes

  • The rows path and the exact-decimal native path can differ in the last place of a double. A number column holding 0.1 and 0.2 sums to 0.3 on PostgreSQL and MySQL native (exact numeric / DECIMAL arithmetic) and to 0.30000000000000004 on the rows path and on SQLite (double arithmetic). So having { s: { $eq: 0.3 } } keeps that group on PG / MySQL native and on no other face. Measured identical at base and at head: this PR moves the type, not the arithmetic. Reported to the seat as a separate finding.
  • readPresentationKind's 'number' kind (used by min / max and distinct()) reads numericFields, which includes the driver-internal integer / int / float aliases, on every dialect. formatOutput narrows its row pass to numericValueFields on PostgreSQL and MySQL, to keep an introspected external bigint above 2^53 as a string. Not measured, not changed here.
  • The REST test's PostgreSQL and MySQL cells are provisioned by no CI job today; the live servers are attached to driver-sql's suite. CI runs their SQLite cell. The PostgreSQL and MySQL value pins run in CI through the driver-sql file. Carrier: none.

Generated by Claude Code

…s on every dialect

node-postgres hands bigint (count) and numeric (sum, avg) back as strings and
mysql2 does the same for DECIMAL, so the native aggregate path answered "2"
where SQLite and the engine's rows path answer 2. Key the 'number' read
presentation on the aggregate function the query asked for, with one precision
policy: a JS double, the loss beyond a double's precision declared.

Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx
Co-authored-by: Claude <noreply@anthropic.com>
…g across both paths

Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx
Co-authored-by: Claude <noreply@anthropic.com>
… with the precision policy

Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx
Co-authored-by: Claude <noreply@anthropic.com>
…ialect

The docblock still said the numeric coercion is SQLite-only, which the
numeric-representation change had already made false and the aggregate
presentation makes false a second time.

Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation tests tooling labels Sep 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/driver-sql, touching 3 documentable anchor(s).

13 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/ai/natural-language-queries.mdx (via count_distinct (symbol, a field of const object AGGREGATE_ANSWER_KIND))
  • content/docs/data-modeling/drivers.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/data-modeling/index.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/data-modeling/queries.mdx (via count_distinct (symbol, a field of const object AGGREGATE_ANSWER_KIND))
  • content/docs/deployment/validating-metadata.mdx (via count_distinct (symbol, a field of const object AGGREGATE_ANSWER_KIND))
  • content/docs/kernel/contracts/data-engine.mdx (via count_distinct (symbol, a field of const object AGGREGATE_ANSWER_KIND))
  • content/docs/permissions/tenant-audit-census.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/plugins/packages.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/protocol/kernel/index.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/protocol/kernel/lifecycle.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/protocol/objectql/query-syntax.mdx (via SqlDriver (symbol, a top-level class), count_distinct (symbol, a field of const object AGGREGATE_ANSWER_KIND))
  • content/docs/protocol/objectql/types.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/ui/dashboards.mdx (via count_distinct (symbol, a field of const object AGGREGATE_ANSWER_KIND))

⛔ 2 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/v15.mdx (via count_distinct (symbol, a field of const object AGGREGATE_ANSWER_KIND))
  • content/docs/releases/v17/17-0.mdx (via SqlDriver (symbol, a top-level class), count_distinct (symbol, a field of const object AGGREGATE_ANSWER_KIND))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • 5 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 11 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json a88a1bb399ea1ba6d717b8f6fefa0b13ec3e66a8 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 03e73ce74f8534883ed7fbfd940b711d1eafa48c — the merge of head f3b9e1390c0bcea6aa49033485960a41647a48db into base a88a1bb399ea1ba6d717b8f6fefa0b13ec3e66a8, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 03e73ce74f8534883ed7fbfd940b711d1eafa48c && git checkout 03e73ce74f8534883ed7fbfd940b711d1eafa48c
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin a88a1bb399ea1ba6d717b8f6fefa0b13ec3e66a8 f3b9e1390c0bcea6aa49033485960a41647a48db && git checkout -B drift-repro a88a1bb399ea1ba6d717b8f6fefa0b13ec3e66a8 && git merge --no-ff f3b9e1390c0bcea6aa49033485960a41647a48db

node scripts/docs-audit/affected-docs.mjs --json a88a1bb399ea1ba6d717b8f6fefa0b13ec3e66a8

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs a88a1bb399ea1ba6d717b8f6fefa0b13ec3e66a8 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: f3b9e1390c0bcea6aa49033485960a41647a48db
Local-runs: none

Inputs read, and nothing else: card #20335 (body and all four comments — triage 5861208644, claim 5861869486, os-dev-report 5862967111, takeover claim 5863887111); PR #20372 body and file list (4 files, equal to git diff origin/main... at the head: sql-driver.ts, two new test files, one changeset); the head's check-runs: 34 runs, 31 success, 3 skipped — Console Pin Gate, Build Docs, Packed-tarball smoke (opt-in), all three rostered as expected skips in scripts/pm/check-expected-skips.mjs; none queued or in progress, none red; the one commit status (Vercel, ignored build step) is success. Read-only git and REST GETs only; nothing built, run or re-run.

① Derived judgments

  1. SqlDriver.aggregate answer type for count / count_distinct / sum / avg: on PostgreSQL (all four) and MySQL (sum / avg) a numeric string becomes a JS number; SQLite and the engine's rows path are byte-identical. Public surface: the value's runtime type on two dialects moves TO what the spec already declares (packages/spec/src/data/aggregation-conformance.ts types AggregationExpectation.value as number, "stays a number for every case"; service-analytics measure-result-type.ts publishes number for these four). Accept set: unchanged — no query shape is newly accepted or newly refused; only the answer side moves. RIGHT: a fix toward the declared contract, neither a widening nor a narrowing.

  2. Precision policy (the dev's option C): the answer is Number(text), one IEEE-754 double on every dialect; a total at or above 2^53, or one needing more than ~17 significant digits, rounds to the nearest double and is never a string. Public surface: a PG / MySQL native sum / avg beyond double precision left the driver as exact text before and leaves as a rounded double now. RIGHT: it is the bound ADR-0104 D1 (valueSchemaFor gives the numeric class z.number().finite()) and formatOutput's numeric pass already put on a find() of the same column, and the bound in-memory-aggregation.ts (toNumber, JS reduce) always had on the rows path; a value-dependent type (option B) would reopen the card's defect for exactly the large totals. Declared on AGGREGATE_ANSWER_KIND and in the changeset, as triage asked. Residual: PG numeric 'NaN' stays a string under the presenter's existing rule — declared in the docblock; right to leave.

  3. AGGREGATE_ANSWER_KIND: a module-private const (no export) — no new export, no public surface. It is a Readonly Record over AggregationFunction; the spec enum (query.zod.ts:159) has exactly six members — count, sum, avg, min, max, count_distinct — and all six carry an entry; funcName = agg.function is typed by AggregationNodeSchema.function: AggregationFunction, so the index is total and a seventh member fails tsc (Type Check · workspace green). RIGHT.

  4. The min / max branch re-keyed to AGGREGATE_ANSWER_KIND[funcName] === 'column': same membership as the former funcName === 'min' || funcName === 'max'; the [finding] AGGREGATION_ROWS has no boolean column, so the cross-driver aggregation conformance family cannot see a boolean aggregand on any face #11152 boolean exception (kind !== 'boolean') and the temporal presentation are kept verbatim. RIGHT — no behaviour moves; the live temporal pins ran green in the head's Temporal Conformance (live PG + MySQL).

  5. Ordering: presentReadColumns(this.foldEmptyAggregateAnswers(rows, foldedOutput), presentedOutput) — the sum over a column that is NULL in every row of a group: driver-sql answers null, the engine's in-memory aggregate tier answers 0 — same query, same rows, fork chosen by a driver capability bit #15546 fold runs first; emptyGroupValueFor gives 0 (a number) for count / count_distinct / sum, which the 'number' presenter passes untouched since it rewrites only a non-empty string that Number() parses; avg over all-NULL stays null (value == null returns as is). RIGHT; the driver test pins both.

  6. Dialect-ungated on purpose: the presenter rewrites only strings, so better-sqlite3's numbers and mysql2's COUNT (a number) pass through — no "string-answering dialects" list to drift. RIGHT; SQLite byte-identity is what the dev's rows table and the ablation show, and the SQLite cells of both files ran under Test Core (green).

  7. Unaliased aggregations stay untracked — AggregationNodeSchema.alias is z.string() (required), so that branch is defensive only. RIGHT.

  8. find(), distinct(), the where comparand coercion, the node-pg / mysql2 connection-level parsers and packages/objectql: untouched — the net diff touches aggregate() and three docblocks in one file, exactly the surface the claim named and inside every ⛔ of the dispatch. RIGHT.

  9. The head commit f3b9e13 is a comment-only docblock correction (verified: git diff 06349dc973 f3b9e1390c is 3 insertions, 1 deletion, all inside a comment). Its claim — the numeric coercion runs on every dialect — is TRUE on this head: readPresentationKind's 'number' arm reads numericFields with no dialect gate, and formatOutput's numeric pass runs on every dialect with a per-dialect registry (numericFields on SQLite, numericValueFields elsewhere). RIGHT.

  10. Changeset prose (the shipped CHANGELOG text, a review face): every sentence checks against the diff and the code — the PG / MySQL wire types named (bigint for count and sum over an integer column, numeric for sum / avg over the exact-decimal family and avg over an integer column; mysql2 DECIMAL for SUM / AVG); the having outcomes match the card's table and the dev's measured rows; "min / max unchanged (a declared numeric field was already a number)" matches readPresentationKind; "find() / distinct() unchanged, no connection-level type parser touched" matches the diff; the precision paragraph matches AGGREGATE_ANSWER_KIND; and the consumer note names the one visible move honestly (a consumer that compared these as strings, or checked typeof value === 'string', now receives a number). RIGHT.

  11. Consumers, by reading: packages/objectql/src/summary-backfill.ts:423 counts nonEmpty only when typeof computed === 'number', so the PR's "under-counted on PG / MySQL before, counts now" reading is right; having-filter.ts branches on typeof value === 'string' only to format a diagnostic (line 357), never to compare. No consumer in this repo compares these values as strings. RIGHT.

  12. Reach of the gates on this head: the driver-sql test declares its cells through declareDialectCell, and Temporal Conformance (live PG + MySQL) runs pnpm --filter @objectstack/driver-sql test — the whole suite — with both URLs and OS_EXPECT_LIVE_DIALECT_MATRIX=1 (ci.yml ~1382–1399); that check-run is success, so the PostgreSQL and MySQL value pins RAN on this head and are green. The REST file's SQLite cell ran under Test Core (packages/rest local project, configDefaults.include). RIGHT — the fix's live cells are red-capable in CI, and the dev's reverse verification (7 driver-sql failures, every one a PG / MySQL string cell; 13 REST failures, the PG / MySQL cells; SQLite green throughout) shows the pins turn on exactly the one deleted line.

  13. The REST test's "rows" axis: a per-aggregation filter forces the engine's in-memory path (engine.ts ~16505, engine.aggregate: add per-aggregation filter to the contract — ruled half of #10413 (measure-level filters on the ObjectQL analytics path) #10576), so its two paths are genuinely native and rows, not native twice. RIGHT.

② Semver level

  • .changeset/20335-pg-aggregate-numbers.md declares '@objectstack/driver-sql': patch. The only released package whose shipped code changes is @objectstack/driver-sql (17.4.0, not private); @objectstack/rest gains a test file only and publishes nothing, so it owes no changeset. A bug fix in a released package takes patch, never none and never skip-changeset (Post-Task Checklist §3). Check Changeset and Lint & Repo Gates are green on the head. RIGHT.
  • Not minor: no new accepted input and no new export. Not breaking: nothing an author can write is removed or renamed, and the value-type move on two dialects is toward the spec's declared number, not away from it; the changeset discloses the one visible move and the precision policy, which is the disclosure triage 5861208644 asked for. RIGHT.
  • Clause-②: line — Clause-②: no in the PR body, in the changeset body (where check-adr-0087-registration reads it) and in both claims; no (widening) / (narrowing) arm. The answer to 「本卡放宽接受集或扩大公开面吗」 is no: the accept set is unchanged and the public surface gains nothing; the arm is rightly absent because the accept set is not narrowed either. Consistent on every carrier. RIGHT.

③ Boundary flags

open_questions: [] — none to answer.

Dev flags (os-dev-report 5862967111, deviations D1–D6 and out_of_scope_findings F1–F3), each answered or escalated:

  • D1 (PG role without superuser) and D2 (MySQL socket via symlink): local test-environment shapes on the dev's container; no repository effect — the diff is the four files. Answered: no action.
  • D3 (stray /suite2.pid at the dev container's filesystem root): not in the tree (git ls-tree of the head lists no *.pid); the branch carries nothing from it. Answered: no repository action; the container belongs to the stood-down os-sales seat.
  • D4 (comment-only docblock correction in presentReadValue): inside the claimed file and judged TRUE in ①.9. Answered: right.
  • D5 (whole suites at 06349dc, gate union at f3b9e13): the two heads differ by three comment lines, and every check-run on f3b9e13 itself is green, so nothing rests on the dev's local run. Answered.
  • D6 / F3 (the REST test reads OS_TEST_POSTGRES_URL / OS_TEST_MYSQL_URL directly with describe.skipIf(!config); its PG / MySQL cells are provisioned by no CI job): two consequences.
    • Accepted: the card's execution note 3 (pin having $in / $eq on count / sum / avg on SQLite, PostgreSQL and MySQL, both paths) is met in the file and was measured green by the dev with live URLs (36 passed). In CI the load-bearing half — the driver answering numbers on PG and MySQL — is pinned red-capable by the driver-sql file (①.12), and the having evaluation is objectql's dialect-agnostic code, exercised on both paths by the SQLite cell. Not a FAIL.
    • Escalated to the seat (carrier none): unlike its sibling packages/rest/src/data-date-read-year-below-100.test.ts, the new file does not honour OS_EXPECT_LIVE_DIALECT_MATRIX=1 — under that flag with a URL missing the sibling reds by design and the new file stays a skip. No CI job runs packages/rest under the flag today, so no gate moves on this head; it is a latent vacuous-pass hole should rest ever join the live job. A follow-up hygiene card, or a one-line pin when that wiring happens; not a condition on this PR.
  • F1 (class a): exact-decimal native arithmetic (0.1 + 0.2 = 0.3 on PG numeric / MySQL DECIMAL) against double arithmetic on the rows path and SQLite (0.30000000000000004), so having { s: { $eq: 0.3 } } keeps the group on PG / MySQL native only. Measured identical at base and head — this PR moves the type, not the arithmetic — and correctly kept outside a card whose dispatch excluded objectql's having and the arithmetic path. Escalated: the seat files it bare (the report carries the dedupe words); no card is cited as filed yet. Not a condition on this PR.
  • F2 (carrier none): readPresentationKind's 'number' kind reads numericFields (driver-internal integer / int / float aliases included) on every dialect, while formatOutput narrows to numericValueFields on PG / MySQL to keep an introspected external bigint above 2^53 a string. Pre-existing since [finding] The NUMERIC column family diverges from driver-sql in both migration formats — driver real, generators numeric, and rating is real against integer #16318; reaches min / max and distinct() on introspected tables, not this card's four functions, whose 'number' presentation is the declared policy. Answered: not a condition; the seat may hold it for a later card.
  • The card's citation of a string-count acceptance note on PR fix(objectql,core): a per-aggregation filter and having read a temporal comparand by the column's storage rule — one rule in core, shared with both drivers' where #20202 did not reproduce (the dev read five temporal notes there); nothing in the fix turned on it. Answered: no action.

Implemented-by: claude/issue-20335-pg-aggregate-numbers
Reviewed-by: session_01N8TPEsoJxPsdSdNKGnNGEN

VERDICT: PASS

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review September 28, 2026 05:36
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Sep 28, 2026
Merged via the queue into main with commit 15bf186 Sep 28, 2026
36 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-20335-pg-aggregate-numbers branch September 28, 2026 05:58
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Sep 28, 2026
…o fields of different comparison classes when it is authored (objectstack-ai#20347) (objectstack-ai#20403)

Fixes objectstack-ai#20347
Clause-②: yes (narrowing)

The spec half of the objectstack-ai#20347 triage split (`5862073027`), dispatched on
claim `5863797885`. Base `eee09742`, head `bef47d1d`. The engine half is
objectstack-ai#20355, which stays open and reads the export this PR adds. The
changeset declares `Clause-②: yes (narrowing)`, BREAKING, `minor` on
both `@objectstack/spec` (new exports, a widening) and
`@objectstack/lint` (a new authoring refusal, a narrowing).

## What changes

- **One classification, exported once**
(`packages/spec/src/data/filter-cross-field-comparison-class.ts`,
re-exported from `@objectstack/spec/data`, beside
`filter-text-operator-declared-type.ts`).
- Six classes (`CROSS_FIELD_COMPARISON_CLASSES`: `numeric`, `text`,
`boolean`, `date`, `datetime`, `time`) and three families with none
(`CROSS_FIELD_NO_CLASS_REASONS`: `list-or-object`, `file`, `formula`).
- `CROSS_FIELD_COMPARISON_TYPE_CLASSES` classifies every `FieldType`
member exactly once, by reference to the existing `field-value.zod.ts`
sets. Nothing is re-listed.
- Two pure verdicts. `crossFieldColumnVerdict(field)` answers one
declared column; `multiple: true` on a multi-capable type holds a list.
`crossFieldComparisonVerdict(left, right)` answers two: `comparable`,
`cross-class`, `no-class`, or `unjudged` for a type outside `FieldType`.
- It is lifted case for case from driver-sql's module-private
`crossFieldComparisonClass` (the objectstack-ai#5222 boundary). `sql-driver.ts` is
untouched: objectstack-ai#20355 rewires it, and PR objectstack-ai#20372 holds that file.
- **Parity with driver-sql, run against both**
(`packages/drivers/driver-sql/src/sql-driver-20347-cross-field-class-parity.test.ts`).
One object declares every `FieldType` member (49), plus the 6
multi-capable members flagged `multiple: true`. Every ordered pair (55 ×
55 = 3,025) is compiled as `{ a: { $eq: { $field: b } } }` on a real
`:memory:` SQLite driver. The driver's admit or refuse must equal
`crossFieldComparisonVerdict(a, b) === 'comparable'` on every pair. A
refusal counts only in the cross-field boundary's own withheld
`INVALID_FILTER` / 400 form (`withheldFilterDiagnosticOf` non-null),
never by prose.
- **The authoring door** (`packages/lint`).
- `validateRlsPredicateEnforceability` gains a cross-class arm.
`crossClassComparisons` reads the lowered filter's `{ $field }` sites
against the declared field map. It reports `rls-predicate-unenforceable`
for every comparison whose two columns are not `comparable`: `==`, `!=`,
`>`, `>=`, `<`, `<=`, either side, under `!` too.
  - It covers `using` and `check` on every operation.
- `validateSharingRuleEnforceability` reads the same function and
reports `sharing-rule-unlowerable-condition` on a sharing rule's lowered
`condition`.
- A comparison against a list or an object stays the existing objectstack-ai#19886
arm's finding, so no comparison is reported twice. The new arm runs
ahead of the engine-judge pass, like the list arm: one defect, one
finding.
- The finding names each comparison, each column's declared type and
class (or why it has none), and the clause's measured run-time
consequence. The hint lists every class with the declared types it
holds, derived from the spec table.

## Measured before (lint as on `main`), then after

Real `os validate` (`packages/cli/bin/run-dev.js validate` on a probe
stack), plus the real plugin-security + ObjectQL on driver-sql
(`better-sqlite3` `:memory:`, one RLS policy on a `text` / `number` /
`image` / `formula` object).

| predicate | `os validate` before | `find` (`using`) | insert (`check`)
| insert (`using` as check) | by-id update / delete (`using`) | `os
validate` after |
|:--|:--|:--|:--|:--|:--|:--|
| `record.status != record.amount` (text vs number) | valid, exit 0 |
`INVALID_FILTER` / 400 | admitted, stored | admitted, stored | 403 / 403
| `rls-predicate-unenforceable`, exit 1 |
| `record.status != record.photo` (text vs image) | valid, exit 0 | 400
| admitted, stored | admitted, stored | 403 / 403 | refused, exit 1 |
| `record.status != record.is_open` (text vs formula; the card's NOT
MEASURED cell) | valid, exit 0 | 400 | admitted, stored | admitted,
stored | 403 / 403 | refused, exit 1 |
| `record.amount > record.status` (number vs text) | — | 400 | 403 (JS
`5 > 'open'` is false) | 403 | 403 / 403 | refused (lint unit and door
pins) |
| control `record.status != record.note` (text vs text) | valid, exit 0
| rows `[r1]` | admitted | admitted | updated / deleted | valid, exit 0
|

The `check` rows on `insert` read the same at `os validate`: valid
before, `rls-predicate-unenforceable` after. Sharing-rule conditions,
measured at the real `os validate`, first with the arm ablated (the
before-state) and then restored: `record.status != record.amount` and
`record.status != record.photo` went from valid (exit 0) to
`sharing-rule-unlowerable-condition` (exit 1). The control
`record.status != record.note` stayed valid. At run time the seeded
rule's criteria query meets the same driver-sql refusal the list-holding
class meets (objectstack-ai#20375 measured that path).

The write-check answer is whatever JavaScript's comparison of the two
raw values gives, so the permissive side of the policy is the write.
That half is objectstack-ai#20355's.

## Census (expected 0): 0

A script over `git ls-files examples packages` (tests, fixtures, docs,
generated bundles excluded; 3,140 files at `bef47d1d`) extracts every
`using` / `check` / `condition` string literal: 163. It lowers each
through the real `compileCelToFilter` (RLS through `sqlPredicateToCel`
first); 105 lower. It then lists every `{ $field }` comparison: 2.
- `examples/app-showcase/src/data/hooks/index.ts:88`: `record.spent >
record.budget`, a hook condition, both `number`.
- `packages/lint/scripts/check-doc-formula-expressions.mjs:1396`:
`record.a > record.b`, a gate fixture.

Neither is an RLS predicate or a sharing-rule condition, and both are
same-class. The only programmatic predicate constant is
`OWNERSHIP_FLOOR_PREDICATE` (`created_by == current_user.id`), which is
not field-to-field. So no shipped policy or sharing condition moves, and
nothing re-grades to p1. The cloud repository was not in this session:
NOT MEASURED.

## Ablation (one-time proof, committed state `bef47d1d`)

Two ablations, both run from the committed state `bef47d1d`, each
through `scripts/ablation-replace.mjs`. That tool landed each mutation
(anchor count 1 to 0, blob changed) and restored it (the blob equals
`HEAD`, and `git diff HEAD` is empty). A shell `trap` re-checked each
restore by hash. The direction observed is the normal one: red.

1. **The lint arm.** The guard line in `crossClassComparisons` was
replaced with an unconditional `continue`, so the arm reports nothing.
`ablation-dist-preflight` found the marker in 4 built
`@objectstack/lint` files, so the mutation reached the `dist/` the CLI
consumes.
- lint unit, the four cross-class and list-holding files: **525 failed /
485 passed** of 1,010. Restored: **1,010 / 1,010 passed**.
- CLI integration `rls-policy-authoring-admission.test.ts`: **6 failed /
33 passed**. The 6 are exactly the new REFUSED rows. Restored: **39 / 39
passed**.
- Real `os validate`, 9 cells. Ablated: all nine exit 0 with no finding,
which is the before-state, sharing cells included. Restored: the 3 RLS
`using` cells, the 2 RLS `check` cells and the 2 sharing cells exit 1,
each with exactly one finding; both controls exit 0.
- On restore, `ablation-dist-preflight --absent` passed its `dist/`
reading (the marker is absent from all 14 built files). Its tree reading
exited 3 only because two untracked scratch files were present at that
moment; both are deleted now.
2. **The driver half of the parity pin.** Temporarily, never committed:
in `sql-driver.ts`'s `crossFieldComparisonClass`, `if (type === 'time')
return 'time'` was changed to return `'datetime'`. The parity test
imports driver source, so no build was needed. Result: **2 failed / 54
passed**. The two are `f_datetime` and `f_time`, naming exactly
`f_datetime vs f_time: spec says cross-class, driver admitted` and its
mirror. Restored: **56 / 56 passed**, blob equal to `HEAD`.

## Tests (at `bef47d1d`)

All at `bef47d1d`, after the last commit, on a shared box.
- `@objectstack/spec`
- `vitest run --project local src/data`: 103 files, **3,458 passed**, 1
todo. The new classification test contributes 19.
  - `typecheck` (tsc, scripts and the test layer): exit 0.
- `@objectstack/lint`
  - `pnpm test`: 115 files, **5,314 passed**.
  - `typecheck` (with the test layer): exit 0.
- `@objectstack/driver-sql`
- The parity test plus the two existing cross-field suites
(`sql-driver-cross-field-reference`,
`sql-driver-cross-field-conformance`): **221 passed**, 2 skipped. The
parity test alone: 56 passed, one test per probe column (55 × 55 pairs),
plus the coverage pin.
  - `typecheck`: exit 0.
- `@objectstack/cli`
- `--project integration test/rls-policy-authoring-admission.test.ts`,
the only CLI file touched (integration tier): **39 passed**, 9 of them
new.
  - `typecheck`: exit 0.
- The unit tier is declared to CI: no CLI source file and no unit-tier
file changed.
- Real `os validate` over the examples: `app-crm`, `app-multi-package`
and `app-todo` exit 0, with 0 `rls-predicate-*` / `sharing-rule-*`
findings. `app-showcase` is NOT MEASURED this way: its config imports
`@objectstack/connector-mcp`, which is outside this worktree's build
closure. Its security files are in the text census above.
- Spec generated artifacts: `check:generated` named `api-surface/` and
`export-origins/` stale, both additive only. Both were regenerated with
their generators, and `check:api-surface` and `check:export-origins` are
green.
- Gates: `dispatch-gates --ran` accounts for 88 of 88 derived families.
86 exited 0. Two are NOT MEASURED, and CI owns both:
- `check:dual-build-cjs-loads` answered PREREQUISITE NOT MET: it needs a
full `pnpm build`.
- `check:type-check-debt`: its `--re-measure` passed the 400 s local
timeout. The kill left `packages/spec/dist` without declarations, so the
spec was rebuilt (64 `.d.ts`) before every lint, driver-sql and cli
reading above.
- The derivation warned that the tree is behind `origin/main` by one
family file (`scripts/cross-package-test-inputs.mjs`).
`check:cross-package-test-inputs` was run from this tree and is green.

## Decisions

- **Formula has no class, whatever its `returnType`.** That is
driver-sql's answer: a formula is virtual, with no column to reference.
The text-operator door reads `returnType`, but a column-to-column
comparison needs a column on both sides. The measured runtime agrees
(400 on the read).
- **The file family is refused by name.** That is driver-sql's answer
too (the ADR-0104 dual-encoding window), so `image == image` is refused
as well.
- **A type outside `FieldType` is `unjudged`.** A driver's aliases
(`integer`, `object`, the absent-type `string` default) stay layered in
the driver, as `field-value.zod.ts`'s header says every alias does.
objectstack-ai#20355's rewire keeps those aliases above the export. At the door, an
out-of-vocabulary type is Zod's to refuse, and the arm reports nothing.
- **Registry-injected columns are judged** by the definition the
registry provisions. `record.status != record.created_at` is refused
(text vs datetime), because the driver sees the same column. `id` has no
definition in the graph, so it is not judged.
- **Same rule ids as the list arm.** The author's edit is the same kind:
rewrite which two columns are compared.
- **Two existing pins changed**, one in each objectstack-ai#19886 list-holding test.
"A single-valued `file` field is one value" asserted *no finding at all*
for `record.status != record.subject` with `subject` a single `file`.
driver-sql refuses that comparison (the file family has no class), so
the no-finding reading was never the runtime's. Each pin now asserts
that the list arm stays silent and the class arm refuses once. `select`
/ `lookup` / `user` keep the no-finding pin.
- **File surface beyond the claim, both required by the dispatch.** The
driver-sql parity test: the classification can only be run "against
both" there, and it adds no line to `sql-driver.ts`. And
`validate-sharing-rule-enforceability.ts` plus its tests: the direction
covers sharing conditions, and that rule is where they are judged.

## Acceptance notes

- `listHoldingComparisons` still reads `STRUCTURED_JSON_TYPES` +
`isMultiValueField` directly. That is the same family as the export's
`list-or-object` reason, and the two agree by construction (pinned in
the spec test), but it is two spellings. Converging it onto
`crossFieldColumnVerdict` is the natural edit for whoever next touches
that function (carrier: objectstack-ai#20355 or the next objectstack-ai#19886-family change). Noted,
not filed.
- The metadata save door for a `sharing_rule` does not run
`validateSharingRuleEnforceability`, as objectstack-ai#20375 recorded. The new sharing
arm therefore shows at `os validate` / `os build` / `os lint` only, like
the list arm. Noted, not filed.
- The `check` consequence sentence describes today's write check, which
admits by raw comparison. When objectstack-ai#20355 moves the write check onto this
classification, that sentence changes in the same change (a code comment
at `crossClassConsequence` says so).

---
_Generated by [Claude
Code](https://claude.ai/code/session_01QcAS3qiYYZNezaxZxaUdMV)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
veigajoao pushed a commit to veigajoao/objectstack that referenced this pull request Sep 29, 2026
…QL, as on SQLite and the rows path (objectstack-ai#20486)

Fixes objectstack-ai#20387

Clause-②: no

## What changed

`SqlDriver.aggregate` (`packages/drivers/driver-sql/src/sql-driver.ts`)
now makes PostgreSQL and MySQL accumulate `sum` / `avg` in double. That
is the arithmetic SQLite and the engine's rows path (`objectql`'s
`in-memory-aggregation.ts`) already use. This is route (a) of triage
5865067775, under the one-double policy that PR objectstack-ai#20372's changeset
stated for the answer's type.

- `AGGREGATE_ACCUMULATION` is a `Record` over the spec's
`AggregationFunction`, a sibling of `AGGREGATE_ANSWER_KIND`. Its entries
are `avg: 'double'`, `sum: 'double-over-fractional'`, and `count` /
`count_distinct` / `min` / `max`: `'as-stored'`. A function added to the
enum without an entry fails `tsc`.
- `fractionalNumericFields` is a registry filled beside `numericFields`
in both fills and aliased for shards. It holds the declared columns
whose type stores fractions: `numericColumnFor(type).kind === 'exact'`
(`number`, `currency`, `percent`, `slider`, `progress`, `summary`) and
the driver's `float` alias.
- In `aggregate()`, on PostgreSQL and MySQL only, two cases take a
double operand: `avg` over a declared numeric or boolean column, and
`sum` over a fractional column. The operand is `cast(cast(x as text) as
double precision)` on PostgreSQL and `cast(cast(x as char) as double)`
on MySQL. The column stays one `??` binding.
- A `sum` over an integer-valued column (`rating`, the `integer` / `int`
aliases, a boolean) keeps the database's exact total. A column with no
numeric or boolean declaration keeps the database's own arithmetic.
SQLite is untouched.

**Why the operand is the column's text, not a plain cast.** The text is
what the SQL client hands `find()`, so it is what the rows path adds.

- On the exact-decimal columns the two are identical. Both servers turn
a decimal into a double through its text. Measured on 20,000 random
`numeric(65,30)` / `DECIMAL(65,30)` values per server: 0 differ.
- On a binary32 column they are not identical. A table created before
the exact-decimal columns (landed in `9cdffbe36`) still has `real`
(PostgreSQL) / `FLOAT(8,2)` (MySQL) under a `number` declaration. There
a plain cast adds the widened binary value, `0.30000000447034836` for
`0.1 + 0.2`, where `find()` reads `0.1` and `0.2`.
- Ablation 2 below pins this difference.

## Route picked by measurement: (a)

**H1: reproduced at base `75b216924`.** Measured through
`engine.aggregate` and `POST /api/v1/data/:object/query` on SQLite, a
private live PostgreSQL 16.13 (`Asia/Shanghai`) and a private live MySQL
8.0.46 (`+08:00`). The rows path was forced with a filtered sibling
aggregation. The two doors agree cell for cell.

| `having` (object with `number` w, `rating` st, `boolean` flag) |
SQLite native | SQLite rows | PG native | PG rows | MySQL native | MySQL
rows |
|:--|:--|:--|:--|:--|:--|:--|
| `s $eq 0.3`, `s $in [0.3]` (sum w of 0.1, 0.2) | none | none |
**kept** | none | **kept** | none |
| `a $eq 0.15`, `a $in [0.15]` (avg w) | none | none | **kept** | none |
**kept** | none |
| `s $eq 0.1 + 0.2` | kept | kept | **none** | kept | **none** | kept |
| `sa $eq 5 / 3` (avg st of 1, 2, 2) | kept | kept | kept | kept |
**none** (`1.6667`) | kept |
| `fa $eq 1 / 3` (avg flag of 1, 0, 0) | kept | kept | kept | kept |
**none** (`0.3333`) | kept |

Base values: PG and MySQL native `sum` = `0.3`, `avg` = `0.15`; every
other face `0.30000000000000004` / `0.15000000000000002`.

**At head** (driver-sql source identical from `07f81ea19` to
`4caf9e6ef`), same doors:

- `s $eq 0.3`, `s $in [0.3]`, `a $eq 0.15` and `a $in [0.15]` keep no
group on any face.
- `s $eq 0.1 + 0.2` and `a $eq (0.1 + 0.2) / 2` keep the group on all
six.
- `sa $eq 5 / 3` and `fa $eq 1 / 3` keep it on all six.
- PG and MySQL native answer `0.30000000000000004` /
`0.15000000000000002`.

**H2: confirmed, with the order question answered.**

- `sum(float8)` on PostgreSQL and `SUM(DOUBLE)` on MySQL add in scan
order, one value after another, as the rows path's `reduce` does. Raw
SQL over 0.1, 0.2, 0.3 answered `0.6000000000000001`, and `avg` answered
`0.20000000000000004`, on both. The JS left fold gives the same.
- SQLite 3.53.4, as bundled by better-sqlite3, answered `0.6` /
`0.19999999999999998`. It uses compensated (Kahan-Babuska-Neumaier)
summation, which SQLite has done since 3.43.
- With two addends, no order and no compensation scheme can move the
last place. The pin `0.1 + 0.2` is therefore deterministic across all
four faces. For three or more fractions it is not deterministic on
SQLite's native face: see the residual below.
- `count` / `min` / `max` are untouched (pinned).
- An integer column's `sum` stays exact (pinned). A `bigint` column
holding 2^53 + 1 and 1 answers `9007199254740994` on all three cells,
where adding doubles would give `9007199254740992`.

**H3: (b) does not hold, on SQLite, for the pin itself.** SQLite's
native `sum` over a REAL column adds the stored doubles,
`0.30000000000000004`. An exact rows path (`0.3`) would disagree with it
for exactly `0.1 + 0.2`, and changing SQLite's native face is outside
route (b). The scale half of H3 also holds: in `examples/` at
`4caf9e6ef`, 2 of the 70 fractional-family declarations carry a `scale`
(app-todo `estimated_hours` and `actual_hours`).

### In-place fix, declared: `avg` over an integer-valued column

Triage's route (a) names "a non-integer column". `avg` over an integer
column is the same defect class: a native decimal quotient against the
rows path's double. It sits in the same function and is closed by the
same operand. No other claim holds the file, and it runs in the same
gate family. So `avg` is `'double'` whatever the column holds. Evidence:

- MySQL rounds a `DECIMAL` average to `div_precision_increment` (4)
places: `1.6667` and `0.3333` above.
- PostgreSQL's `numeric` average rounds to 16 places before the
presenter rounds again. Over sums 1 to 3000 and counts 3 to 13 (27,962
non-integral pairs), 10 differ from JS division. Example: `11 / 9`
answered `1.2222222222222222`, JS `1.2222222222222223`.
- Pinned by the `nine` and `three` groups of the new test.

### Residual, stated and not fixed: SQLite's compensated sum

For three or more fractional addends, SQLite's native face can still
differ from every other face in the last place. `0.1 + 0.2 + 0.3`
answers `0.6` on SQLite native and `0.6000000000000001` everywhere else,
so `having { s: { $eq: 0.6 } }` keeps the group on SQLite native only,
at head.

- This was already true between SQLite's own two paths at base.
- Neither route reaches it. SQLite's `sum` cannot be made to add naively
in SQL, and PostgreSQL / MySQL cannot add with compensation in SQL.
- It is reported to the seat as a separate finding.

## Tests

- New:
`packages/drivers/driver-sql/src/sql-driver-20387-aggregate-double-accumulation.test.ts`.
It has 5 cases for each cell of the live matrix (`declareDialectCell`),
so the PostgreSQL and MySQL cells run in `Temporal Conformance (live PG
+ MySQL)`, which runs the whole driver-sql suite with both live URLs and
`OS_EXPECT_LIVE_DIALECT_MATRIX=1`:
- **The pin.** `sum` / `avg` over `number` and `currency` columns
holding 0.1 and 0.2 equal the rows path's arithmetic over `find()`'s own
rows, and the literals `0.30000000000000004` / `0.15000000000000002`. So
`having` `$eq 0.3` / `$in [0.3]` / `$eq 0.15` keep no group and `$eq 0.1
+ 0.2` keeps it: `having-filter.ts` compares the value through
`@objectstack/formula`'s `looseEq`, which is strict `===` for numbers
(seat edit after review 5875498653).
- **Binary32.** A `number` column retyped to `real` / `FLOAT(8,2)` adds
what `find()` reads.
- **Integer average.** `avg` over `rating` and `boolean` columns is the
double quotient (5 / 3, 11 / 9, 1 / 3).
- **Integer sum.** `sum` over an integer column keeps the exact total
(the 2^53 + 2 case).
  - **Unchanged functions.** `count` / `min` / `max` are unchanged.
- Result: 15 passed (5 cases × SQLite, live PostgreSQL, live MySQL),
every cell exercised (verbose reporter).
- **Ablations.** The fix was committed first, and each mutation went
through `scripts/ablation-replace.mjs` in wrap mode. The test imports
`./sql-driver.js` (source), so no `dist/` leg exists. Each run was the
new file on all three cells:
1. The operand was disabled (`false &&`). Anchor 1 → 0, blob
`63e5dac455` → `5e33daf1c6`. **6 failed | 9 passed:** the PG and MySQL
pin (`expected 0.3 to be 0.30000000000000004`), binary32 (`0.3`) and
integer average (PG `nine avg(rating): expected 1.222222222222222…`,
MySQL `three avg(rating): expected 1.6667`). Every SQLite case, the
integer sum and count/min/max stayed green.
2. A plain cast replaced the text operand. Blob → `4929128349`. **2
failed | 13 passed:** the binary32 case on PG and MySQL only (`expected
0.30000000447034836 to be 0.30000000000000004`). The pin stayed green,
so the plain cast equals the text cast on exact decimals.
3. `sum: 'double'` dropped the integer exception. Blob → `958360a033`.
**2 failed | 13 passed:** the integer sum on PG and MySQL (`expected
9007199254740992 to be 9007199254740994`).
- All three failed in the predicted direction, on the predicted cells.
Each restore reported blob == HEAD (`63e5dac455`) and an empty `git diff
HEAD`, and `git status --porcelain` was empty afterwards.
- **Whole `@objectstack/driver-sql` suite** at `4caf9e6ef`, with live
PostgreSQL and MySQL, `TZ=America/New_York` and
`OS_EXPECT_LIVE_DIALECT_MATRIX=1`: 207 files passed, 4720 passed | 1
skipped. The reporter confirmed that all 3 dialects were exercised; the
1 skip is for a reason other than a missing backend.
- `packages/rest/src/rest-aggregate-numeric-having.test.ts`, the pins
from PR objectstack-ai#20372, ran with both live URLs against this driver: 36 passed
(12 cases × 3 cells).
- **Typecheck** at `4caf9e6ef`: `@objectstack/driver-sql`, and its two
subclasses `@objectstack/driver-turso` and
`@objectstack/driver-sqlite-wasm`, all exit 0. The new test file is in
driver-sql's `tsc` program (`--listFiles`).

## Gates

Measured at head `4caf9e6ef`. That head is a true merge of `origin/main`
(`8e0285918`), after a full workspace build (`turbo run build`, 72 of 72
tasks).

- `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` derived 63 commands. All 63 ran, each exit
code captured before any pipe, and all 63 exited 0. `--ran`
reconciliation: 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN.
- `origin/main` moved during the run, so the three `--base` gates were
also run with `--base 8e02859`, each exiting 0.
`check-changeset-no-major` has no PR payload locally; its level axis
reads the PR in CI.
- `check:driver-conformance` before and after: 50 covered, 0 DEBT, 0
exempt; dialect axis 8 suites, 0 in the DIALECT ledger. The ledger did
not move.
- **Lint, a proved narrowing.** `eslint --no-inline-config --format
json` over the two touched TypeScript files: 2 files, 0 errors, 0
warnings. `eslint --print-config` resolves both files with
`parserOptions` `ecmaVersion` / `sourceType` only, with no `project` and
no type-aware rules. So this diff cannot move the verdict on an
untouched file. The repo-wide `pnpm lint` is CI's.
- NOT MEASURED locally, owned by CI:
- the workspace type-check lanes, the `Test Core` shards, `Dogfood`,
`Build Core`, and `Temporal Conformance` as CI spells it;
- the five CI-variable families dispatch-gates names:
`check-issue-citations --census`, three `check-shard-attestation
--emit`, and `check-test-completeness`.

## Acceptance notes

- **Legacy binary32 columns change answer.** On a table created before
the exact-decimal columns, the native `sum` / `avg` over a `real` /
`FLOAT` column used to be computed as follows. PostgreSQL's `sum(real)`
accumulated in float4 (0.1, 0.2, 1234567.9 answered `1.2345681e+06`).
MySQL's `SUM(FLOAT(8,2))` answered a double shown to 2 decimals
(`123457.20` for 0.1, 0.2, 123456.9). Both now answer the rows path's
double, the sum of what `find()` reads: `1234568.2` on the PostgreSQL
example, where a plain cast would have answered `1234568.1750000045`.
- **MySQL floor.** `CAST(… AS DOUBLE)` needs MySQL 8.0.17 or later. CI's
`mysql:8.0` image is newer than that. An older server refuses `sum` /
`avg` over a declared numeric column with a syntax error, which is loud,
not a wrong number.
- **Analytics face, read and not measured.** service-analytics'
`NativeSQLStrategy` (`AGGREGATE_SQL` in `native-sql-strategy.ts`) emits
a raw `SUM(col)` / `AVG(col)`. `AnalyticsServicePlugin` auto-bridges it
when the data engine exposes `execute()`. By reading, a cube measure on
PostgreSQL / MySQL still adds exact decimals there. No door was
measured, so there is no `reach:`. Carrier: none.
- **Where the `having` pin lives.** The `having` pin at the engine and
REST doors across dialects is this PR's local measurement (above). In CI
the value pins sit in driver-sql's matrix, and those values decide
`having`. The REST test's PostgreSQL / MySQL cells are still provisioned
by no CI job, as PR objectstack-ai#20372 recorded.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

3 participants