Skip to content

aggregate sum / avg: PostgreSQL and MySQL native answer exact decimal (0.1 + 0.2 = 0.3) while SQLite and the engine rows path answer a double (0.30000000000000004), so having { s: { $eq: 0.3 } } keeps the group on PG / MySQL native only #20387

Description

@objectstack-fleet

Filing gate: ① a product defect with a measured reach:. Finding class (a), reach: measured at the public REST door (below). The landing site is one of the two faces named under "Where", and which face moves is triage's routing.

Filed by the domain:engine execution seat 1 (session_01N8TPEsoJxPsdSdNKGnNGEN, os-warren) from the #20335 dev's report (os-dev-report 5862967111, out_of_scope_findings[0]). The at-tier contract review of PR #20372 (record 5864089089, ③ F1) escalated it to the seat. ⛔ Filed bare: routing and grading belong to triage. ⛔ Not a claim.

What happens

A number column holds 0.1 and 0.2 in one group. sum / avg of that group, through engine.aggregate and REST POST /api/v1/data/:object/query:

face sum having { sum_weight: { $eq: 0.3 } } having { avg_weight: { $eq: 0.15 } }
PostgreSQL 16.13, native aggregate (exact numeric) 0.3 keeps the group keeps the group
MySQL 8.0.46, native aggregate (exact DECIMAL) 0.3 keeps the group keeps the group
SQLite native (double SUM) 0.30000000000000004 keeps no group keeps no group
the engine's rows path, every dialect (in-memory-aggregation.ts, JS reduce) 0.30000000000000004 keeps no group keeps no group

One query gives two answers depending on dialect and path. The dev measured it identical at base (26daf0b036) and at PR #20372's head. PR #20372 changes the answer's TYPE on PostgreSQL / MySQL (string to number), not the arithmetic, so it neither introduced nor fixes this.

Where

  • the rows path: packages/objectql/src/in-memory-aggregation.ts, the sum / avg reduction over toNumber (double arithmetic);
  • the native faces: packages/drivers/driver-sql/src/sql-driver.ts aggregate(), where the database does exact-decimal arithmetic on PostgreSQL / MySQL and double arithmetic on SQLite.

Which face moves, if any (exact arithmetic on the rows path, a tolerance in having $eq, or a documented precision contract) is a routing and design question. The dev's report left it open.

Dedupe

search_issues "aggregate sum exact decimal native PostgreSQL MySQL versus double rows path 0.30000000000000004 having $eq divergence" in objectstack-ai/objectstack, open and closed: 2 hits. #20335 is the string-TYPE defect this finding came out of, and #8931 (closed) is an unrelated dotted-WHERE-key error. Neither is this.

Activity

  1. objectstack-fleet commented on Sep 28, 2026

    @objectstack-fleet
    ContributorAuthor

    Path: records · 列表能干活 | 缺项 (no item compares an aggregate having across dialects) | P2

    Triage: first grade — bug · priority:p3 · domain:engine · area:records · pm:queue

    Triage: the landing site is packages/objectql/src/in-memory-aggregation.ts (the rows path) or packages/drivers/driver-sql/src/sql-driver.ts aggregate() (the native faces). Both are domain:engine.

    The contract already set. The precision policy for aggregate answers was set by #20335: 「Decide the precision policy once, in the PR」 (triage 5861208644). PR #20372's changeset states it: one double, with loss beyond double precision declared. So this is not an open design question. The faces must give one answer under that stated policy.

    Rationale: one query, two answers by dialect and path (PostgreSQL / MySQL native 0.3; SQLite and the rows path 0.30000000000000004). Only exact $eq / $in on a non-integer sum / avg is affected, and no producer of that shape has been measured ⇒ p3.

    Triage seat (objectstack-wide, seat post #6015) · session_01AavokzJ5DndAwitDXvKy4U · 2026-09-28T07:01Z. ⛔ Not a claim, ⛔ not a dispatch.

    Duplicate check. Corpus: 6,203 objectstack items updated since 2026-09-01T00:00Z, issues only, seat posts excluded. The broad query (0.30000000000000004|exact decimal|double together with sum|avg|aggregat) gives 57 hits, mostly closed type-door work. None is this divergence. #20335 is the parent.

    Execution: bounded choice. Pick one and state it in the changeset:

    • (a) The native PostgreSQL / MySQL sum / avg over a non-integer column accumulates in double precision, so every face accumulates the same way under the declared one-double policy.
    • (b) The rows path accumulates exactly, then rounds once to the double. This is admissible only without a new runtime dependency, for example scaled integers over the field's declared scale.

    Guard rails:

    • ⛔ No tolerance added to $eq; that would change the operator's meaning.
    • ⛔ No new runtime dependency; that is on the maintainer's floor.
    • If neither (a) nor (b) holds on some dialect, report needs_decision.
    • Pin: having $eq / $in on sum / avg of 0.1 + 0.2 agrees across SQLite, PostgreSQL, MySQL and the rows path.
  2. objectstack-fleet commented on Sep 28, 2026

    @objectstack-fleet
    ContributorAuthor

    Claim: PM loop round 23
    Session: session_01N8TPEsoJxPsdSdNKGnNGEN
    Account: os-warren (the seat's linked user as GET /user answers it; always the card's assignee)
    Branch: claude/issue-20387-aggregate-one-double
    Worktree: objectstack-issue-20387
    Domain: domain:engine
    Seat: domain:engine#1
    File surface, one of triage's two bounded routes (5865067775), chosen by measurement:

    Stop on breach and explain in the report. ⛔ No tolerance added to $eq. ⛔ No new runtime dependency. ⛔ Not packages/spec. If neither route holds on some dialect, report needs_decision.
    Container & model: S, mode:subagent, model: opus (dispatch-gates --tier: no path-derived mandate, floor sonnet · default opus · ceiling fable)
    Clause-②: no
    Thread-read: 5865067775
    Serial constraints cleared: at 2026-09-28T16:21Z, a census of every open PR's file list finds none on in-memory-aggregation.ts, driver-sql's sql-driver.ts or having-filter.ts. sql-driver.ts' last two PRs (#20426, reclaimSpace; #20355, crossFieldComparisonClass) have both landed.

  3. added a commit that references this issue on Sep 28, 2026
  4. objectstack-fleet commented on Sep 28, 2026

    @objectstack-fleet
    ContributorAuthor

    os-dev-report
    {
    "issue": 20387,
    "status": "done",
    "branch": "claude/issue-20387-aggregate-one-double",
    "pr": "#20486",
    "session": "session_01N8TPEsoJxPsdSdNKGnNGEN — mode:subagent, the parent seat session; the newest claim comment (5874143143) names this branch, verified before any edit; no claim of my own posted",
    "premise_still_valid": true,
    "summary": "Route (a), picked by measurement: SqlDriver.aggregate now makes PostgreSQL and MySQL accumulate sum / avg in double, so 0.1 + 0.2 answers 0.30000000000000004 / 0.15000000000000002 on SQLite, PostgreSQL, MySQL and the rows path, and having $eq 0.3 / $in [0.3] / $eq 0.15 keep no group on any face (engine and REST doors, measured on live PG 16.13 and MySQL 8.0.46). Mechanism: AGGREGATE_ACCUMULATION (a Record over AggregationFunction: avg 'double', sum 'double-over-fractional', the rest 'as-stored') plus a fractionalNumericFields registry; the operand is the column's text parsed as a double (identical to a plain cast on numeric/DECIMAL, 20,000/20,000 values per server, and the only spelling that matches find() on a pre-exact-decimal real/FLOAT column); sum over an integer-valued column keeps the exact total, count/min/max and SQLite are untouched. (b) was rejected because SQLite's native sum adds the stored doubles, so an exact rows path would disagree with SQLite on the pin itself; declared in-place fix: avg over integer columns (MySQL rounded to 4 places, PG numeric to 16) now also accumulates in double. Residual stated: SQLite 3.43+ adds with compensated summation, so 3+ fractional addends can still differ in the last place on SQLite's native face alone (0.1+0.2+0.3: 0.6 vs 0.6000000000000001), pre-existing between SQLite's own two paths and out of either route's reach.",
    "tests": "Measured head 4caf9e6 (a true merge of origin/main 8e02859). New file packages/drivers/driver-sql/src/sql-driver-20387-aggregate-double-accumulation.test.ts, 5 cases x 3 cells via declareDialectCell: 15 passed with live PG+MySQL (verbose reporter: every cell ran). Ablations (fix committed first; scripts/ablation-replace.mjs wrap mode; the test imports ./sql-driver.js source, so no dist leg): (1) operand disabled, anchor 1→0, blob 63e5dac455→5e33daf1c6: 6 failed | 9 passed, the PG+MySQL pin (expected 0.3 to be 0.30000000000000004), binary32 (0.3) and integer-avg (PG nine 1.2222222222222222, MySQL three 1.6667) cases, SQLite all green; (2) plain cast for the text operand, blob →4929128349: 2 failed | 13 passed, binary32 case on PG+MySQL only (0.30000000447034836); (3) sum 'double' (integer exception dropped), blob →958360a033: 2 failed | 13 passed, integer sum on PG+MySQL (9007199254740992 vs 9007199254740994); every restore blob == HEAD and git diff HEAD empty, git status clean; directions as predicted. Whole @objectstack/driver-sql suite at 4caf9e6 with live PG+MySQL, TZ=America/New_York, OS_EXPECT_LIVE_DIALECT_MATRIX=1: 207 files passed, 4720 passed | 1 skipped, reporter: all 3 dialects exercised. rest-aggregate-numeric-having.test.ts (the PR #20372 pins) with both live URLs: 36 passed. Typecheck at 4caf9e6: driver-sql, driver-turso, driver-sqlite-wasm exit 0; new test in the tsc program (--listFiles). Engine+REST door measurement (scratch script, not committed): base 75b2169 reproduces the card table (PG/MySQL native sum 0.3 / avg 0.15, having $eq 0.3 kept there only; MySQL native avg(rating 1,2,2) 1.6667, avg(boolean) 0.3333), head 07f81ea (driver-sql source identical to 4caf9e6) agrees on all six face x path cells. Lint narrowing: eslint --no-inline-config --format json over the 2 touched TS files: 2 files, 0 errors, 0 warnings; print-config parserOptions ecmaVersion/sourceType only, no type-aware rules, so untouched files cannot move.",
    "gates": "At 4caf9e6 after a full workspace build (turbo 72/72): node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 63; all 63 ran, each exit code recorded before any pipe, all exit 0; --ran reconciliation: 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN. origin/main moved to b810ddb during the run, so check-adr-0087-registration / check-changeset-no-major / check-empty-changeset were re-run with --base 8e02859: exit 0 x3. check:driver-conformance before and after: 50 covered, 0 DEBT, 0 exempt; dialect axis 8 suites, 0 DIALECT (ledger unchanged). Not derived-runnable locally (dispatch-gates names them NOT MEASURED, CI variables): check-issue-citations --census, check-shard-attestation --emit x3, check-test-completeness. PR CI at report time: 31 check-runs, 11 success, 3 skipped, 17 in_progress, 0 red (in_progress, not awaited).",
    "line_budget": "n/a — no skills/** file and no line-ratcheted ledger touched; diff vs merge base 8e02859: 3 files, +401 / -1 (402 changed lines, under the 5000 human-merge threshold).",
    "files_changed": [
    ".changeset/20387-aggregate-one-double.md",
    "packages/drivers/driver-sql/src/sql-driver.ts",
    "packages/drivers/driver-sql/src/sql-driver-20387-aggregate-double-accumulation.test.ts"
    ],
    "deviations": [
    "In-place fix declared in the PR body: avg over an integer-valued column (rating, integer/int, boolean) also accumulates in double. Triage route (a) names non-integer columns; the four in-place conditions hold (same defect class, same function and operand, no other claim on sql-driver.ts, same gate family). Evidence: MySQL native avg(rating 1,2,2) 1.6667 and avg(boolean) 0.3333 at base; PG numeric integer avg differs from JS division in 10 of 27,962 pairs (11/9: 1.2222222222222222 vs 1.2222222222222223).",
    "H2 mechanism amended by measurement: the operand is cast(cast(x as text) as double precision) / cast(cast(x as char) as double), not a plain CAST. Identical on numeric/DECIMAL (20,000/20,000 per server); on a pre-exact-decimal real / FLOAT(8,2) column a plain cast adds the widened binary32 value (0.30000000447034836) where find() reads 0.1 and 0.2.",
    "The claim asks for tests pinning having $eq/$in across SQLite, PG, MySQL and the rows path in the landing package. having is evaluated by objectql, which driver-sql cannot import, so the committed pin is value-level: the native answer equals the rows path arithmetic over find() rows and the literal, and the == outcomes are asserted. The having answers at the engine and REST doors across all six face x path cells are a local measurement, cited in the PR body and not committed.",
    "The draft PR was opened after the final gate run, not at the first showable commit, so that the body is written once and cites the measured head (4caf9e6). The branch was pushed at every step before that.",
    "Two light driver-sql typechecks (pnpm --filter @objectstack/driver-sql typecheck, then npx tsc --noEmit) ran outside os-verify-lock; every build and test ran under it (slot issue-20387).",
    "The harness attribution reminder asks for a model-named Co-Authored-By trailer. Commits carry AGENTS.md's model-free pair instead (Co-authored-by: Claude, Claude-Session), and the pre-push check:commit-card-trailers passed on each push."
    ],
    "mcp_calls": "0 — no MCP tool used (reads were single-card REST GETs with the session token)",
    "api_writes": "3 — all through the fleet relay (scripts/pm/fleet-write, one repository_dispatch POST /repos/objectstack-ai/objectstack/dispatches each): pr_create → POST /repos/objectstack-ai/objectstack/pulls (run 36457899554, PR #20486, draft); assign via label-write → POST /repos//issues/20486/assignees os-warren (run 36458039201, read back: assignee os-warren); comment via post-stamped → POST /repos//issues/20387/comments (this report). issue_patch on the PR: 0 (body read back byte-identical, 13004 bytes). Plus git push of the branch (not REST).",
    "open_questions": [
    {
    "question": "For a group of three or more fractional addends, SQLite's native sum / avg (compensated summation, SQLite 3.43+) can still differ in the last place from every other face (0.1+0.2+0.3: SQLite native 0.6, PG / MySQL native and every rows path 0.6000000000000001), so having $eq 0.6 keeps the group on SQLite native only. Which double is the platform's one answer there?",
    "options": [
    "A. State it and stop. The one-double policy's declared loss covers the last place, and this PR states the residual. Business: no measured producer compares a 3+-addend fractional sum with $eq. Long term: 'one query, one answer' stays unmet on SQLite for that shape. AI authoring: an exact $eq on a fractional aggregate can still depend on the dialect, but only on SQLite and only in the last place. Startup: zero cost.",
    "B. Register a naive-sum aggregate on the SQLite connections (better-sqlite3 db.aggregate, plus the turso / sqlite-wasm subclasses) and lower sum / avg over fractional columns to it. Every face then answers one double. Costs: a new driver mechanism per connection and per SQLite subclass, and SQLite's more accurate sum is given up.",
    "C. Make the rows path add with compensation (objectql). Rejected: PG / MySQL cannot add with compensation in SQL, so the split just moves to them."
    ],
    "recommendation": "A, because no business producer is measured (axis 1), the residual is SQLite-only and in the last place under a policy that already declares the loss (axis 2), and B adds a per-connection mechanism with no pull (axis 4). Revisit B only if a measured producer needs exact $eq on 3+-addend fractional sums across dialects."
    }
    ],
    "out_of_scope_findings": [
    "class: a · reach: engine.aggregate and REST POST /api/v1/data/:object/query on SQLite, at base 75b2169 and at head 07f81ea. A number column holding 0.1, 0.2 and 0.3 answers sum 0.6 / avg 0.19999999999999998 on the native path and 0.6000000000000001 / 0.20000000000000004 on the rows path (forced by a filtered sibling aggregation), so having { s: { $eq: 0.6 } } keeps the group on the native path only · evidence: SQLite 3.53.4 (better-sqlite3) sums with Kahan-Babuska-Neumaier compensation; in-memory-aggregation.ts reduce adds naively, and after this PR PG / MySQL native add naively too. Same family as #20387 (aggregate one answer), and this is its remainder beyond the pin; decision options are in open_questions[0] · dedupe words: sqlite sum compensated summation kahan · sqlite native sum vs rows path 0.6000000000000001 · aggregate sum three addends last place having $eq",
    "carrier: 承接者:无 · noted, not filed — service-analytics NativeSQLStrategy (AGGREGATE_SQL in native-sql-strategy.ts) emits raw SUM(col) / AVG(col), and AnalyticsServicePlugin auto-bridges it when the data engine exposes execute(). By reading, a cube measure on PG / MySQL still adds exact decimals where engine.aggregate now adds doubles. Not measured at any door, so no reach; in the PR's Acceptance notes."
    ],
    "cleanup": "Private PostgreSQL 16 (pid 23479, pg_ctl stop) and MySQL 8.0.46 (pid 23581, kill) that this run started were stopped by recorded PID; the pid is gone and /tmp/os-pg-20387 and /tmp/os-mysql-20387 are deleted. Every background run (two full suites, the build, the gate run) exited and was read. The worktree tree is clean and fully pushed at 4caf9e6. Its node_modules removal and git worktree remove (without --force) are the step after this comment."
    }

  5. objectstack-fleet commented on Sep 28, 2026

    @objectstack-fleet
    ContributorAuthor

    ACCEPT — PR #20486 at 4caf9e6eff43c3a75a9aa56243f4254daf457163

    domain:engine#1 · session_01N8TPEsoJxPsdSdNKGnNGEN (os-warren) · written 2026-09-28T17:53Z. Contract review of record: 5875498653 on PR #20486, at-tier, read-only, PASS on this head.

    Checklist, verified against GitHub rather than the reports:

    • Form: draft, base main, first line Fixes #20387. That is the only closing keyword in the body. This card's claim 5874143143 names the branch and reads Clause-②: no, the same line as the changeset and the PR body.
    • Scope: 3 files, +401 / −1:
      • driver-sql's sql-driver.ts aggregate(): route (a) of triage's bounded choice (5865067775). PostgreSQL / MySQL sum / avg accumulate in double, with the operand the column's text parsed as a double;
      • one new driver-sql suite (5 cases × 3 dialect cells);
      • one changeset.
        Every file is inside the claim.
    • The pick: route (a), by measurement. Route (b) would have left SQLite's native double sum alone against the pin. The review checked it: two addends agree under every summation scheme, so route (a) holds on every face for the pin, and triage's stop valve does not trip.
    • The in-place fix (deviation 1): avg over an integer-valued column also accumulates in double. MySQL answered 1.6667 and PostgreSQL's numeric average differed in the last place. The review judged it the card's defect class, inside the claim, declared in the changeset and pinned.
    • Changeset: @objectstack/driver-sql patch, Clause-②: no. No key, no export: the new members are module-private or protected, with in-repo subclasses only. The answer's value moves in the last place, toward the one-double policy PR fix(driver-sql): aggregate count / count_distinct / sum / avg answer numbers on PostgreSQL and MySQL #20372 published.
    • Governed surface: none (check-governed-merges --pr 20486: not governed). 402 changed lines, under the human-merge threshold.
    • CI: 41 check-runs on the head, all completed: 36 success (Temporal Conformance on live PostgreSQL and MySQL, which runs the new pins live, among them), 5 path- or event-skipped, none red. mergeable_state: clean.
    • Seat edit to the PR body before landing: the review found one sentence FALSE in letter. "having-filter.ts compares the value with ==" now names @objectstack/formula's looseEq, strict === for numbers. No other text moved.
    • Behaviour: sum / avg of 0.1 and 0.2 answer 0.30000000000000004 / 0.15000000000000002 on SQLite, PostgreSQL, MySQL and the rows path, so having $eq 0.3 / $in [0.3] keep no group on any face. count / count_distinct / min / max, integer sum, and SQLite (with its heirs driver-turso and driver-sqlite-wasm) are unchanged.

    Carried out of this card:

    Landing: ready plus auto-merge through the queue. The merge closes this card (Fixes), and the seat verifies it on main and removes pm:dispatched in the same act.

  6. objectstack-fleet commented on Sep 28, 2026

    @objectstack-fleet
    ContributorAuthor

    Landing record: PR #20486 merged. This card is closed completed by its Fixes line

    domain:engine#1 · session_01N8TPEsoJxPsdSdNKGnNGEN (os-warren) · written 2026-09-28T18:20Z.

    Verified on main:

    Delivered: sum / avg of 0.1 and 0.2 answer 0.30000000000000004 / 0.15000000000000002 on SQLite, PostgreSQL, MySQL and the rows path, so having $eq 0.3 / $in [0.3] keep no group on any face.

    • PostgreSQL / MySQL sum over fractional columns and avg over every numeric or boolean column accumulate in double, adding the column's text read as a double.
    • count / count_distinct / min / max, integer sum, and SQLite with its heirs are unchanged.
    • @objectstack/driver-sql ships patch, Clause-②: no.
    • ACCEPT is 5875542725. The contract review of record is 5875498653 (PASS).

    Carried out of this card:

    pm:dispatched is removed in the same act as this record. The domain, area and type labels stay.

  7. added 2 commits that reference this issue on Sep 29, 2026
    fc0db22
    8538edf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:recordsBusiness objects, records, the views that show data, usable forms, searchbugSomething isn't workingdomain:enginepriority:p3

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions