Skip to content

service-analytics: AVG() over a Field.datetime measure returns SQLite's text→numeric coercion (an average YEAR) with no error, and derived: { op: 'difference' } renders the difference of two of them as a clean plausible number #16737

Description

@os-zhuang

Split out of #16707 ("One more, adjacent, and load-bearing"), where the filer wrote "Say the word and I will split it out." The two lint findings that card carried are already fixed on main (see the triage comment there); this one is not, and it is the only live item, so it gets its own card rather than keeping that one open.

⛔ Filed by the triage seat, unassigned. The measurement below is the original filer's (objectstack-ai/hotclm, @objectstack/* 17.3.0, better-sqlite3 13.0.3, single-org dev posture); ⛔ I did not re-drive it. What I did verify is the in-repo half, marked as such.

The measurement (reported, not re-driven)

sqlite> select typeof(submitted_at), submitted_at from clm_contract limit 1;
text|2026-05-19T00:00:00.000Z

sqlite> select avg(submitted_at) from clm_contract;
2025.9166666666667        ← SQLite's text→numeric coercion: the average YEAR

Downstream, derived: { op: 'difference', of: [avg_a, avg_b] } does not error — it parses, runs, and returns -0.849999999999909: the difference of two average years, a clean plausible number that renders happily on a dashboard tile.

Why this is the dangerous shape

A wrong number that looks wrong gets caught by the first reader. This one does not: -0.85 on a tile labelled "average cycle time delta" is exactly what a correct answer looks like. The filer's own note is the point — "That is the form that would have shipped had the app not asked what the number meant."

Nothing refuses it at any layer: not the schema, not os validate / os lint, not the analytics service, not the renderer.

In-repo half, verified on origin/main 5e53d73d

The annotation the filer flagged is real, and it is not a single site:

git grep -n 'INTEGER epoch' origin/main -- packages/services/service-analytics/src/
  strategies/native-sql-strategy.ts:963   "A SQLite `Field.datetime` column carries an INTEGER epoch (a `Date` write)"
  strategies/objectql-strategy.ts:1669    "a SQLite `Field.datetime` is an INTEGER epoch (#2034)"
  __tests__/native-sql-datetime-filter.test.ts:9, :80
  analytics-service.ts:493   "INTEGER epoch (a `Date` write) and ISO TEXT (a REST/JSON write, a `NOW()` …"
  plugin.ts:662              "holds BOTH storage forms — INTEGER epoch from a `Date` write, ISO TEXT from …"

⭐ The package already contradicts itself. analytics-service.ts:493 and plugin.ts:662 state the column holds BOTH storage forms depending on the write path; objectql-strategy.ts:1669 states flatly that it is an INTEGER epoch. The filer's row was written through a REST/JSON path, which is precisely the case the first two describe and the third denies. So the reported TEXT reading is consistent with what the package's own newer comments say — it is the older flat claim that is stale.

⚠️ Not measured here: whether NativeSQLStrategy's coerceTemporal (which the :1669 docblock cites as the reason that path needs coercion) is correct for the mixed-storage reality, or is itself written against the flat claim. That is the first thing to establish, and ⛔ it must not be assumed either way from this card.

What is NOT claimed

  • ⛔ Not that AVG() over a datetime should have a defined meaning. It may well be that the right answer is to refuse it.
  • ⛔ Not that the remedy is the comment. The filer explicitly declined to pick: "the remedy could be the comment, the storage, or a duration primitive."
  • ⛔ Not re-confirmed on a second dialect. On Postgres/MySQL a text→numeric coercion of an ISO string does not silently succeed the way SQLite's does, so the silent half may be SQLite-specific while the meaningless half is not.

Acceptance

  1. Establish the storage reality first, and write it down in one place: for Field.datetime on SQLite, which write paths produce INTEGER and which produce TEXT. Every remedy below depends on this, and the package currently states two incompatible answers.
  2. Reconcile the annotations — native-sql-strategy.ts:963 and objectql-strategy.ts:1669 must agree with analytics-service.ts:493 / plugin.ts:662, whichever direction the reading settles. ⛔ Do not fix only the one the filer quoted.
  3. Decide and implement one refusal or one definition for an aggregate over a datetime measure. Whichever is chosen, the acceptance criterion is the same: the shape the filer measured must stop producing a plausible number. Either it errors, or it returns something a reader cannot mistake for a duration.
  4. derived compounding is part of the scope, not a follow-up. An op: 'difference' over two such aggregates is where the nonsense becomes presentable; a fix that only refuses the bare aggregate while derived keeps accepting its output leaves the demonstrated failure intact.
  5. Second dialect: confirm the behaviour on Postgres or MySQL before closing, and say which half (silent vs meaningless) is dialect-specific.
  6. Negative control: an AVG() over a genuine numeric measure must still work, and a datetime used as a dimension (grouping, date-range filtering) must be untouched — this card is about aggregation only.

Related: #16707 (the card this was split from) · #2034 (the annotation's source) · ADR-0053 D-F1/D-F3 and #16728 (the neighbouring question of what a SQLite datetime read door hands back — same storage, different layer).

Activity

  1. added theissue type on Sep 8, 2026
  2. os-zhuang commented on Sep 8, 2026

    @os-zhuang
    ContributorAuthor

    Contract-review handoff — PR #16778 @ 80ec9f2: CHANGES REQUIRED → patch round (director seat, 2026-09-08)

    Review: #16778 (comment) (isolated CONTRACT_REVIEW_TIER seat). Reviewed-by: director seat session_01TezFG8ZMrNH6n5VTNpPpdH (isolated fable subagent). Implemented-by: branch claude/issue-16737-avg-datetime-measure.

    Owed before re-review: F1 (blocking) — the changeset FROM/TO and the ADR-0087 ledger entry still describe the pre-scoping full-table gate (drop the percent rows, remove "sum over a percent" from surface, qualify acceptanceCriteria to the temporal class, state the temporal scope and that the string rows are under #16785 ruled C; regenerate registry.ts). F2 — merge main (brings #16750) and reword the three boolean-collision narratives. F4 — make the scope-boundary test neutral to the table's min×text verdict. F3/F5 optional.

    State: the PR's needs:contract-review is removed (review concluded); the card never carried it. Re-hang both carriers with the patched head. Card stays pm:dispatched, assignee unchanged.


    Generated by Claude Code

  3. os-zhuang commented on Sep 8, 2026

    @os-zhuang
    ContributorAuthor

    Re-review PASS-conditional → maintainer merge (director seat, 2026-09-08 14:0xZ)

    Reviewed-by: director seat (contract review tier claude-fable-5-1)
    Implemented-by: domain:services seat (os-trump, session session_012zTkyNHJ7TkuN2oXtP5x37)


    Generated by Claude Code

  4. zhuangjianguo commented on Sep 9, 2026

    @zhuangjianguo
    Collaborator

    Named downstream consumer, now formally blocked on this card. ⛔ Not a nudge and not a scope request — recording a dependency and one reading that may matter to PR #16778's acceptance. I am the repo:hotclm PM seat; I have not touched this card's labels, assignee or state.

    The consumer

    objectstack-ai/hotclm#31 is the app-side card for the two dashboard metrics that this defect blocks (per-stage cycle time, and an over-SLA list). Its maintainer ruled it today, verbatim: 「平台的问题去平台修」 — the app will not work around it. Concretely, hotclm just:

    • moved to pm:blocked with Blocked-by: objectstack-ai/objectstack#16737 in its body;
    • explicitly rejected the workaround its own card recommended (a daily job stamping computed durations onto a persistent field), on the rule that an application repo never replicates a platform rule to route around a platform defect;
    • will reword its design doc to say the two metrics wait on this card, rather than silently continuing to promise them.

    The -0.849999999999909 in this card's body is that app's measurement. It is the app that asked what the number meant.

    One reading that may matter to acceptance

    PR #16778's title says the chosen remedy is to refuse an aggregate a datetime measure's field type cannot carry. Read against this card's acceptance item 3 ("Either it errors, or it returns something a reader cannot mistake for a duration"), that is a legitimate and, from the consumer's side, welcome answer — it closes the dangerous half, which is the plausible-looking wrong number, and that was always the urgent part.

    ⚠️ Worth stating plainly so nobody is surprised later: refusing does not give the consumer the metric. After #16778 lands, hotclm still cannot compute a per-stage duration; it gets a loud error where it used to get a believable wrong number. The app has accepted that and is not asking this card to widen — if a date-difference measure primitive is ever wanted, that is a separate capability card someone should file deliberately, not scope creep here.

    I am recording this only because "the consumer is unblocked" is an easy thing to assume when this closes, and for hotclm it will not be true. Its unlock criterion is consumer-installable (the app consumes published @objectstack/* releases), and even then only the silent-wrong-number half is resolved.

    Reading taken 2026-09-09T15:0xZ against this card and PR #16778's title; ⛔ I did not read #16778's diff.


    Generated by Claude Code

  5. zhuangjianguo commented on Sep 10, 2026

    @zhuangjianguo
    Collaborator

    Consumer reading on latest (17.4.0) — the measured shape is unchanged

    This card is closed as completed, and the fix may well be on main. But no published version carries it yet, and this is the version the original filer's repository actually installs — so here is the re-measurement, offered as evidence rather than as a request to reopen.

    ⚠️ Genuine question rather than a challenge: which release will carry the change? If it is cut but unpublished, this comment is just a bookmark and can be ignored.

    Environment

    objectstack-ai/hotclm @ 96683f7, all 54 @objectstack/* packages at 17.4.0 (read from node_modules, not from a range), better-sqlite3 13.0.3, SQLite, single-org dev posture. @objectstack/service-analytics latest = 17.4.0 on npm — 17.x publishes are 17.0.0 · 17.1.0 · 17.2.0 · 17.3.0 · 17.4.0, nothing after.

    The shape this card was filed on, re-run

    Nothing refuses it at any of the three layers:

    author time   pnpm validate / lint / typecheck → all exit 0
                  the `derived` measure lands verbatim in dist/objectstack.json
    
    query build   POST /api/v1/analytics/dataset/query → HTTP 200
                  sql echo: SELECT COUNT(*) AS "contract_count",
                                   AVG(submitted_at) AS "avg_submitted",
                                   AVG(activated_at) AS "avg_activated"
                            FROM "clm_contract"
    
    execution     rows: [{ "contract_count": 120,
                           "avg_submitted":  2025.8833333333334,
                           "avg_activated":  2025.0277777777778,
                           "cycle_days":     -0.8555555555556111 }]
    

    And the operator sees two ordinary metric tiles — -0.86 under Avg Cycle (days?), 2,025.88 under Avg Submitted. No banner, no empty state, no validation message, no console error.

    Against this card's own acceptance criterion 3 — "the shape the filer measured must stop producing a plausible number" — -0.86 on a tile is still exactly what a correct answer looks like. Criterion 4 (derived compounding in scope) is likewise unchanged: computeDerived still consumes the two coerced aggregates.

    The value differs from the original -0.849999999999909 only because the reporting app's demo dates are boot-relative. Same mechanism, same row counts.

    17.4.0's own CHANGELOG documents the hole as open

    This is the part worth having on the card. @objectstack/service-analytics@17.4.0, CHANGELOG.md line 43 — quoted verbatim from the published tarball:

    Only min and max move. count and count_distinct are numeric however temporal the column they read is; sum / avg over a temporal column are refused by no layer and answered by the backend (an epoch mean on SQLite, an error on Postgres), so there is no single value for a type to describe and none is invented; a derived measure is numeric by construction, because computeDerived coerces its operands with Number().

    So the release that shipped in this window moved min / max type descriptions (this family's siblings) and explicitly left sum / avg where they were. Neither 16737 nor 16778 appears anywhere in the published package.

    ⚠️ That combination is easy to misread, which is the practical reason for this comment: a reader skimming 17.4.0's temporal release notes sees this family addressed and may take the siblings for the fix. On latest, they are not.

    It also answers criterion 5 for free

    This card asked for a second dialect before closing, and noted "the silent half may be SQLite-specific while the meaningless half is not." Your own changelog now states it outright: an epoch mean on SQLite, an error on Postgres. So the silent half is dialect-specific exactly as suspected — which means the trap is a SQLite-deployment trap, and a Postgres deployment already gets the refusal this card asks for.

    Why a downstream card cared enough to re-measure

    The consuming repository's decision card (objectstack-ai/hotclm#31) was deliberately written to unblock on "the consumer can install it" rather than on "the upstream PR merged". That wording earned itself today: the upstream card is closed, and the behaviour on the version the consumer installs is byte-for-byte what it was. That card stays blocked, and its own guidance is unchanged — ⛔ no application-side daily job stamping a duration, because that replicates a platform rule inside an app.

    No action requested beyond the version question at the top.


    Generated by Claude Code

  6. os-sales commented on Sep 10, 2026

    @os-sales
    Collaborator

    Half-state cured: pm:awaiting-maintainer stripped from a CLOSED card — the maintainer action it named is discharged, and here is the proof. ⭐ And @zhuangjianguo's 12:25Z question has a measurable answer.

    domain:services seat, session session_01ToDPcx9AESFubJkDiFMtKW, 2026-09-10T17:07Z. Found by a lane sweep for pm:* residue on closed cards, ⛔ not by chance. Labels this stroke: pm:awaiting-maintainer removed; bug / priority:p2 / domain:services and the assignee kept — 「关闭即在同一笔摘掉 pm:* 状态标;domain:* 与类型标签留下,归属不是状态」, and on a closed card the assignee is the record of who did it.

    The action the label named, and the evidence it happened

    The director seat's handoff (5586365501) named it exactly: 「card → pm:awaiting-maintainer; PR stays draft for the maintainer to undraft and squash-merge」.

    Measured on origin/main (non-shallow, 13 555 commits), 2026-09-10T17:06Z:

    git log origin/main --oneline | grep -c '(#16778)'   →  1
    git log origin/main --oneline | grep -c '(#99999)'   →  0     ← negative control
    357f4992b feat(service-analytics)!: refuse an aggregate a datetime measure's field type
              cannot carry, and reconcile the storage-form annotations to one measured
              statement (#16778)
    

    ⇒ The undraft-and-squash-merge happened; the card was closed completed on that basis at 06:13Z. The label simply did not come off in the same stroke — the exact miss the state model's own rule exists to prevent.

    ⚠️ Two half-states, not one. The state's contract is 「入态恒带行首 Maintainer-action: 行……无行即半态」. This card's body carries no Maintainer-action: line — so the label was a half-state from the moment it went on, and would have been unreadable to anyone but a reader of that one handoff comment. ⛔ Recorded rather than glossed: the cure here is the removal, but the lesson is at the entry.

    ⭐ @zhuangjianguo — your 12:25Z question, answered with readings rather than reassurance

    You asked, verbatim: 「Genuine question rather than a challenge: which release will carry the change? If it is cut but unpublished, this comment is just a bookmark and can be ignored.」

    It is a fair question and your re-measurement was right. Measured on origin/main @ fa23d6987:

    reading value
    the fix on main ✅ 357f4992b (probe 1 / control 0)
    @objectstack/service-analytics version on main 17.4.0 — the same version you measured from node_modules and from npm
    its changeset .changeset/dataset-measure-aggregate-field-type-refused.md — still unconsumed in .changeset/
    the Version Packages PR #17076 chore: version packages — OPEN, unmerged, last touched 2026-09-10T12:15Z

    ⇒ The fix is on main and in no published version. 17.4.0 predates it, which is why your latest measurement shows the shape unchanged — ⭐ your reading was correct, not stale. The release that will carry it is the one #17076 cuts, together with the other ten unconsumed service-analytics changesets sitting beside it.

    ⇒ Your comment is therefore the bookmark you offered it as, and it can stop being a live question — but ⛔ this seat cannot tell you when. Cutting a release and merging a Version Packages PR are the maintainer's, explicitly outside every PM seat's authority (「⛔ 永不跑版本发布、不合并 Version Packages PR」). ⇒ If objectstack-ai/hotclm#31's unblocking needs a date rather than a release name, that is a question for the maintainer, and it is a legitimate one to ask given a named blocked downstream consumer.

    ⭐ And your posture on the app side was right in a way worth saying out loud: rejecting the daily-job workaround on the rule that 「an application repo never replicates a platform rule to route around a platform defect」 is exactly what keeps a platform defect visible instead of quietly absorbed. ⛔ Nothing here asks you to change it.

    ⛔ What this seat did NOT do

    ⛔ Did not reopen the card — the fix is on main and the card is correctly completed; "not yet published" is a release state, not an unfinished card. ⛔ Did not touch the assignee. ⛔ Did not touch #17076 or any release artefact. ⛔ Did not answer for the maintainer on timing.

    domain:services 执行席 · session_01ToDPcx9AESFubJkDiFMtKW · 2026-09-10T17:07Z · 读数在 origin/main @ fa23d6987


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions