Skip to content

fix(objectql,metadata-protocol)!: the per-aggregation filter takes where's remaining doors — a bad date, an addDays numeric pair, an undeclared { $field } and an unknown key are refused; a Date bound compares as an instant - #20174

Merged
objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/issue-20148-aggregation-filter-doors
Sep 27, 2026

Conversation

@objectstack-fleet

@objectstack-fleet objectstack-fleet Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #20148
Clause-②: no (narrowing)

A per-aggregation filter (aggregations[i].filter) now meets the doors where meets. A bad date, an addDays numeric pair, an undeclared { $field } and an unknown key each answered 200 with the filtered count 0 (every row under a negation). Now each is refused 400, identically on an empty and on a populated table, before any driver call. A Date bound is compared as an instant, as the same bound in a where is.

Head 4da90165cd: a prose-only patch round (contract review 5853139988, seat amendment 5853144038 on #20148) over the reviewed head f22f0e875f. Every code and test blob is unchanged: engine.ts 4e1ebbf2a4, having-filter.ts ab6ba1df00, protocol.ts a9a77c8f6d, and the three test files. Base 49144fccc8. The branch merged main at 369bcbeda7; the merge touched none of these files. Written by session session_01Bvd69VPa6puiNzzPUroDBx, branch claude/issue-20148-aggregation-filter-doors.

commit change
3129311d33 engine loop: temporal door and declared-reference rules; having-filter.ts: reference walk, Date instant comparison; protocol.ts: filter keys through where's gate
d831680d7c pins in objectql, metadata-protocol and rest; two changesets
f22f0e875f merge of main at 369bcbeda7
4da90165cd changeset prose only: the 20148-* wording (see Patch round), and a DELIBERATE CORRECTION of one clause in each of two pending changesets (see that section)

1. Measured first (H1)

A scratch harness (not committed) ran every shape through ObjectQL.aggregate and through POST /api/v1/data/:object/query (RestServer, then findData, then ObjectQL.aggregate). It used a real InMemoryDriver and a real SqlDriver (better-sqlite3 :memory:), both schema-synced, on a populated table (6 rows, groups c1 / c2 / c3) and an empty one, with driver reads counted. The REST body is JSON round-tripped, as the wire carries it. Every per-aggregation shape ran grouped and ungrouped. Beside each ran the same condition as the call's where (the twin), on the aggregate verb and on find. 2896 cells per tree.

Legend: m is the filtered count, over n rows. "Base" and "head" are the per-aggregation filter's answer; the twin is identical at base and head.

# aggregations[i].filter base, memory and sql, engine and REST head the where twin
1 { placed_on: { $gt: 'not-a-date' } } (date) empty m0, populated m0 of 6 INVALID_FILTER / 400, 0 reads, all 8 cells INVALID_FILTER / 400, 0 reads, all 8 cells
1+ the same as an $in member / a $between endpoint / implicit / behind a held $or / under $not; 'noon' on a time populated m1 / m6 / m0 / m6 / m6 / m0 INVALID_FILTER / 400, 0 reads, all 8 cells INVALID_FILTER / 400
1+ 'last_30_days' on a datetime engine m0; REST VALIDATION_FAILED (ingress) engine INVALID_FILTER; REST unchanged the same split
2 { amount: { $gt: { $field: 'cap', addDays: 1 } } } populated m0 INVALID_FILTER / 400, 0 reads, all 8 cells sql INVALID_FILTER / 400; memory n0
2+ addDays date-to-numeric, date-to-datetime, text-to-date, time pair, text or date offset column, a numeric pair behind a held $or m0 (date/datetime m4, held $or m6) INVALID_FILTER / 400, 0 reads sql INVALID_FILTER; memory a count
3 { amount: { $gt: { $field: 'nope' } } } populated m0 INVALID_FILTER / 400, 0 reads, all 8 cells sql INVALID_FILTER / 400; memory n0
3+ under $ne / $not / a held $or; a dotted referent; an undeclared addDays offset column m6 / m6 / m6 / m0 / m3 INVALID_FILTER / 400, 0 reads sql INVALID_FILTER; memory a count
4 { nope: 1 } over REST 200, m0 REST INVALID_FIELD / 400, 0 reads; the engine door is unchanged (a count) REST INVALID_FIELD / 400; engine: sql INVALID_FILTER, memory n0
4+ under $and / $ne / $not / a held $or m0 / m6 / m6 / m6 REST INVALID_FIELD / 400 REST INVALID_FIELD / 400
4+ { nope: { $median: 1 } } INVALID_FILTER everywhere REST INVALID_FIELD (the field gate runs first, as for where); engine INVALID_FILTER the same split
5 { opened_at: { $gt: new Date('2026-02-01') } }, in-process m0 m4 n4 on both drivers

Found beyond the card: the $between and held-$or rows counted every row, not zero. The card's row 5 reads "4 rows on both drivers" for the twin, and the #20122 dev report reads memory 0. Both are right about their harness: driver-memory coerces a Date comparand by the column's storage rule only after syncSchema has indexed the object's temporal fields. Schema-synced, as a booted runtime is, the twin answers 4 on both drivers.

2. What changed

  • packages/objectql/src/engine.ts, the per-aggregation loop of ObjectQL.aggregate.
    • (a) It runs assertTemporalComparandsInterpretable, the function where runs fourth on its seam, fourth here too: after the text-operator door, before the type door.
    • (b) It hands assertAggregationFilterIsEvaluable the object's declared field map, and a sink that logs the withheld diagnostic at warn.
  • packages/objectql/src/having-filter.ts.
    • assertAggregationFilterIsEvaluable(filter, index, declared?): after the walker's own refusals, assertAggregationFilterReferencesAreDeclared walks each scalar-comparison { $field }. The referent, and an addDays offset's nested $field, must name a declared field (the field map plus id, created_at, updated_at: the set the REST field gate reads). An addDays pair must pass the class rule.
    • The class rule is objectql having: a { $field } reference with addDays against a non-temporal aggregated column answers by epoch-ms coercion, where SQL push-down refuses the same pair on where — the aggregated row declares no temporal class to judge it by #20127's, factored out of assertOffsetPairIsTemporal into offsetPairViolation, so having and this position judge one rule. having's words are byte-identical.
    • The words withhold the fields, the operator and the reason, as driver-sql withholds them for the same comparison in where. They name the aggregation (aggregations[1].filter) and the rule, and say the diagnostic is in the server log. The message is sized under the REST envelope's 500-character bound.
    • checkCondition: when a Date sits on either side of an equality, ordering, $between or list arm, and both sides denote an instant (utcInstantMs, @objectstack/spec/data), it compares instants. That is the lift @objectstack/formula's evaluator applies. Every other pair compares exactly as before.
  • packages/metadata-protocol/src/protocol.ts, findData's assertAggregationFieldsExist only. After its entry and field checks, each entry's filter goes through assertFilterFieldsExist, the gate the explicit where meets. It uses the same field set and the same unknown, dotted and virtual ladder, answers INVALID_FIELD / 400, and passes aggregations[i].filter as the parameter.

3. The hypotheses

4. Tests

All at code head f22f0e875f unless stated.

  • pnpm --filter @objectstack/objectql exec vitest run (whole suite, before the merge, at d831680d7c): Test Files 317 passed (317) · Tests 5633 passed (5633).
  • pnpm --filter @objectstack/metadata-protocol exec vitest run (whole suite, at d831680d7c): Test Files 189 passed | 3 skipped (192) · Tests 2699 passed | 19 skipped (2718).
  • typecheck for @objectstack/objectql, @objectstack/metadata-protocol and @objectstack/rest: each exit 0. --listFiles shows each new test file in its package's program. check:test-typecheck for objectql: 40 file(s) / 234 error(s) / 65 pinned, unchanged.
  • REST aggregate tests (aggregation-filter-where-doors, list-view-grouping-query-door, request-schema-gate.conformance, rest-server-canonical-query-ast): 96 passed | 1 skipped.
  • driver-sql aggregate suites (10 files): 125 passed | 10 skipped (live PG / MySQL not run). driver-memory aggregate suites (3 files): 75 passed.
  • @objectstack/service-analytics, the consumer that lowers measure filters here (whole suite): 128 files, 3017 passed.
  • After the merge: the objectql aggregate / having files, the two protocol files and the four REST files were green again.
  • Reverse verification. The fix was committed first. The three source files were restored to their base blobs (bc4f337220, 876293a1e9, ba5fbccde6) under an EXIT / INT / TERM trap. On-disk hashes and marker counts (0 / 0 / 0) were checked. objectql and metadata-protocol were force-rebuilt, and ablation-dist-preflight --absent passed for both.
    • The new pins then read: engine-aggregate-filter.test.ts 34 failed of 107, protocol.aggregation-filter-fields.test.ts 7 of 13, aggregation-filter-where-doors.test.ts 10 of 15.
    • Those are exactly the new refusal and Date rows. The controls, the walker-first row and the "not refused by the engine" control stayed green.
    • Restored with git checkout HEAD --. Proven by HEAD-blob equality for all three and an empty whole-tree git status. dist/ was rebuilt from HEAD and the preflight found the markers present.

5. Gates

  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at f22f0e875f: 65 commands, 65 run.
  • --ran with the exit codes: 65 derived famil(ies) accounted for — 65 run, 0 NOT-MEASURED.
  • check:dual-build-cjs-loads and check:type-check-debt first answered PREREQUISITE NOT MET (3). After turbo run build --filter='./packages/*' --filter='./packages/*/*', both exit 0.
  • node scripts/check-issue-citations.mjs --base 369bcbeda7: 20 citations, all resolve.
  • pnpm check:nul-bytes: exit 0. A control-byte self-scan of the 8 changed files found none.
  • Lint, a declared narrowing:
    • eslint --no-inline-config --format json on the 6 changed .ts files: 6 files, 0 errors, 0 warnings, 0 fatal.
    • --print-config returns a config for each.
    • eslint.config.mjs enables no type-aware linting (no parserOptions.project), so this diff cannot move a verdict on an untouched file.

Patch round (contract review 5853139988)

Prose only; every code and test blob is identical to f22f0e875f (git diff --name-only f22f0e875f 4da90165cd lists three .changeset/ files and nothing else).

  • .changeset/20148-aggregation-filter-where-doors.md (git diff --numstat: 2 2):
    • The having Date sentence is qualified. It now reads that a Date bound keeps the groups its ISO spelling keeps on a datetime column. On a date-class column (min / max of a date field), a UTC-midnight Date follows the calendar-day reading a where gives ({ $gte: new Date('2026-02-01') } keeps the group whose max is 2026-02-01, and so does $eq), which the ISO-instant text, compared as text, did not.
    • "refused in the words driver-sql uses" now reads "refused in the withholding posture driver-sql applies". The sentence on the wire is new; only the posture is borrowed.
  • Gates at 4da90165cd:
    • check-empty-changeset --base origin/main: exit 1. It names exactly .changeset/20099-having-where-doors.md and .changeset/20127-having-adddays-temporal-pair.md, each "present on the merge base and CHANGED by this PR". This is by design; see the next section.
    • check-adr-0087-registration: "4 declared-breaking changeset(s), each carrying an ADR-0087 disposition" with --base 49144fccc8, and 2 with --base origin/main. Exit 0.
    • check-changeset-no-major --base 49144fccc8 --event (this body, as a pull_request payload): exit 0.
    • check-issue-citations --base 369bcbeda7: 20 citations, all resolve. Exit 0.

DELIBERATE CORRECTION

Two PENDING changesets carried one clause each that is false at this head. Each file's git diff --numstat is 1 1, and every other line is byte-identical. check-empty-changeset refuses both by name, which is the gate's own path for this class: the correction is said here, for confirmation.

Acceptance notes

Deviations

  • The claim names assertAggregationFilterIsEvaluable and checkCondition in having-filter.ts. This PR also adds module-level helpers there: the reference walk, the withheld error, the instant helpers, and the AggregationFilterDeclaration interface. That interface is exported at module level but package-private, because having-filter.ts is not re-exported by packages/objectql/src/index.ts. The PR also refactors assertOffsetPairIsTemporal into offsetPairViolation plus a wrapper so both positions share one rule, adds one import, and adds a paragraph to the header comment. having's cells are byte-identical (H3).

…s remaining doors

The per-aggregation filter now runs the temporal-comparand door, judges a
{ $field } referent and an addDays pair against the object's declared
fields (withheld words, as where gets for the same comparison), compares a
Date bound as an instant, and the REST door judges its keys with where's
unknown-field gate.

Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx
Co-authored-by: Claude <noreply@anthropic.com>
…'s where doors; changesets

Engine pins (both driver kinds, empty and populated, grouped and ungrouped,
zero driver calls) for the temporal door, the declared-referent and addDays
class rules with their withheld words and logged diagnostic, and the Date
instant comparison; a findData pin for the filter-key gate; and a REST pin
over a real SqlDriver with the where twin beside every row.

Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/xl documentation Improvements or additions to documentation tests tooling labels Sep 27, 2026
@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/metadata-protocol, @objectstack/objectql, touching 18 documentable anchor(s).

17 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: node scripts/docs-audit/affected-docs.mjs --json 3bd28e2b2ea81206dc9f5507de205b564bd97097.

⛔ 4 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails.

What this run could not see
  • 1 anchor(s) matched too much of the corpus to be a work list: created_at (literal, 34 pages)
  • 3 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 54 of 207 client-bound route-ledger rows — the other 153 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 153: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 98 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 24 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 3bd28e2b2ea81206dc9f5507de205b564bd97097 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 084b27869487861f3f0eafb801a7292c6bd6fca0 — the merge of head 4da90165cd7deebc159117dfb9c2b5b26e26a9a3 into base 3bd28e2b2ea81206dc9f5507de205b564bd97097, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 084b27869487861f3f0eafb801a7292c6bd6fca0 && git checkout 084b27869487861f3f0eafb801a7292c6bd6fca0
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 3bd28e2b2ea81206dc9f5507de205b564bd97097 4da90165cd7deebc159117dfb9c2b5b26e26a9a3 && git checkout -B drift-repro 3bd28e2b2ea81206dc9f5507de205b564bd97097 && git merge --no-ff 4da90165cd7deebc159117dfb9c2b5b26e26a9a3

node scripts/docs-audit/affected-docs.mjs --json 3bd28e2b2ea81206dc9f5507de205b564bd97097

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 3bd28e2b2ea81206dc9f5507de205b564bd97097 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: f22f0e875f54c812b39af25c31ebfda8fd5a44b9

Scope: PR #20174 (card #20148, priority:p2, class-closure, 5 pin rows). 8 files, +972/−29, all in the PR's own two commits (3129311d33, d831680d7c); the merge f22f0e875f brings main at 369bcbeda7 and no main commit in 49144fccc8..e2c4e125f9 touches any of the 8 paths (blobs engine.ts bc4f337220→4e1ebbf2a4, having-filter.ts 876293a1e9→ab6ba1df00, protocol.ts ba5fbccde6→a9a77c8f6d; main's copies of all three are still the base blobs). Measured, not inferred: detached worktrees of head and base (49144fccc8) with the rest / driver-sql / driver-memory closures built (dist markers verified head 1/2/4, base 0/0/0), my own harness on a real InMemoryDriver and a real SqlDriver (better-sqlite3 :memory:), schema-synced, populated (6 rows, c1/c2/c3) and empty, through engine.aggregate / engine.find and POST /api/v1/data/:object/query (JSON round-tripped), driver reads counted, engine warn captured: 3136 cells per tree; 2624 identical base→head, 512 moved — 460 per-aggregation cells, 52 having cells, 0 where twins, 0 groupBy. Inputs: card + comments 5834285763 / 5852472098 / 5852943342 / 5852957006, #20122 / #20123 / #20127 / PR #20147, #20099 / PR #20117, #7534, #7929 / #8220, PR body, diff, check-runs. Dev's scratch not read.

① Derived judgments

② Semver level

③ Boundary flags

Implemented-by: claude/issue-20148-aggregation-filter-doors
Reviewed-by: session_01Bvd69VPa6puiNzzPUroDBx

VERDICT: FAIL

Must-change (one file, prose only; code, tests and gates need no change):

  1. .changeset/20148-aggregation-filter-where-doors.md, the "Not refused, but answering differently" bullet: qualify "having shares this comparison, so a Date bound in having keeps the groups its ISO spelling keeps" so it is true for what was measured — e.g. "…keeps the groups its ISO spelling keeps on a datetime column; on a date-class column (min / max of a date field) a UTC-midnight Date follows the calendar-day reading a where gives ({ $gte: Date('2026-02-01') } keeps the group whose max is 2026-02-01), which the ISO-instant text compared as text did not." In the same edit, replace "refused in the words driver-sql uses for the same comparison in a where" with the accurate "refused in the withholding posture driver-sql applies to the same comparison in a where" (the sentence is new; the posture is borrowed).

Optional in the same push (PR body, not shipped prose): H4 "reports 2" → the at-head reading (4, two of them main's); H3 "except" clause → name both max(date) midnight shapes ($gte and $eq).

…withholding posture, and correct two pending changesets' clauses

The #20148 changeset's having sentence holds on a datetime column only; on a
date-class column a UTC-midnight Date follows the calendar-day reading. Its
refusal borrows driver-sql's withholding posture, not its words. Two pending
changesets carry one clause each that is no longer true: #20127's says the
per-aggregation filter is not judged by the addDays class rule (it is, since
this card), and #20099's says an unknown having key keeps no group (it is
refused, since #20123). Every other line is byte-identical.

Claude-Session: https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 4da90165cd7deebc159117dfb9c2b5b26e26a9a3

Scope: delta review of PR #20174 (card #20148) after FAIL 5853139988 at f22f0e875f, under seat amendment 2 (5853144038: file surface widened by two pending changesets for one-clause DELIBERATE CORRECTIONS). This record carries forward my measured code judgments from the review of f22f0e875f on the identical blobs: git diff-tree -r f22f0e875f 4da90165cd lists exactly three paths, all under .changeset/; every non-changeset file is blob-identical (engine.ts 4e1ebbf2a4, having-filter.ts ab6ba1df00, protocol.ts a9a77c8f6d, engine-aggregate-filter.test.ts f9700d89a4, protocol.aggregation-filter-fields.test.ts 13a6e9a0ef, aggregation-filter-where-doors.test.ts e98e39a758, 20148-aggregation-filter-keys-rest.md 4d50a11e34). git diff --numstat f22f0e875f 4da90165cd is exactly 1 1 .changeset/20099-having-where-doors.md, 1 1 .changeset/20127-having-adddays-temporal-pair.md, 2 2 .changeset/20148-aggregation-filter-where-doors.md — the dev's claim holds; in each foreign file the single changed line is the paragraph named below and every other line is byte-identical (unified diff shows one -/+ pair per file). The branch did not re-merge main: merge-base with origin/main is still 369bcbeda7; origin/main is now 455dcc060d (7 commits ahead), and none of those 7 commits touches any of the PR's 10 paths (git diff --name-only 369bcbeda7 origin/main -- <the 10 paths> = empty). A worktree at 4da90165cd was built (closure cache-hit; dist markers 1/1) and my harness re-run: 3136 cells, 3136 identical to the f22f0e875f run, 0 differ — the code judgments transfer without exception.

① Derived judgments

② Semver level

  • Unchanged from the f22f0e875f record: Clause-②: no (narrowing) correct; @objectstack/objectql minor, @objectstack/metadata-protocol minor (BREAKING narrowings under the launch-window convention); both changesets BREAKING with not-required (no-migration-prescription). The two corrected foreign changesets keep their frontmatter and disposition markers untouched.
  • check-changeset-no-major --base 49144fccc8 --event <final PR body> at 4da90165cd: "✓ This diff introduces no major bump." · "✓ LEVEL AXIS: this PR declares clause-② no (narrowing), and no package whose packages/**/src/** it moves is graded patch." · "direction arm: narrowing — a BREAKING change; during the launch window it ships minor" — exit 0.
  • check-adr-0087-registration --base 49144fccc8: "✓ check-adr-0087-registration: 4 declared-breaking changeset(s), each carrying an ADR-0087 disposition." — exit 0. --base origin/main: 2 (this PR's two) — exit 0.
  • check-empty-changeset --base origin/main: exit 1, by design: "This PR changes a changeset it did not add: .changeset/20099-having-where-doors.md — present on the merge base and CHANGED by this PR …; .changeset/20127-having-adddays-temporal-pair.md — present on the merge base and CHANGED by this PR …" — exactly the two files.
  • CI at 4da90165cd, complete: 41 check runs — 34 success, 5 skipped, 2 failure. All seven required contexts success: Lint & Repo Gates, TypeScript Type Check, Test Core, Dogfood Regression Gate, Build Core, Temporal Conformance (live PG + MySQL), Governed Surface Queue Guard. The two failures are both Check Changeset (jobs 108561828229 on synchronize and 108562075562 on edited), each red for the same reason, from the job log: "This PR changes a changeset it did not add: .changeset/20099-having-where-doors.md — present on the merge base and CHANGED by this PR -- this is somebody else's release note; .changeset/20127-having-adddays-temporal-pair.md — present on the merge base and CHANGED by this PR … ##[error]Process completed with exit code 1." — exactly the two corrected files. No other failure.

③ Boundary flags

Implemented-by: claude/issue-20148-aggregation-filter-doors
Reviewed-by: session_01Bvd69VPa6puiNzzPUroDBx

VERDICT: PASS

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review September 27, 2026 06:19
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Sep 27, 2026
Merged via the queue into main with commit cfe2387 Sep 27, 2026
41 of 43 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-20148-aggregation-filter-doors branch September 27, 2026 06:39
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Sep 28, 2026
…al comparand by the column's storage rule — one rule in core, shared with both drivers' where (objectstack-ai#20202)

Fixes objectstack-ai#20176
Clause-②: yes (widening)

A per-aggregation `filter` (`aggregations[i].filter`) and `having` on
`engine.aggregate` now read a temporal comparand by the column's storage
rule — the rule both drivers already apply to the same comparand in a
`where`. The rule lives in one place, `temporalStorageForm(value, kind)`
in `@objectstack/core`, and `driver-sql` and `driver-memory` call it
instead of each carrying a copy. Every `where` answer is byte-identical
before and after, and so is every refusal PR objectstack-ai#20174 added.

## What changed

- **`@objectstack/core`** — new `utils/temporal-storage-form.ts`,
exported from the index: `temporalStorageForm(value, kind)` for `kind`
`'datetime' | 'date' | 'time'`. It is the body that `driver-sql`'s
`canonicalUtcDatetime` / `toDateOnly` / `canonicalTimeOfDay` and
`driver-memory`'s `storageDatetimeValue` / `storageDateValue` /
`storageTimeValue` each carried word for word. It is total,
operator-blind and does not map arrays. `temporal-comparand.ts` docs now
name it.
- **`driver-sql`** — the three functions' bodies call the core rule;
`storageDatetimeValue` still adds MySQL's literal spelling on top (the
dialect half stays in the driver). No other line moved.
- **`driver-memory`** — `coerceTemporalValue` calls the core rule; the
three private copies are deleted (nothing else imported them).
- **`@objectstack/objectql`**
- `having-filter.ts` `checkCondition` takes the column's temporal kind.
When it has one, the row's value and every comparand of `$eq` / `$ne` /
the four orderings / `$between` / `$in` / `$nin` / implicit equality are
put in the storage form first. A bare `YYYY-MM-DD` as the upper bound of
a `datetime` (`$lte`, a `$between` max) means the whole day — the
exclusive bound at the next day's midnight — from the spec's
`nextUtcCalendarDay` (ADR-0053 D-D), as
`SqlDriver.calendarDayUpperBoundRewrite` and `driver-memory`'s `$lte`
arm do. Presence tests, text operators and `{ $field }` references are
read as before. `matchesHaving`, `applyHaving` and
`matchesAggregationFilter` thread a class map down;
`declaredFieldClasses(fields)` is new (the object's declared fields,
classified by the same spec sets objectstack-ai#20127's `aggregatedRowColumnClasses`
uses).
- `in-memory-aggregation.ts` — `applyInMemoryAggregation(rows, ast,
timezone?, fields?)`: the optional declared field map reaches each
per-aggregation filter. Without it the function answers as before.
- `engine.ts` `ObjectQL.aggregate` — reads the declaration once, keeps
`having`'s aggregated column classes (already computed for objectstack-ai#20127's
`addDays` rule) and hands them to both `applyHaving` doors, and hands
the field map to `applyInMemoryAggregation`.

## The rows, measured

Real `InMemoryDriver` and real `SqlDriver` (better-sqlite3,
schema-synced), through `engine.aggregate` and through `POST
/api/v1/data/:object/query` (JSON round-tripped body), on a populated (6
rows, 3 groups) and an empty object; the per-aggregation filter on the
second of two counts, the `where` twin beside it. Base `cfe2387a3b` vs
head (code identical to `b48f417154`). Every empty-object cell is 0 /
`[]` at base and head.

| # | shape | base, populated (all 4 driver × door cells) | head |
`where` twin |
|:--|:--|:--|:--|:--|
| 1 | ISO instant `$gte` on a `date` | 1 | 3 | 3 |
| 2 | ISO instant `$eq` on a `date` | 0 | 2 | 2 |
| 3 | bare day `$lte` on a `datetime` | 2 | 3 | 3 |
| 4 | bare day as `$between` max on a `datetime` | 2 | 3 | 3 |
| 5 | epoch-ms `$gt` on a `datetime` | 0 | 3 | 3 |
| 6 | `Date` 10:00 on a `date`, `$gte` / `$lt` / `$eq` | 1 / 5 / 0 | 3 /
3 / 2 | 3 / 3 / 2 |
| 7 | `Date` `$gt` on a `time` | 0 | 3 | 3 |
| 8 | `having` ISO instant `$gte` on `max(date)`, both doors | c2 | c2,
c3 | the date rule: c2, c3 |
| control | zone-naive `'2026-02-01 09:00'` `$gt` on a `datetime` | 4 |
4 | 4 |

Rows 6 and 7 are in-process only on the engine door; over REST the
`Date` arrives as its ISO text, and that spelling moved the same way.
Rows 6/7 on the base were identical to the card's readings, so PR
objectstack-ai#20174's instant lift did not move them.

The family H1 asked for, all now equal to their `where` twin on every
cell (base in brackets): `$ne` 4 (6), implicit equality 2 (0), `$in`
members 3 (0), `$nin` members 3 (6), `$between` endpoints 4 (3), ISO
`$lt` 3 (5) on a `date`; bare days as both `$between` endpoints 2 (1),
epoch-ms string 3 (6), epoch-ms `$lte` 3 (0) and `$eq` 1 (0), offset
instant `$gte` 4 (3) and `$eq` 1 (0), zone-naive `T` `$eq` 1 (0), ISO
without ms `$eq` 1 (0), `$ne` 5 (6), `$nin` 5 (6) on a `datetime`; ISO
instant `$gt` 3 (0) and `$eq` 1 (0), short wall clock `$eq` 1 (0),
`$lte` 3 (2), `$in` 2 (0), `$between` 3 (2), `.000` spelling `$eq` 1
(0), bare day `$lte` 0 (6), `Date` implicit 1 (0) on a `time`; `Date`
`$in` 2 (0), `$nin` 4 (6), `$between` 4 (3) on a `date`; the ISO row
under `$or` 3 (1), `$not` 3 (5), `$and` 2 (0); two operators on one
`datetime` 2 (1) and a bare-day `$lte` beside an author `$lt` 4 (3); a
bare-day `$lte` of today on `created_at` 6 (0). `having`, both doors: 15
more shapes on `max(date)`, `min(date)`, `min(datetime)`, `max(time)`, a
`groupBy` of a `date` / `datetime` field and a `day` bucket, each moved
to the date/datetime/time rule's answer.

## H2 — do the two drivers' rules agree?

Measured before lifting: `driver-memory`'s `coerceTemporalValue` against
`SqlDriver.temporalFilterValue` (better-sqlite3, pg and mysql2 clients,
field maps seeded, no connection) on 71 shapes × 3 kinds (`Date`s valid,
invalid and out of range, numbers incl. NaN / Infinity / past the Date
range, epoch strings, bare days incl. impossible ones, zone-naive and
zone-explicit timestamps, RFC 2822, wall clocks in and out of range,
junk, placeholders, `null`, booleans, objects, nested and mixed arrays,
bigint): memory = sqlite = pg on all 213 cells; mysql differs on the 36
`datetime` cells only by its literal spelling. Base to head, all 852
cells are byte-identical, so the re-pointing moved nothing.

One divergence sits in the operator-sensitive layer, not in the lifted
function: a bare day as a `$lte` / `$between` max on a **`time`** field.
`driver-memory` widens it (`$lt '00:00:00'`, no row), `driver-sql` does
not (`$lte '00:00:00'`, keeps a midnight row). Neither driver's emitter
is touched here. This position follows the spec: ADR-0053 D-D's table
defines the whole-day reading for `datetime`, and `TEMPORAL_TIME_CASES`
says it must not reach a time column. Reported, not filed here.

## Tests

- New:
`packages/objectql/src/engine-aggregate-temporal-storage-rule.test.ts`
(84 — the card's rows, the family, unmoved positions, `having` on both
doors, the shared `TEMPORAL_CASES` + token spellings +
`TEMPORAL_TIME_CASES` through a per-aggregation filter and through
`having`, and the no-declaration default);
`packages/rest/src/aggregation-filter-temporal-storage-rule.test.ts`
(13, REST + SqlDriver, per-aggregation count equals the `where` twin,
empty and populated, and `having` row 8);
`packages/drivers/driver-memory/src/memory-temporal-storage-form.test.ts`
(59 — `coerceTemporalValue` is the core rule, and the card's `where`
twins on this driver);
`packages/drivers/driver-sql/src/sql-driver-temporal-storage-form.test.ts`
(10 — `temporalFilterValue` is the core rule on three clients);
`packages/core/src/utils/temporal-storage-form.test.ts` (52).
- Whole suites at `b48f417154` (after merging `main`, full package
build): objectql `317 files / 5712 passed`; core `54 / 1410`;
driver-memory `54 / 1328`; driver-sql `188 passed, 11 skipped / 3099
passed, 170 skipped`; REST aggregate files (5 incl. the new one) `109
passed, 1 skipped`; service-analytics `128 / 3017`.
- Live PostgreSQL 16 (private server, server zone Asia/Shanghai, process
`TZ=America/New_York`): driver-sql whole suite `196 files passed, 3
skipped / 3806 tests passed, 86 skipped`;
`sql-driver-temporal-conformance` 103 with only its MySQL cell skipped.
MySQL: NOT MEASURED, no server in this container.
- Skewed process zone (America/New_York, Asia/Kolkata, Pacific/Chatham):
the new objectql + `engine-aggregate-filter`, driver-memory new +
temporal conformance, core new — all green in each zone.
- Typecheck `core`, `driver-memory`, `driver-sql`, `objectql`, `rest`:
exit 0 each; every new test file is in a typecheck program
(`--listFiles`); objectql test-typecheck debt unchanged at 40 files /
234 errors.
- Lint (declared narrowing): `eslint --no-inline-config --format json`
on the 13 changed `.ts` files → 13 files, 0 errors / 0 warnings / 0
fatal; `--print-config` resolves a config for each; `eslint.config.mjs`
enables no type-aware linting, so no untouched file's verdict can move.

## Reverse verification

Fix committed first. `ablation-replace` (wrap mode, `EXIT/INT/TERM` trap
on the absolute path) replaced the one threading site in `matchesHaving`
so no column ever gets its kind: anchor 1 to 0, blob `77400359cc` to
`7a2f5b7f14`. objectql rebuilt; `ablation-dist-preflight --absent`
passed on all 14 built files. New objectql pins `58 failed / 26 passed
(84)` — the 26 are the controls, the unmoved positions, the objectstack-ai#20148
`Date` lift on an undeclared column and the kit cases whose canonical
comparands compare correctly as text. REST pins `11 failed / 2 passed
(13)` — the two controls. The harness in that state reproduced the base
table 5024 / 5024. Restored by `git checkout HEAD --`: blob back to
`77400359cc` = HEAD, `git diff HEAD` empty; rebuilt; preflight marker
present in 4 built files and the tree clean; pins `84` and `13` passed.

## Gates

`dispatch-gates --commands --repo objectstack-ai/objectstack` at
`b48f417154`: 69 commands (the clue list's 56 plus
`check-adr-0087-registration`, `check-empty-changeset`,
`release-rehearsal-clone --self-test`, `check:engine-double-contract`,
`check:objectql-double-limit`, `check:objectui-changeset`,
`check:pm-changeset-deadline-census`, `check:query-options-erasure`,
`check:type-check-coverage`, `check:type-check-debt`,
`check:where-matcher`). 68 exit 0; `--ran` reconciles 69 of 69, 0 NOT
MEASURED. One is red on purpose:

- `check-empty-changeset --base origin/main` exits 1 on
`.changeset/20148-aggregation-filter-where-doors.md` — a **deliberate
correction**, not a collision, and it needs a person's confirmation
here. That pending note said "Two readings still differ from a `where`"
(a `Date` with a time of day on a `date` field, and a `Date` on a `time`
field). This PR closes both, so the sentence would ship false in the
same release. The one-clause correction reads "Two readings still
differed from a `where` in this change alone … (objectstack-ai#20176 closes both in
the same release, reading every temporal comparand at this position by
the column's storage rule)". Nothing else in the note changed.

Also run: the three roster gates whose roster sits under `packages/`
(`check:authz-resolver`, `check:error-code-casing`,
`check:filter-alias-parity`) and `check:published-readme-exports`, exit
0 each; `check-issue-citations --base 560b724` (the merge base): 28
citations, all resolve; a control-byte scan of the changed files found
none.

## Deviations

- The claim scopes `engine.ts` to "the per-aggregation loop only". The
loop runs before filter tokens resolve, so a comparand normalised there
would miss every `{today}`-style bound, which resolves to a bare day
later. The normalisation therefore happens where the filter is evaluated
(`checkCondition`), and the `engine.ts` edit is the threading: one
declaration read beside the existing `having` class read, and the class
maps passed to both `applyHaving` calls and to
`applyInMemoryAggregation`. `in-memory-aggregation.ts` carries the
per-aggregation filter's class map, not `having`'s (which the engine
hands to `applyHaving` directly).
- The re-pointing in `driver-sql` replaces the bodies of
`canonicalUtcDatetime`, `toDateOnly` and `canonicalTimeOfDay` (which
`temporalFilterValue` reaches through `coerceFilterValue`) rather than
only `temporalFilterValue`, so the driver's write and read paths use the
same single rule. The H2 measurement is the byte-identity evidence.
- The REST `where` twin on `InMemoryDriver` is measured (scratch
harness, 5120 cells) but not committed as a pin: `packages/rest` has no
`driver-memory` dependency. That driver's `where` half is pinned in its
own package.
- `.changeset/20148-aggregation-filter-where-doors.md`: one-clause
deliberate correction, above.

## Acceptance notes

- `driver-mongodb`'s `mongodb-temporal.ts` keeps its own copy of the
rule, in BSON `Date` form for `datetime`. Its instants agree with the
core rule, but it is outside this claim and was not re-pointed.
- `core`'s `temporal-comparand.ts` predicates (`readsAsInstant`,
`readsAsCalendarDay`, `readsAsWallClock`) restate the rule's regexes as
yes/no questions for the temporal-comparand door. Deriving them from
`temporalStorageForm` would remove the last mirror in `core`; not done
here.
- `ObjectQL.resolveNowDefault` spells the `date` / `time` forms of a
`Date` by hand; it agrees with the rule for every `Date` it is handed.
- A per-aggregation filter and `having` on a column whose class the
declaration cannot tell (no field map, a formula) keep the previous
comparison, including PR objectstack-ai#20174's instant reading of a `Date` bound.
- An epoch-ms number against a `date` field is not read by the `date`
rule on either driver, and the two drivers' `where` answer differently
(driver-memory 0 of 6, driver-sql 6 of 6). This position answers 0,
before and after; reported separately.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

2 participants