Skip to content

feat(spec,driver-sql,formula): addDays whole-day offset on a field reference, compiled on SQL and evaluated in memory - #15102

Merged
zhuangjianguo merged 4 commits into
mainfrom
claude/issue-14104-field-reference-add-days
Sep 3, 2026
Merged

zhuangjianguo merged 4 commits into
mainfrom
claude/issue-14104-field-reference-add-days

Conversation

@zhuangjianguo

Copy link
Copy Markdown
Collaborator

Fixes #14104

Clause ②: yes — path limb: packages/spec/src/data/filter.zod.ts (a published accept set widens — FieldReferenceSchema gains addDays, regenerated into the JSON schema, the authorable surface and content/docs/references/data/filter.mdx); content limb: the filter grammar gains a key an author can write, so os validate accepts a shape it refused yesterday and two published runtimes (driver-sql, formula) give it a meaning.

What this lands (maintainer ruling A, 2026-09-02, recorded on #14104)

{ completed_at: { $lte: { $field: 'due_date', addDays: { $field: 'grace_days' } } } } — a whole-day offset on a field reference, so a dataset measure can say "completed by its deadline, where the deadline is a stored date plus a grace period held in another column".

  • Spec — FieldReferenceSchema.addDays: an integer literal of any sign (a negative value subtracts; no subDays, no other unit) or a nested { $field } reference (dot-path allowed, exactly as $field allows). No new position: the reference is legal exactly where a bare one is (the whole comparand of the six scalar comparison operators); $in / $nin / $between keep the list-position refusal. A fractional number, a string, or an object without $field is refused at the schema door with a pointed first sentence, and the ordering slot repeats that sentence instead of zod's generic union text. The "Execution support" docblock is rewritten to the landed state (SQL push-down has compiled $field since PR feat(driver-sql): compile $field to column-to-column comparison (#5222) #7582; the offset rides the same arm). No transform, so no ADR-0122 XParsed alias.
  • SQL (driver-sql, inherited unchanged by driver-sqlite-wasm) — the PR feat(driver-sql): compile $field to column-to-column comparison (#5222) #7582 cross-field arm reads the offset off the same comparand node and compiles column OP (referenced + offset days) per dialect: SQLite date(col, 'N days') for a date column and strftime('%Y-%m-%dT%H:%M:%fZ', col, 'N days') for a datetime column (the driver's canonical text form, so a shifted value is byte-identical to a stored one and the compare stays a plain text compare — no julian-day arithmetic anywhere, so no stop); PostgreSQL date + integer / timestamptz + make_interval(days => n); MySQL date_add(col, interval n day). The offset value is coalesce(offset, 0), truncated toward zero on every dialect; a literal binds as a parameter where the column would. The offset rides the four [spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222 rulings (same-table, declared-only, tenant column forbidden, same class) and adds two: day arithmetic compiles only between two date columns or two datetime columns, and the offset column must be numeric. Everything else is INVALID_FILTER / 400, operands withheld from the caller and named in the server log.
  • Memory (formula matchesFilter) — resolveValue shifts the referenced value by the resolved whole days in the shape it arrived in (a calendar day stays a calendar day, so $lte still covers the whole shifted day; an ISO instant keeps its time of day; Date and epoch number likewise); the offset is truncated toward zero, matching the dialects.
  • NULL semantics, stated in the describe and pinned on both paths in the shape $not 的语义在 driver-sql 与 driver-memory / formula 之间分叉:NULL 行的去留相反,$not: {} 一个是 TRUE 一个是 FALSE #5146 used for $not:
Case Both paths answer
offset column NULL zero days — due_date + NULL is due_date
referenced column NULL the comparison is FALSE for every operator, $ne included — no deadline is never "on time"; $not re-admits the row (the predicate is total)
target column NULL its ordinary reading — fails the orderings and $eq, satisfies $ne when the offset deadline exists

In SQL that is (referenced IS NOT NULL AND target IS NOT NULL AND target OP shifted) for the orderings and $eq, and (referenced IS NOT NULL AND (target IS NULL OR target != shifted)) for $ne; in memory a NULL base resolves to a sentinel evalOp answers false for before any operator runs. So with an offset the $eq/$ne pair is deliberately NOT the both-NULL-matching pair the bare arm emits — a row with no due date does not "equal" its deadline.

  • $not composition, measured and pinned — the cross-field arm was already classified null-total (operatorIsNullTotal), so the $not 的语义在 driver-sql 与 driver-memory / formula 之间分叉:NULL 行的去留相反,$not: {} 一个是 TRUE 一个是 FALSE #5146 negation rewrite leaves it unguarded and NOT is the exact complement. The corpus pins $not of $lte / $eq / $ne with a column offset on both paths; rows with no deadline are in every $not set and in no positive set.
  • Conformance corpus (cross-field-conformance-cases.ts) — a second, nine-row fixture (cross_field_task: a date pair and a datetime pair on the same calendar days at 12:00Z, a nullable numeric grace_days) carrying every NULL arrangement of target / base / offset, a negative grace, and a month boundary; 21 expectations generated for both pairs (42 cases) + 10 refusals. driver-sql runs it per DIALECT_CELLS (SQLite here; the live PG + MySQL cells are named skips locally and run in the Temporal Conformance (live PG + MySQL) job, which runs the whole driver-sql suite with both URLs); driver-sqlite-wasm runs it through its own sql.js dialect; the analytics face runs it through service.query (where and read scope).
  • Analytics routing — read and measured, pass-through, no fix needed. comparand-shape.ts isFieldReference is typeof value.$field === 'string' with extra keys ignored (mirroring driver-sql's fieldReferenceOf), so a reference carrying addDays is still detected and NativeSQLStrategy.canHandle still declines it to the engine path; pinned in cross-field-offset-dataset.test.ts together with findCrossFieldComparand under every combinator. The temporal comparand doors treat the object as never judged — measured at this head: packages/objectql/src/temporal-comparand-door.ts findUninterpretableTemporalComparand answers null for the column-offset, literal-offset and bare reference shapes on a datetime field and fires on the junk-string control; packages/core/src/utils/temporal-comparand.ts isUninterpretableTemporalComparand('datetime', ref) answers false for both shapes and true for the control. Neither package is edited. read-scope-sql.ts untouched.
  • Dataset pin — packages/services/service-analytics/src/__tests__/cross-field-offset-dataset.test.ts: a DatasetSchema dataset over the offset fixture with done_on_time (count, filter completed_on <= due_on + grace_days), late, the datetime-pair twin and a derived on_time_rate, driven through queryDataset against a real SqliteWasmDriver: 2 on time, 4 late, rate 2/9, the datetime twin the same 2, grouped by a dimension without losing the filter; and, with both capabilities declared, only the offset-filtered passes decline native SQL (exactly one raw statement for the plain count). Home: beside the analytics service's own cross-field-engine-fallback.test.ts, not the packages/rest/src/analytics-dataset-* family — that family pins the caller's view of a refusal envelope through the route's catch with a failing driver double; this card's claim is about ROWS on a real engine, and the harness for that lives here.
  • The dotted spelling, stated honestly. The ruling's literal shape names duty.grace_days, a relation path. The memory evaluator walks it (pinned in matches-filter-field-reference-offset.test.ts); SQL push-down refuses a dotted reference under the maintainer's 2026-08-06 same-table ruling on [spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222 (no JOIN planning, no alias contract), and the offset inherits that rule rather than reopening it — INVALID_FILTER naming dotted path in the server log, pinned in the corpus refusals and at dataset level. So on a SQL deployment the dotted offset is a loud refusal, never a wrong number, and the same-table spelling is the one that answers on both paths. Recorded as an open question in the report for the PM.
  • Docs — hand-written content/docs/protocol/objectql/query-syntax.mdx gains a "Comparing Two Fields" section (the page had no $field paragraph to sit beside; the section documents the bare reference and the offset, with an os:check typed block that type-checks under check:skill-examples); generated content/docs/references/data/filter.mdx, the JSON schema, the authorable surface and the strictness-ledger counts via check:generated --fix.
  • Changesets — @objectstack/spec minor, @objectstack/driver-sql minor, @objectstack/formula minor. driver-sqlite-wasm and service-analytics carry no changeset: only their tests moved.

Declared trigger-file touch (#6009)

packages/drivers/driver-sql/src/sql-driver.ts is on #6009's Restart-when (any PR touching sqliteCanonicalDatetimeSql or backfillCanonicalDatetimes). This diff does not change either function or any call to them. It does add a TSDoc {@link sqliteCanonicalDatetimeSql} mention in the new crossFieldOffsetExpr docblock (the SQLite day-add is applied on top of the canonical-text expression filterColumnExpr already returns for a not-yet-backfilled column), so a textual scan of the diff will hit the symbol name. Declared here for the hold's owner; no #6009 work is done in this PR.

Verification record — at b45c078 (git rev-parse --short HEAD after the last commit)

Every command run in the worktree at that head, exit captured before any pipe; verdict lines quoted from the tools' own output. Build closure first (pnpm --filter '@objectstack/driver-sql^...' build, pnpm --filter '@objectstack/service-analytics^...' build, then driver-sql, driver-sqlite-wasm, formula, spec, lint, client, client-react), heavy runs under scripts/pm/os-verify-lock.sh.

Check Result
pnpm --filter @objectstack/spec test Test Files 465 passed (465) · Tests 12408 passed (12408) — exit 0
pnpm --filter @objectstack/driver-sql test Test Files 152 passed, 9 skipped (161) · Tests 2345 passed, 140 skipped (2485) — exit 0; the skips are the unprovisioned live PG / MySQL cells, named by declareUnprovisionedCell (NOT MEASURED locally; the Temporal Conformance (live PG + MySQL) job runs them)
pnpm --filter @objectstack/driver-sqlite-wasm test Test Files 26 passed (26) · Tests 501 passed (501) — exit 0
pnpm --filter @objectstack/formula test Test Files 28 passed (28) · Tests 792 passed (792) — exit 0
pnpm --filter @objectstack/service-analytics test Test Files 89 passed (89) · Tests 1968 passed (1968) — exit 0
typecheck × spec, driver-sql, formula, driver-sqlite-wasm, service-analytics exit 0 each; spec's check:test-typecheck: OK — @objectstack/spec's test layer compiles (the new pins are inside its program)
pnpm --filter @objectstack/spec check:generated (second run, no --fix) ✓ All 15 generated artifacts are up to date.
check:authorable-surface / check:liveness / check:docs / check:api-surface ✅ Successfully generated 1600 schemas. · exit 0 · ✅ 229 generated files in sync · public API surface + factory signatures unchanged ✓
node scripts/check-system-context-census.mjs --fix then plain --fix: 0 anchor(s) rewritten · OK — 106 elevation read sites in 20 packages across 45 files, all anchored
eslint --no-inline-config over the 10 edited TypeScript files exit 0, no output
check:doc-authoring ✓ doc authoring guard: 14658 customer-facing string(s) across 723 spec sources clean (a first run at an earlier head flagged tracker ids in three new strings; stripped, with a negative pin added)
node scripts/pm/dispatch-gates.mjs --commands union — 79 commands (24 node, 55 pnpm) derived from the 17-path change set vs merge base 1c7adc7 74 green by their own verdict lines. NOT MEASURED by their own text: check-dev-prereqs.mjs (exit 1 — stale/absent dist/ of 58 unrelated packages, asks for a full pnpm build), check-test-completeness.mjs (exit 3 PREREQUISITE NOT MET — grades a saved turbo test log), check:dual-build-cjs-loads (exit 3 PREREQUISITE NOT MET — 76 packages without dist/), check:type-check-debt (exit 3 PREREQUISITE NOT MET — 25 ledgered deps unbuilt), check:entry-nameability (exit 0 but prints NOT MEASURED: no callable export on @objectstack/spec/qa, pre-existing). CI runs the farm.

Also run: check:skill-examples (✅ 257 prose examples type-check across 3 surface(s) — covers the new os:check block), check:strictness-ledger (is current — 439 site(s) measured), check:nul-bytes OK, check:cross-package-test-inputs OK, check:driver-conformance exit 0.

Reverse verification (two legs, restore proven by blob hash, script with trap restore)

  • Leg 1 — unhook the SQL offset compile (fieldReferenceOffsetOf forced to null; marker count on disk 1): sql-driver-cross-field-conformance.test.ts → 43 failed / 74 passed / 2 skipped; every red is in the [addDays] blocks — 17 distinct SQL push-down disagreed conformance cases plus the offset refusal arm (expected a refusal — the offset silently compiled as a bare comparison); 0 in-memory evaluator disagreed. The formula pins stayed green (28 passed) — memory untouched. Restored with git checkout HEAD --; git hash-object = HEAD blob 61008e7c…, marker count 0.
  • Leg 2 — unhook the memory offset (resolveValue returns the bare base; an executable string marker): matches-filter-field-reference-offset.test.ts → 20 failed / 8 passed (the 8 are the offset-free controls). Rebuilt formula; node scripts/ablation-dist-preflight.mjs @objectstack/formula 'ABLATION-14104-MEM' → ✓ dist/: marker present in 2 built files; the dist-resolving driver-sql conformance suite → 34 failed / 83 passed, every assertion in-memory evaluator disagreed, 0 SQL disagreements. Restored (hash-object = HEAD blob 087ba9fc…, marker count 0), rebuilt, preflight --absent → ✓ dist/: marker absent from all 6 built files, suite back to 117 passed / 2 skipped. (A first attempt used a comment marker the bundler strips — the preflight correctly refused it as "only in sourcemaps"; the leg was re-run with the executable marker above.)

🤖 Generated with Claude Code

https://claude.ai/code/session_019vx3536MUFc8XYVLNoKhs3


Generated by Claude Code

…corpus rows on both paths (wip)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019vx3536MUFc8XYVLNoKhs3
…tax section, changesets, regenerated spec artifacts (wip)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019vx3536MUFc8XYVLNoKhs3
…ference, compiled on SQL and evaluated in memory

FieldReferenceSchema gains addDays — an integer literal of any sign or a nested
{ $field } reference to a numeric column — so a dataset measure can express
completed_at <= due_date + grace_days. driver-sql compiles the offset on the
cross-field arm per dialect with the ruled NULL semantics written into the
predicate; matchesFilter evaluates it identically; the shared conformance corpus
carries the literal, column, negative, NULL-offset, NULL-base and $not rows on
both paths; the analytics detector keeps routing the reference to the engine
path and a dataset-level pin drives the on-time count on both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019vx3536MUFc8XYVLNoKhs3
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 3 package(s): @objectstack/driver-sql, @objectstack/formula, @objectstack/spec, touching 55 documentable anchor(s). ⚠️ 2 changed file(s) yielded no anchor (packages/drivers/driver-sql/src/index.ts, packages/spec/authorable-surface/data.json), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

48 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: node scripts/docs-audit/affected-docs.mjs --json 29db3cd2ada4d88f97c52c00634609df9c573444.

⛔ 4 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails.

What this run could not see
  • 2 changed file(s) yielded no anchor (packages/drivers/driver-sql/src/index.ts, packages/spec/authorable-surface/data.json) — pages documenting those are invisible to this run
  • 9 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 132 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 29db3cd2ada4d88f97c52c00634609df9c573444 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from cb9d4b8f06101bdb4c8a77c90974991fbd9d25bb — the merge of head b45c07858b2a27cdc0ab78b80f7aa1544e8cd2de into base 29db3cd2ada4d88f97c52c00634609df9c573444, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin cb9d4b8f06101bdb4c8a77c90974991fbd9d25bb && git checkout cb9d4b8f06101bdb4c8a77c90974991fbd9d25bb
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 29db3cd2ada4d88f97c52c00634609df9c573444 b45c07858b2a27cdc0ab78b80f7aa1544e8cd2de && git checkout -B drift-repro 29db3cd2ada4d88f97c52c00634609df9c573444 && git merge --no-ff b45c07858b2a27cdc0ab78b80f7aa1544e8cd2de

node scripts/docs-audit/affected-docs.mjs --json 29db3cd2ada4d88f97c52c00634609df9c573444

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 29db3cd2ada4d88f97c52c00634609df9c573444 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

Copy link
Copy Markdown
Collaborator Author

Landing provenance (PM seat domain:spec, session_0174WZTU6XcFcS7g2kykC53i, 2026-09-03T22:57Z) — flipping to ready and enabling auto-merge (squash).


Generated by Claude Code

@zhuangjianguo
zhuangjianguo marked this pull request as ready for review September 3, 2026 22:57
@zhuangjianguo
zhuangjianguo added this pull request to the merge queue Sep 3, 2026
Merged via the queue into main with commit 4a4a35d Sep 3, 2026
43 checks passed
@zhuangjianguo
zhuangjianguo deleted the claude/issue-14104-field-reference-add-days branch September 3, 2026 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:data size/xl tests tooling

Projects

None yet

2 participants