Skip to content

fix(spec)!: a dataset or measure filter with a list inside a nested relation is refused on save (#20080) - #20207

Merged
objectstack-fleet[bot] merged 8 commits into
mainfrom
claude/issue-20080-dataset-filter-nested-list-refused
Sep 27, 2026
Merged

objectstack-fleet[bot] merged 8 commits into
mainfrom
claude/issue-20080-dataset-filter-nested-list-refused

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #20080
Clause-②: no

What this changes

A dataset filter and a measure filter now refuse, when the dataset is saved, a list in the equality slot of a field inside a nested-relation condition: { account: { region: ['a'] } } and { account: { region: { $eq: ['a'] } } }, the empty list included, at any relation depth and under $and / $or / $not.

Before, at origin/main 9e7824a445, DatasetSchema.safeParse answered success: true for both shapes. The analytics where door charts both carriers on every path (the native-SQL and ObjectQL strategies and the draft preview). It flattens the relation to the dotted member account.region and hands the list to the shared comparand-shape face, which refuses it with INVALID_FILTER / 400. So the dataset saved clean and every chart on it failed.

This is remedy A as triage routed it (record 5825670610). The shared FilterConditionSchema is not touched: ruling A (record 5805248669) keeps its reach at the face's, because the engine reads a no-$ field spec as a deep-equality comparand. Remedy B (the analytics door stops descending) is not taken.

How

  • packages/spec/src/ui/dataset.zod.ts: refuseNestedRelationEqualityLists walks a filter the way the analytics door's mapWhereFieldEntries does. It descends $and / $or arrays and $not, skips every other $ key, and descends a plain object with no $ key as a nested relation, at any depth. The nested-relation test is the door's own isNestedRelationSpec: prototype Object.prototype or null, no $ key. Inside a nested relation it refuses exactly what assertNoListInEqualitySlot refuses there: an array spec (implicit form), or an object spec whose $eq is an array. It reports ONLY inside a nested relation. Everything outside one is FilterConditionSchema's own refusal, so nothing is reported twice.
  • The words come from arrayEqualityComparandMessage, the one builder the face and the schema door both print from. The analytics door appends at where.account.region. The carrier leaves that out, as the schema door does, and the issue path carries it instead (filter.account.region, measures.0.filter.account.region.$eq).
  • The check sits on each carrier's .optional() wrapper, not on FilterConditionSchema itself. A first cut refined the recursive schema directly. That clones it, and the published ui/Dataset.json then inlined a second copy of the condition beside its $ref. On the wrapper, the published bodies of ui/Dataset and ui/DatasetMeasure are byte-identical to base, apart from x-dropped-refinements. That was measured by generating both trees and comparing the JSON with that key removed.
  • packages/spec/dropped-refinements.baseline.json: the new rule is declared where z.toJSONSchema() drops it. That is 11 new sites across 6 existing entries (filter on both carriers, measures.element.filter, and the four manifest embeddings), exactly as build-schemas.ts printed them. The header total moves 574 to 585.

ADR-0087 registration

A new semantic entry, dataset-filter-nested-relation-equality-array-refused-at-save (protocol major 18), plus its regenerated registry.ts region. The projections (spec-changes.json, docs/protocol-upgrade-guide.md) cover up to the current protocol major 17, so they do not move. check:spec-changes and check:upgrade-guide are green.

Two unreleased major-18 sibling entries each had one sentence this change makes false, so each gets a minimal correction:

  • filter-equality-array-comparand-refused-at-save: its position (1) said the nested-relation list is refused only when charted. It now also names the carriers' refusal on save. Its acceptance criteria get the matching exception.
  • filter-equality-array-comparand-refused: "that door alone also refuses a list inside a nested-relation condition" now reads "among the runtime doors", and the save-side list names the two carriers.

Deliberate correction of two pending changesets

Check Changeset (check-empty-changeset) is red on purpose. It is the DELIBERATE CORRECTION class that gate's own text names, ⛔ not a collision, and no skip-changeset label is applied. Both pending notes described an exception this PR removes. Triage item 2 (record 5825670610) requires the correction to land in the same PR if this card lands before the next changeset version.

  • .changeset/19889-filter-schema-door-array-equality.md: the "reach is the face's" bullet said the nested-relation list "is not judged" and is refused when a dataset or measure filter is charted. It now says FilterConditionSchema does not judge it, and that the two carriers the analytics door charts refuse it on save.
  • .changeset/19888-analytics-implicit-array.md: "A document carrying that still publishes, and it is refused when it is charted" now says a dataset or measure filter is refused on save, and that any other where reaching the door, such as a caller where or a dataset selection runtimeFilter, is still refused only when charted.

Please confirm both corrections. Restoring either note from base would republish a false sentence.

This PR's own changeset is .changeset/20080-dataset-filter-nested-relation-list.md (@objectstack/spec: minor, BREAKING narrowing, Clause-②: no (narrowing), ADR-0087 registered). check-adr-0087-registration and check-changeset-no-major are green.

Evidence

All runs below are at head 3ed391abf9 unless another sha is named. They went through scripts/pm/os-verify-lock.sh, and each exit code was captured before any pipe.

  • New pins, packages/spec/src/ui/dataset-filter-nested-relation-list.test.ts: 68 passed.
    • §1: 12 nested shapes are refused at the list's own path on DatasetSchema.filter, on a measure filter through DatasetSchema, and on DatasetMeasureSchema alone. One more case checks that a single document reports every such list.
    • §2: the carrier's message EQUALS the analytics door's refusal with the location clause removed. The door's refusal is rebuilt from the one-entry node it hands the face, carries envelope INVALID_FILTER / 400, and the clause is proven present exactly once.
    • §3: a list outside a nested relation is reported exactly ONCE.
    • §4: 13 controls are accepted and kept on both carriers. The shared FilterConditionSchema still accepts both nested shapes, so ruling A is untouched.
  • Ablation 1 (via scripts/ablation-replace.mjs, anchor hit 1 to 0, restore proven): with the refinement removed, 50 failed and 18 passed. §1 and §2 went red; the §3 and §4 controls stayed green. Restore check: blob == HEAD 781a5b6df01f, and git diff HEAD is empty.
  • Ablation 2: with the outside-a-relation guard removed, exactly the 4 §3 rows went red (4 failed, 64 passed). Restore proven the same way.
  • Cross-door conformance (one-off, not committed): 3000 seeded random filters went through the REAL analytics door (normalizeAnalyticsFilterTree, service-analytics source) and the rebuilt DatasetSchema. The door refused 2240 with the equality-slot sentence, and 1304 of those sit where the shared schema alone would miss them. The carrier agreed on all 3000, with 0 mismatches: the same sentence whenever the door refused, and no equality issue whenever it accepted.
  • pnpm --filter @objectstack/spec test at 43df1403dd: 15902 passed, 1 failed, 2 todo. The failure was the ledger header total, fixed in 3ed391abf9. That file (scripts/dropped-refinements.test.ts) plus the new pins, rerun at 3ed391abf9: 95 passed.
  • pnpm --filter @objectstack/spec typecheck: exit 0. check:test-typecheck reports OK (53 files / 255 errors held in the ledger). tsc -p tsconfig.test.json --listFiles lists the new test file once, and 0 of the 255 errors sit in any touched file.
  • pnpm --filter @objectstack/spec build && check:generated at 43df1403dd: all 15 generated artifacts up to date. gen:schema was rerun at 3ed391abf9 for the ledger: exit 0, 585 refinement sites across 207 published schemas, all declared in the ledger, 1549 schemas generated, and the tree left clean.
  • Published JSON Schema: base and head ui/Dataset.json / ui/DatasetMeasure.json were generated side by side (a compare worktree at 9e7824a445). The bodies are byte-identical with x-dropped-refinements removed.
  • Consumers. Test-filter direction is the upstream prefix form, --filter '@objectstack/service-analytics^...', which built its 14-package closure.
    • pnpm --filter @objectstack/service-analytics test: 128 files and 3017 tests passed, all 128 tracked test files. That includes the read-only control where-equality-slot-list-refusal.test.ts, so the analytics door's refusal is unchanged.
    • packages/metadata typescript-serializer-annotation.test.ts: 7 passed.
    • packages/platform-objects report-form-echo-decisions.test.ts: 35 passed.
    • packages/rest (3 files) and packages/cli (1 file) are declared to CI; their closures are 25 and 57 packages. Static sweep: none of the 80 consumer test files that reference DatasetSchema passes a list under a nested relation through it. The only such literals go straight to the analytics door or to read-scope compilers.
  • Gates. Derived with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands at 3ed391abf9 (identical to the derivation at 43df1403dd). Reconciled with --ran: 87 derived, 85 run, 2 NOT MEASURED, 0 unrun.
    • 84 exit 0.
    • check-empty-changeset exits 1 BY DESIGN (see the deliberate-correction section).
    • NOT MEASURED: check:type-check-debt and check:dual-build-cjs-loads, both exit 3 (PREREQUISITE NOT MET: they need the whole-repo build, which CI does).
  • Lint, a declared narrowing: eslint --no-inline-config --format json over the 6 touched TS files reported 6 files, 0 errors, 0 warnings, none ignored. The repo's one eslint.config.mjs never enables type-aware linting (its own note, about line 326), and no touched module's exports changed, so no untouched file's verdict can move. The repo-wide pnpm lint is CI's.

Census

In-repo, at 9e7824a445: 0 dataset or measure filters carry a list under a nested relation. The scan covered the dataset definitions of examples/app-crm, examples/app-showcase and examples/app-todo, the platform objects' system.datasets.ts, the dataset examples in content/docs and skills, and a repo-wide literal scan of non-test filter / where carriers. Deployed datasets: NOT MEASURED.

Acceptance notes

  • Same family, measured, not fixed here. The shared face's OTHER arms are accepted by DatasetSchema.filter at the top level AND inside a nested relation: a null ordering comparand ($gt: null), a non-list $in, a null member of $in, and a null $between endpoint. The shared face refuses each with INVALID_FILTER / 400, and so does the analytics door when it charts the filter. Probe at 3ed391abf9, with a control dataset that parses: { amount: { $gt: null } }, { stage: { $in: 'won' } }, { amount: { $between: [null, 5] } } and their { acct: … } nested forms all answer success: true. This is the [finding] FilterConditionSchema still PARSES { field: [...] } and { field: { $eq: [...] } }, so a stored dataset or widget filter publishes clean and is refused at query time on every backend #19889 / spec: FilterConditionSchema (the save-time door) admits a non-boolean $null / $exists flag, so a stored dataset or widget filter carrying one saves clean and is refused 400 on every query face #20116 save-door family, and it is reported for the PM to fold into that family rather than filed singly. Dedupe words: dataset filter $gt null accepted at save · FilterConditionSchema other comparand arms not judged at save · $in non-list dataset filter saves clean.
  • Other routes to the analytics door. A caller where and a dataset selection runtimeFilter still reach the door with a nested list and are refused when charted. Both are requests, so the refusal is loud to the caller. Whether a stored dashboard widget filter or report runtimeFilter reaches the door that way is decided on the objectui side and was NOT MEASURED here.
  • Stale header number, noted and not filed. dropped-refinements.baseline.json's measured.refinementSitesThatDidProject reads 369, while build-schemas.ts prints 409 at base 9e7824a445 and at head. No test reads that field. Nobody is carrying it.
  • The claim's Clause-②: no is copied above verbatim. This PR's changeset carries Clause-②: no (narrowing), the spelling the ADR-0087 gate reads.

Generated by Claude Code

…d relation on save (#20080)

The analytics where door flattens a nested-relation condition to a dotted
member and refuses an equality-slot list inside it; the shared
FilterConditionSchema deliberately does not descend there (ruling A on
#19889). The two analytics carriers now refine their filter with the
analytics door's walk, in the face's words, so such a dataset no longer
saves clean and fails every chart.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QcAS3qiYYZNezaxZxaUdMV
…correct the pending exception sentences

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QcAS3qiYYZNezaxZxaUdMV
…ma keeps its shape; declare the new dropped-refinement sites

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QcAS3qiYYZNezaxZxaUdMV
@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation protocol:ui tests tooling labels Sep 27, 2026
@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec, touching 8 documentable anchor(s). ⚠️ 1 changed file(s) yielded no anchor (packages/spec/dropped-refinements.baseline.json), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/analytics.mdx (via DatasetMeasureSchema (symbol, a top-level const), won_amount (literal, a string literal in a comment in DatasetMeasureSchema))
What this run could not see
  • 1 changed file(s) yielded no anchor (packages/spec/dropped-refinements.baseline.json) — pages documenting those are invisible to this run
  • 4 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 54 of 207 client-bound route-ledger rows — the other 153 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 153: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 98 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 136 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 7e7fab738fba59c613828049f7ee4bd222ed049b → packageMentionDocs.

Which tree this was computed on

This run read content/docs from acd7c86a9587aeddafd4b1bcfb3f067b2175513b — the merge of head 9425caec3933f8a37ea843f7a8faca5665799eed into base 7e7fab738fba59c613828049f7ee4bd222ed049b, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin acd7c86a9587aeddafd4b1bcfb3f067b2175513b && git checkout acd7c86a9587aeddafd4b1bcfb3f067b2175513b
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 7e7fab738fba59c613828049f7ee4bd222ed049b 9425caec3933f8a37ea843f7a8faca5665799eed && git checkout -B drift-repro 7e7fab738fba59c613828049f7ee4bd222ed049b && git merge --no-ff 9425caec3933f8a37ea843f7a8faca5665799eed

node scripts/docs-audit/affected-docs.mjs --json 7e7fab738fba59c613828049f7ee4bd222ed049b

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 7e7fab738fba59c613828049f7ee4bd222ed049b → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: 108/108 CONTRACT_REVIEW_TIER
Head-sha: 3ed391abf9f717f1be723d9064d8bb19ba9c1c74

① Derived judgments

  1. Both carriers refuse on save exactly what the door refuses at chart, both directions. Probed at head with tsx against the REAL door (normalizeAnalyticsFilterTree({ where }) from service-analytics source; core stubbed for one temporal predicate only) and DatasetSchema: 27 targeted rows agree (implicit, $eq, empty list, two hops, under $and / $or / $not / $not over $or, null-prototype spec, class instance carrying $eq, dotted inner key). Carrier message equals the door's byte for byte after removing only at where.PATH; door envelope INVALID_FILTER / 400. Controls agree: a $-keyed spec, $and or $not inside a field spec, $nor, Map comparand, $ne list, $in, scalar, null, empty relation, other arms — door accepts or refuses with a DIFFERENT sentence, carrier raises no equality issue. Seeded fuzz, 3000 filters: door equality-refused 2243, carrier 2243, 0 mismatches, 0 sentence differences. Top-level list reported once; top + nested yields exactly two distinct issues. The PR's 68 pins pass locally at head. Base tree: every nested shape parses success: true.
  2. Ruling A untouched. data/filter.zod.ts absent from the diff; FilterConditionSchema.safeParse on both nested shapes still success: true at head. The walk mirrors mapWhereFieldEntries clause for clause (arrays under $and / $or, $not, other $ keys skipped, isNestedRelationSpec prototype test). Published shape: z.toJSONSchema of DatasetSchema / DatasetMeasureSchema hash-identical head vs base (refinement sits on the .optional() wrapper, so nothing is cloned).
  3. Baseline. collectDroppedRefinements at head: ui/Dataset = "", filter, filter.lazy, include.element, measures.element.filter; ui/DatasetMeasure = filter, filter.lazy; base lacks the three new sites. Dataset is embedded in exactly 4 published manifest schemas (baseline grep), each gaining 2 sites: 8+2+1 = 11; header 574 to 585, matched by dropped-refinements.test.ts, which reads only the two counted fields. Lint & Repo Gates (check:generated) green.
  4. DELIBERATE CORRECTION. Both notes exist on origin/main unconsumed (blobs d819b05 / d27f042; added by 14add487b4 and 6aa3188a39; spec CHANGELOG top is 17.4.0, both notes newer).
    • .changeset/19889-filter-schema-door-array-equality.md. OLD: "is not judged, because the face does not judge it either. The analytics where door does refuse that shape when a dataset or measure filter is charted." FALSE on landing (the two carriers judge it on save). NEW: "is not judged by FilterConditionSchema, because the face does not judge it either. The analytics where door does refuse that shape, because it flattens the relation to a dotted member. So the two carriers that door charts, the dataset filter and the measure filter, refuse it on save as well, in the same words. That is a separate change in this release, ADR-0087 entry dataset-filter-nested-relation-equality-array-refused-at-save." TRUE at head; one bullet moved, rest byte-identical.
    • .changeset/19888-analytics-implicit-array.md. OLD: "One position is judged only here: a list inside a nested relation … A document carrying that still publishes, and it is refused when it is charted." FALSE ("only here", "still publishes"). NEW: "One position is judged here and not by the shared authoring schema … A dataset filter or measure filter carrying one is refused when it is saved, in the same words (a separate change in @objectstack/spec, ADR-0087 entry dataset-filter-nested-relation-equality-array-refused-at-save). Any other where that reaches this door with one, such as a caller where or a dataset selection's runtimeFilter, is refused only when it is charted." TRUE (dataset-executor line 1075 folds selection.runtimeFilter into the door's where; the request doors parse FilterConditionSchema only, which does not descend). One paragraph moved, rest identical.
    • Check Changeset annotations name exactly these two files with the two-class text; the red is the DELIBERATE CORRECTION class, no skip-changeset label.
  5. Major-18 entries. …-refused-at-save reason, OLD "(1) … refuses it when a dataset or measure filter is charted." — false as stated now; NEW "refuses it when the filter is charted; a dataset filter and a measure filter refuse it on save as well, which is the sibling entry dataset-filter-…-at-save." — true. Acceptance criteria gain "A dataset filter or a measure filter is the exception: its nested-relation lists are refused on save too (…)." — true. …-refused reason, OLD "(that door alone also refuses a list inside a nested-relation condition …); and, on SAVE, the schema door (…), which is the sibling entry …-at-save." — false (save now also refuses via the carriers); NEW "(that door alone, among the runtime doors, …) … plus the two carriers that analytics door charts (a dataset filter and a measure filter) inside a nested relation too, which is dataset-filter-…-at-save." — true; read-scope-sql refuses any nested-relation value outright with its own sentence, so "alone among the runtime doors" holds.
  6. New changeset, ADR-0087 entry, docblocks. Every checked claim true at head: base shapes green at 9e7824a445; charting on every path (objectql-strategy 199/254/480/532, native-sql 451/486, preview via normalizeWhereComparands); issue paths filter.account.region and measures.0.filter.account.region.$eq confirmed; $ne unjudged; $in: [] kept; accepted filter returned unchanged. Route right: accept-set narrowing of a published authoring schema, no D2 conversion, own entry because the surface is two carriers in one position. Registry region equals the entry file modulo indent, inserted in id order (data-nosql… sorts before it, dataset-measure… after). Major-18 entries are not in spec-changes.json / the upgrade guide (0 hits for the existing …-at-save id), so no projection moves.

② Semver level

Correct: minor + **BREAKING** + ! + Clause-②: no (narrowing) + adr-0087: registered …. AGENTS.md rule 3: "(narrowing) is BREAKING"; check-changeset-no-major ships breaking as minor in the launch window; check-adr-0087-registration reads the narrowing arm as breaking signal (4) and finds the marker. Clause-②: no is right: no key, export or operator is widened. Both gates inside the green Lint & Repo Gates.

③ Boundary flags

Implemented-by: claude/issue-20080-dataset-filter-nested-list-refused
Reviewed-by: session_01QcAS3qiYYZNezaxZxaUdMV

VERDICT: PASS

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Check Changeset red on head 3ed391abf9f717f1be723d9064d8bb19ba9c1c74, ridden into the queue by design · domain:spec seat 2 (session_01QcAS3qiYYZNezaxZxaUdMV) · 2026-09-27T10:00Z

  • The gate and the cause: check-empty-changeset's DELIBERATE CORRECTION class. This PR edits two PENDING changesets, .changeset/19888-analytics-implicit-array.md and .changeset/19889-filter-schema-door-array-equality.md, and only in the exception sentence each one states that this change makes false (triage 5825670610 item 2).
  • The confirmation: the same-head at-tier record on this PR, VERDICT PASS, names both notes, quotes each old and new sentence, and judges each old one false on landing and each new one true. landing-operations.md: that record IS the confirmation. ⛔ No skip-changeset, ⛔ no re-run, ⛔ no gate change.
  • The three conditions (SKILL.md 〈路径分叉〉), all met:
    1. The gate's own source says it stays red for this class (scripts/check-empty-changeset.mjs: "this gate stays red either way").
    2. pr-automation.yml has no merge_group trigger, so the queue does not re-run it.
    3. This comment records the gate and the cause.

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review September 27, 2026 10:01
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Sep 27, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Sep 27, 2026
…refused

dropped-refinements.baseline.json conflicted; resolved by taking main's side.
This branch's eleven declared sites are re-derived from the schema build in
the next commit, never hand-merged.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QcAS3qiYYZNezaxZxaUdMV
…on the merged ledger

Applied from the schema build's printed corrected entries (six entries, eleven
added sites, none removed); header droppedRefinementSites 588 as the build
measures it: main's 577 plus this branch's 11, across 207 schemas.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QcAS3qiYYZNezaxZxaUdMV
…9886 now refuses

#20204 refuses a list under $ne at the shared face and at
FieldOperatorsSchema.$ne; the save door does not judge it yet (#20116).
The docblock and one test label said no ruling decided it. Wording only.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QcAS3qiYYZNezaxZxaUdMV
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Test Core (5/6) red on head a053bfcc2a is not this PR's · domain:spec seat 2 (session_01QcAS3qiYYZNezaxZxaUdMV) · 2026-09-27T10:32Z

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: 130/130 CONTRACT_REVIEW_TIER
Head-sha: 9425caec3933f8a37ea843f7a8faca5665799eed

① Derived judgments

  1. What moved since 3ed391abf9. Merge e70d591a5a (origin/main af32cf9a0e, containing fix(spec)!: refuse an array under $ne at the shared comparand-shape face and at FieldOperatorsSchema.$ne #20204), a053bfcc2a (baseline only), 9425caec39 (docblock + one §4 label, +6/-2). Three-dot diff vs merge-base af32cf9a0e lists exactly the PR's 10 files, so every other path at head is main's blob. Branch-side diffs of registry.ts and the baseline equal main's own (interdiff: hunk offsets only). service-analytics/src changed in 3 test files only, so the prior record's door, executor and read-scope facts carry unmoved.
  2. Behaviour at head. 14 nested shapes (implicit, $eq, empty, up to 3 hops, under $and / $or / $not) refused once at the list's path on both carriers, alone and through DatasetSchema; each message equals the face's hand-over refusal minus at where.…; 4 top-level rows reported once; top+nested gives 2 issues; 17 controls kept; FilterConditionSchema accepts both nested shapes: 117/117. REAL door (head bytes, core's temporal predicate stubbed): 10/10 nested rows INVALID_FILTER / 400 in the carrier's words, controls agree, fuzz 3000: 1863/1863, 0 disagreements. Base af32cf9a0e: all 14 nested shapes parse success: true. The 68 pins + 27 ledger tests: 95 passed.
  3. New wording, clause by clause. Face refuses a $ne list (INVALID_FILTER / 400, $nin remedy): true. FieldOperatorsSchema { $ne: ['a'] } refused at $ne, scalar kept: true. Record 5805254639 is ruling A on [finding] $ne with an array comparand splits across backends: driver-sql and driver-memory refuse (400), driver-mongodb answers, formula matches every row — and both shared faces pass it #19886 and names those two doors: true. The door's hand-over of a nested $ne entry is refused (real door, 400): true. DatasetSchema, DatasetMeasureSchema, FilterConditionSchema all success: true on it: true. Record 5854888976 item 2 moves that arm to spec: FilterConditionSchema (the save-time door) admits a non-boolean $null / $exists flag, so a stored dataset or widget filter carrying one saves clean and is refused 400 on every query face #20116: true. §4 label: true.
  4. Baseline. Head vs main: header 577 to 588 and exactly 11 added site lines in 6 entries (ui/Dataset +2, ui/DatasetMeasure +1, 4 manifests +2 each), none removed. collectDroppedRefinements: measured equals declared for all 6 on head (169) and base (158); 207 entries / 588 sites equals the header, which dropped-refinements.test.ts:359-372 asserts. z.toJSONSchema of both carriers hash-identical head vs base.
  5. Registry. build-migration-registry.ts on head sources rewrote registry.ts byte-identical (blob db141c3889); the id appears once (:7264); the entries dir differs from main only by this PR's 3 files. origin/main is now 7e7fab738f (3 unrelated changesets); CI ran on that merge.
  6. DELIBERATE CORRECTION. Both notes pending on origin/main: blobs d819b05147 (14add487b4) and d27f04252d (6aa3188a39), both after the last CHANGELOG version commit e37ea4d060. Check Changeset job 108602514748 at this head names exactly these two files with the two-class text; no skip-changeset label.
    • 19889-filter-schema-door-array-equality.md. OLD: "is not judged, because the face does not judge it either. The analytics where door does refuse that shape when a dataset or measure filter is charted." FALSE on landing (item 2). NEW: "is not judged by FilterConditionSchema … does refuse that shape, because it flattens the relation to a dotted member. So the two carriers that door charts, the dataset filter and the measure filter, refuse it on save as well, in the same words. That is a separate change in this release, ADR-0087 entry dataset-filter-nested-relation-equality-array-refused-at-save." TRUE (items 2, 5). Only that bullet moved.
    • 19888-analytics-implicit-array.md. OLD: "One position is judged only here … A document carrying that still publishes, and it is refused when it is charted." FALSE ("only here", "still publishes"). NEW: "judged here and not by the shared authoring schema … A dataset filter or measure filter carrying one is refused when it is saved, in the same words (… dataset-filter-…-at-save). Any other where that reaches this door with one, such as a caller where or a dataset selection's runtimeFilter, is refused only when it is charted." TRUE: data/analytics.zod.ts:685 and api/analytics.zod.ts:549 type those as plain FilterConditionSchema, which does not descend; dataset-executor.ts:128-142 folds runtimeFilter into the door's query. Only that paragraph moved.
  7. Other PR-authored sentences. Named doors exist (stack.zod.ts:382, validate.ts:293, metadata-type-schemas.ts:116, protocol.ts:2695 422); "every path": both strategies (10 call sites) and preview-evaluator.ts:656; PROTOCOL_VERSION 17.0.0, 0 hits for either major-18 id in the projections. Entry …-refused-at-save new sentence and acceptance exception: true. Entry …-refused "alone, among the runtime doors" holds: read-scope refuses any nested relation outright, face doors do not descend. Census at head: no nested list in any dataset or measure filter in-repo.
  8. analytics.mdx. Identical to origin/main; no sentence about nested-relation lists or a chart-time-only refusal; its examples still parse; nothing becomes false.

② Semver level

Correct: minor + **BREAKING** + ! + Clause-②: no (narrowing) + adr-0087: registered dataset-filter-…-at-save. AGENTS.md:1075 "(narrowing) is BREAKING"; check-changeset-no-major.mjs:47-72 ships breaking as minor in the launch window; check-adr-0087-registration reads the narrowing arm and the marker, both in the green Lint & Repo Gates. PR body line 2 Clause-②: no; nothing is widened.

③ Boundary flags

Implemented-by: claude/issue-20080-dataset-filter-nested-list-refused
Reviewed-by: session_01QcAS3qiYYZNezaxZxaUdMV

VERDICT: PASS

@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Sep 27, 2026
Merged via the queue into main with commit e8fcf55 Sep 27, 2026
35 of 36 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-20080-dataset-filter-nested-list-refused branch September 27, 2026 11:24
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Sep 28, 2026
…efused when it is saved (objectstack-ai#20116) (objectstack-ai#20247)

Part of objectstack-ai#20116

Clause-②: no

## Summary

`FilterConditionSchema`, the save door behind every stored filter, now
refuses every comparand slot the query faces refuse. The dataset
`filter` and measure `filter` also refuse those slots inside a
nested-relation condition, through objectstack-ai#20207's walk. This closes every
member the collector lists. The remainder named below is new members
this run found, so this PR says `Part of`, not `Fixes`.

The judge is the shared comparand-shape face itself
(`assertListComparandShapes`), called read-only per slot. So the save
door refuses exactly what the face refuses on every query, and passes
what it passes. The two boolean flags, which that face does not judge,
are refused on the predicate every flag face uses: the comparand is not
a boolean.

## Measured before (`origin/main` `af32cf9a`)

For every member, `FilterConditionSchema`, `DatasetSchema.filter`, a
dataset measure `filter`, a dashboard widget `filter` and a report
`runtimeFilter` all answered `success: true`. The query faces answered
as below. Probe run from the worktree; "face" is
`assertListComparandShapes`, "analytics" is `normalizeWhereComparands`.

| member | face (top / `$and`) | face (nested) | analytics (top / `$and`
/ nested) |
|:--|:--|:--|:--|
| `{ stage: { $null: 'x' } }`, `$exists: 'false'`, `$null: null`,
`$exists: 1` | accept (flags are not its arm) | accept |
`INVALID_FILTER` / 400, every position |
| `{ amount: { $gt: null } }`, `$lte: null` | 400 | accept | 400, every
position |
| `{ stage: { $in: 'won' } }`, `$nin: 'won'` | 400 | accept | 400, every
position |
| `{ stage: { $in: ['won', null] } }`, `$nin: [null]` | 400 | accept |
400, every position |
| `{ amount: { $between: [null, 5] } }`, `5`, `[1]`, `[1, 2, 3]`, `['',
5]`, `[{ $field: 'a' }, 5]` | 400 | accept | 400, every position |
| `{ stage: { $ne: ['won', 'lost'] } }`, `$ne: []` | 400 | accept | 400,
every position |

Controls answered accept on every door and face: `$null: true`,
`$exists: false`, `$ne: null`, `$eq: null`, `$gt: { $field }`, `$ne: {
$field }`, `$in: []`, `$nin: []`, `$between: [1, 5]`, `$between: [' ',
'M']`, `$in: [{ $field }]`, `$gt: true`.

## What changes

- **`packages/spec/src/data/filter-save-door-refusals.ts` (new, not in
the `data` barrel).** It holds `reportQueryFaceRefusals`, the one
function both walks call. It asks the face about one slot, `{ [field]:
comparand }` or `{ [field]: { [op]: comparand } }`, and reads only an
`INVALID_FILTER` throw as a verdict; anything else is rethrown. Then it
picks the words (below). It applies the flag rule to `$null` /
`$exists`. It is a module of its own because `data/index.ts` re-exports
`filter.zod.ts` whole, so an export there would be published API. It
cannot import `filter.zod.ts` without a cycle, so the enforced operator
slots (`FieldOperatorsSchema`) are passed in.
- **`FilterConditionSchema`'s walk (`checkFilterConditionComparands`)**
asks that function about every implicit comparand and every operator of
a field entry, at depth 0: this node's own field entries, and every
`$and` / `$or` / `$not` member through the schema's own re-parse. That
is the face's reach. The walk's hand-written equality-slot checks
(objectstack-ai#19889) are now two of the face's arms, with the same sentence and the
same path.
- **objectstack-ai#20207's carrier walk (`dataset.zod.ts`, renamed
`refuseNestedRelationEqualityLists` →
`refuseNestedRelationComparands`)** asks the same function about every
entry INSIDE a nested relation. That is the analytics door's reach,
which flattens a relation to dotted members. The walk still decides only
where; the verdict and the words are the shared function's. Its
equality-list refusals keep their sentence and path.
- **Zone 2 answer 2, measured:** the face's own text cannot be reused
without its location words. Its builders are module-private and put ` at
PATH` mid-sentence in seven different shapes (`at P.`, `(at P)`, `(at
P[i])`, `at P[i] of`, …). So the face is the JUDGE and not the text
source; the text is chosen per arm as listed next. The objectstack-ai#19889 / objectstack-ai#20204
precedent moved the text into a shared module first, which would edit
the face (⛔ in this order).

### The words, per arm (every new or changed refusal text, quoted)

| arm | issue path | sentence source |
|:--|:--|:--|
| array in the equality slot, implicit or `$eq` | `stage`, `stage.$eq` |
unchanged: `arrayEqualityComparandMessage`, shared with the face |
| array under `$ne` (route A) | `stage.$ne` |
`arrayInequalityComparandMessage` with the field, the face's sentence
less its location |
| `null` ordering comparand | `amount.$gt` | read off
`FieldOperatorsSchema`'s slot for the same comparand |
| `null` `$in` / `$nin` member | `stage.$in.1` | read off the slot |
| `null`, blank or `{ $field }` `$between` endpoint |
`amount.$between.0` | read off the slot |
| non-list `$in` / `$nin` | `stage.$in` | NEW: the face's
`nonListComparandError` sentence less ` at PATH`; the slot has only
zod's generic wording |
| `$between` that is not a pair | `amount.$between` | NEW: the face's
`malformedRangeComparandError` sentence less ` at PATH` |
| non-boolean `$null` / `$exists` | `stage.$null` | NEW: `driver-sql`'s
first sentence word for word, then the analytics door's reason and
prescription |

The texts as printed (`FilterConditionSchema.safeParse`):

```text
stage.$null ← { stage: { $null: "x" } }
Operator "$null" on field "stage" requires a boolean comparand (true or false). Received a string ("x"). @objectstack/spec FieldOperatorsSchema declares $null as a boolean, and a non-boolean is refused rather than coerced because the backends read one in OPPOSITE directions — one as IS NULL, another as IS NOT NULL. Write the boolean itself: "$null": true matches rows whose "stage" has no value, "$null": false rows whose "stage" has a value. The filter was NOT applied.

stage.$exists ← { stage: { $exists: "false" } }
Operator "$exists" on field "stage" requires a boolean comparand (true or false). Received a string ("false"). @objectstack/spec FieldOperatorsSchema declares $exists as a boolean, and a non-boolean is refused rather than coerced because the backends read one in OPPOSITE directions — one as IS NULL, another as IS NOT NULL. Write the boolean itself: "$exists": true matches rows whose "stage" has a value, "$exists": false rows whose "stage" has no value. The filter was NOT applied.

stage.$in ← { stage: { $in: "won" } }
Operator "$in" on field "stage" requires an ARRAY of values. Received string ("won"). "$in" tests membership of a list — write ["won"] for a single value, or use "=" ($eq) to compare against it. Authoring spellings: in. The filter was NOT applied, and an unapplied filter would have returned the UNFILTERED result set.

amount.$between ← { amount: { $between: 5 } }
Operator "$between" on field "amount" requires a [min, max] value array. Received number (5). A range needs exactly two bounds, in order; the authoring spelling that lowers to "$between" is "between". The filter was NOT applied, and an unapplied filter would have returned the UNFILTERED result set.

stage.$ne ← { stage: { $ne: ["won","lost"] } }
Operator "$ne" on field "stage" requires a single comparable value, but received an array (["won","lost"]). For "none of these values" use {"$nin": […]} (authoring: nin, not_in, notin). The filter was NOT applied, and an unapplied filter would have returned the UNFILTERED result set.

amount.$gt ← { amount: { $gt: null } }   (the operator slot's existing sentence)
null is not a valid $gt comparand. null is not ordered, and no two evaluation faces agree on what an ordering against it matches (driver-memory's live path reads two absences as equal; its reference matcher compares through JS coercion). State absence with the null predicate instead: {"$eq": null} is "has no value", {"$ne": null} is "has a value". Ruled 2026-09-01: a null ordering comparand is refused at the validation entrance.

stage.$in.1 ← { stage: { $in: ["won", null] } }   (the operator slot's existing sentence)
null is not a valid $in member at index 1. No two backends agree on what a null in a list-comparand position matches (the SQL family answers a NULL under NOT IN unconditionally; the JS matchers split over the two readings of "no value"). State absence explicitly with the null predicate instead: {"$or": [{"$in": […]}, {"$null": true}]} is "one of […] OR has no value", and {"$null": false} is the has-a-value half. Ruled 2026-08-31: a null list member is refused at the validation entrance.

amount.$between.0 ← { amount: { $between: [null, 5] } }   (the operator slot's existing sentence)
null is not a valid $between endpoint at index 0. (… the same null-member sentence as above …)

amount.$between.0 ← { amount: { $between: ["", 5] } }   (the operator slot's existing sentence)
A blank value is not a valid $between endpoint at index 0 (the MIN bound). A closed interval [min, max] requires BOTH endpoints present and non-empty: an empty string is not an interval endpoint at any backend — it is compared as a value, so the range stops bounding on that side while still reading as a complete range. Write the bound you meant; and if only ONE side is genuinely bounded, that is not a range at all — drop $between and write the side you have as a scalar comparison ({"$gte": min} for a lower bound, {"$lte": max} for an upper one). Ruled 2026-09-17: a blank $between bound is refused at the validation entrance.

amount.$between.0 ← { amount: { $between: [{ $field: "floor" }, 5] } }   (the operator slot's existing sentence)
A { "$field": … } reference is not a valid $between endpoint at index 0. No evaluation path resolves a field reference inside a list: the in-memory evaluator (matchesFilter) leaves the list unresolved and compares the raw reference OBJECT, so it silently matches nothing, and both SQL drivers refuse the position with INVALID_FILTER / 400. Write a literal value here, or move the reference to a scalar comparison operator ($eq/$ne/$gt/$gte/$lt/$lte), whose WHOLE comparand a { $field } reference may be. Ruled 2026-08-11: declared = enforced (ADR-0049).
```

A nested member on a dataset carrier prints the same sentence as its
top-level form, with the leaf field named, at its own path, for example
`filter.acct.stage.$in.1`.

The changeset also carries a FROM → TO table for the HTTP doors. `POST
/analytics/dataset/query` answers a one-bound `$between` in
`selection.runtimeFilter` with `400 VALIDATION_FAILED`, located on the
member, instead of `400 INVALID_FILTER` from the normalizer, and `POST
/analytics/query` refuses the same shape in `where` at its request
schema. It also carries rows for the producer's two spellings, `$in:
[null, ""]` and `$nin: [null, ""]`.

### Changed docblocks

- `checkFilterConditionComparands`: a new section, "Every slot the query
faces refuse is refused on save (objectstack-ai#20116)". The old bullet "⛔ `$ne` is
not judged by this walk" is replaced; it was made false.
- `inequalityComparandSchema`: the scope note "its own walk … does not
judge `$ne`" now says the walk refuses the same shape through the face.
- `refuseNestedRelationComparands`: the "⛔ Not judged here: `$ne` … and
the face's OTHER arms inside a nested relation" paragraph is replaced by
"Every slot the door refuses inside a relation (objectstack-ai#20116)".

## What stays accepted (pinned, both doors)

The null predicate (`$eq: null`, `$ne: null`), `$null` / `$exists`
`true` and `false`, and a `{ $field }` reference as the whole comparand
of `$eq` / `$ne` / `$gt` / `$gte` / `$lt` / `$lte`. Also a
column-to-column range as two bounds, `$in: []` / `$nin: []`, `$in` with
a `{ $field }` member (the face does not judge members), a whitespace or
falsy `$between` endpoint, and relation traversal with no operator. On
the shared schema (every carrier but the two dataset ones), any member
shape INSIDE a nested relation stays accepted: neither the face nor the
drivers' flag checks descend one.

## Remainder, named (why `Part of`)

Every member the collector lists (`5854575239`, `5854887743`, and this
card's own) is closed. This run measured two more positions of the same
family:

1. **Nested-relation forms on a dashboard widget `filter` and a report
`runtimeFilter`.** Both reach the analytics `where` door, which refuses
every member inside a relation: `dataset-executor.ts` sends
`combineFilters(compiled.filter, selection.runtimeFilter)` to it. Both
still save the shape, because objectstack-ai#20207's carrier refinement sits on the
two dataset carriers only (triage record `5825670610`). The same gap
holds for objectstack-ai#20080's own equality-list shape on these two carriers.
Closing it means applying the same refinement to
`DashboardWidgetSchema.filter` and the report `runtimeFilter`s, which
are outside this order's file surface.
2. **The comparand-TYPE face's refusals.**
`normalizeFilterComparandTypes` refuses a plain-object or `Map`
comparand, for example `{ stage: { $eq: { a: 1 } } }` or `{ stage: {
$in: [{ a: 1 }] } }`. The analytics door refuses both, in every
position, and every save door accepts both. The collector's enumeration
names the shape face's tables and the flag arm, not the type face, so
this PR does not move it.

## Producer census (Zone 2 answer 4)

A literal-comparand `git grep -P` for each member shape, with a lit
control per shape, over non-test, non-doc files:

| tree | member hits that are authored filters | control hits (`$null` /
`$exists` boolean, `$in: [`, `$between: [`) |
|:--|:--|:--|
| this repo `examples/**` at `af32cf9a` | 0 | 0 / 3 / 0 |
| this repo `packages/**` at `af32cf9a` | 0 (every hit is prose, a type
table or an operator map) | 87 / 230 / 40 |
| objectui at the pin `f8a9d0fb05` | **1 producer**:
`packages/fields/src/widgets/FilterConditionField.tsx:240-241` | 25 / 12
/ 2 |
| cloud `main` `48d70663ab` | 0 (one code comment) | 0 / 11 / 1 |

The producer is objectui's filter-condition widget. `condToMongo` writes
"is empty" as `{ [field]: { $in: [null, ''] } }` and "is not empty" as
`{ [field]: { $nin: [null, ''] } }`. Both are offered by default for
text, number, date, select and lookup fields. `field.form.ts` puts this
widget on `relatedListFilter` and on a rollup's
`summaryOperations.filter`, both `FilterConditionSchema` carriers, and
it also edits `sys_sharing_rule.criteria_json`. The face has refused a
`null` list member on every query since the 2026-08-31 ruling, so such a
filter already fails its related list or rollup. After this PR, the
Studio save is refused instead, at the slot, with the `$or` / `$null`
prescription. No D2 conversion: the 2026-08-31 ruling declined to give a
`null` list member any meaning, so a conversion would have to invent
one. The producer fix is objectui's and is reported to the PM for
routing. Blind spot: a multi-line literal or a runtime-built comparand
is not matched by a line grep.

## Tests

All heavy runs went through `scripts/pm/os-verify-lock.sh`. Each reading
names the tree it was taken on.

| run | tree | reading |
|:--|:--|:--|
| `pnpm --filter @objectstack/spec build`, `check:generated`,
`typecheck` | `c300e80e` (final) | exit 0, exit 0, exit 0 |
| `@objectstack/spec` full suite (`vitest run --maxWorkers=2`) |
`c300e80e` (final) | 579 files, 16731 passed, 2 todo |
| `@objectstack/spec` full suite | `70e9a610` plus the regenerated
registry (`31ddfa40`) | 577 files, 16700 passed, 2 todo |
| the two pin files (`filter-save-door-face-parity.test.ts`,
`dataset-filter-nested-relation-list.test.ts`) | `70e9a610`, after the
ablations were restored | 149 passed |
| consumer: `@objectstack/service-analytics` full suite (the `where`
door) | `70e9a610` | 128 files, 3022 passed |
| consumer: `@objectstack/lint` full suite (`validate-chart-bindings`
and the dataset readers) | `70e9a610` | 109 files, 4232 passed |
| derived gates (`dispatch-gates.mjs --commands`, 87 lines) | `c300e80e`
| 85 exit 0; 2 exit 3 PREREQUISITE NOT MET
(`check:dual-build-cjs-loads`, `check:type-check-debt`, both need the
whole-repo build); `--ran` reconciliation: 87 accounted, 0 unrun |

**Patch round (review 5856977110, CI `Test Core (4/6)` at `c300e80e`).**
`packages/rest/src/analytics-filter-refusal-envelope.test.ts` asserted
the one-bound `$between` as `400 INVALID_FILTER` from the normalizer.
`DatasetSelectionSchema.runtimeFilter` and
`AnalyticsQueryRequestSchema.where` are `FilterConditionSchema`, so the
route's schema door now answers first. Readings on `ab146a9a` (the tree
batch F ran on; the batch header reads `dfca00f0` because the last
commit, a test and changeset edit, landed before its spec step): rest
full suite (`--project local`) 201 files, 3576 passed, 1 skipped; the
flipped file 31 passed; service-analytics full suite 129 files, 3041
passed; spec full suite 581 files, 16770 passed, 1 todo (was 2: the
`$ne` §5 todo is now a pin); spec `check:generated` exit 0; derived
gates 87 derived, 85 exit 0, 2 exit 3 PREREQUISITE NOT MET
(`check:dual-build-cjs-loads`, `check:type-check-debt`), `--ran` 87
accounted. Consumer sweep: a `git grep -P` for every refused member
shape over all non-spec test files found 62 files. Only this one drives
a request or schema door (`DatasetSchema` or `DatasetSelectionSchema` on
`/analytics/dataset/query`, `AnalyticsQueryRequestSchema` on
`/analytics/query`, `ObjectSchema` at registration) with a refused
shape. Every other hit is an engine `where`, an RLS `scope`, a driver
input or a comment.

**Pin sweep.** Two published-behaviour pins flipped and were rewritten
to assert the new semantics with their substance:

- `filter.test.ts` "is not judged by the loose FilterConditionSchema —
and neither is the objectstack-ai#7596 shape" became "is refused by
FilterConditionSchema too, at the endpoint, in the operator slot's
words". It asserts the path `age.$between.1` and equality with
`FieldOperatorsSchema`'s message for the same pair.
- objectstack-ai#20207's §4 control row "`$ne` carrying a list … not yet at this save
door (objectstack-ai#20116)" became a refusal test on both carriers. It asserts
`INVALID_FILTER` / 400 at the analytics door, then the carrier's message
equal to the door's less its location, and the `$nin` remedy.

- **Patch round:**
`packages/rest/src/analytics-filter-refusal-envelope.test.ts`. "a
$between with one bound → 400 INVALID_FILTER" moved from the normalizer
table to the `[objectstack-ai#17551]` door table: it now asserts 400,
`VALIDATION_FAILED`, exactly one `details.fields[]` entry at
`selection.runtimeFilter.amount.$between`, and the face's sentence
without ` at where.`. The sibling-schema control now asserts the refusal
at `where.amount.$between` in the same sentence. The pass-list control
is three spellings, not four. Its objectstack-ai#20010 note moved with the row and
still holds: the sentence is still the face's.
- **Patch round:**
`packages/spec/src/data/filter-ne-array-schema-door.test.ts` §5. Its
`it.todo` ("the carrier still saves the shape") is fulfilled by this
PR's `$ne` arm. It is now a real pin: path `stage.$ne`, message equal to
the face's less its location, the `$nin` remedy, a dataset carrier at
`filter.$or.0.stage.$ne`, and a null / scalar control.

**Ablation (reverse verification), one-off, no permanent file.** Each
leg went through `scripts/ablation-replace.mjs` WRAP mode from the
committed tree `70e9a610`. The anchor hit exactly 1 time (x1 → x0,
replacement x0 → x1), and the restore was proven by blob hash equal to
HEAD with an empty `git diff HEAD`:

| leg | mutation | result on the two pin files |
|:--|:--|:--|
| 1 | the face's verdict dropped (`if (face && false)`) | red: 95 failed
/ 54 passed, both files |
| 2 | the flag rule dropped | red: 18 failed / 131 passed (the flag rows
and the four §1 tables) |
| 3 | the shared walk's reach widened into nested relations (`depth ===
0` removed) | red: 30 failed / 119 passed. More diagnostics, not fewer:
nested slots are refused twice, and the shared reach pins go red |
| 4 | the carrier walk's operator-map arm dropped | red: 28 failed / 121
passed (nested `$eq` / §5 rows) |
| restore | none | 149 passed |

## Acceptance notes

- The face's docblock still names the schema door's twins as
"`nullOrderingComparandMessage` (`./filter.zod.ts`)" and so on. Those
builders did not move; the new module reads the same sentences off
`FieldOperatorsSchema`'s slots. No edit to the face was needed or made.
- `FieldOperatorsSchema.$in` / `$nin` / `$between` / `$null` / `$exists`
still print zod's generic wording for a non-list, a malformed range and
a non-boolean flag. `FilterConditionSchema` prints the pointed sentences
above. Pointing the operator slots too is polish, and is not done here.
- The four sentences read off the operator slot name the operator and
the index but not the field, because the slot cannot see the field. The
issue's path names it.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Sep 28, 2026
… save, and every charted presentation filter judges its nested relations (objectstack-ai#20116) (objectstack-ai#20325)

Fixes objectstack-ai#20116
Clause-②: no (narrowing)

**BREAKING** (an accept-set narrowing at the save doors; the changeset
carries the ADR-0087 disposition `registered
filter-comparand-types-and-widget-nested-slots-refused-at-save`).

Stage 2 of the save-door ↔ query-face parity collector: the two open
members of the seat's release `5857575995`. With the report half folded
in (seat answer `5859432781`, after PR objectstack-ai#20238 landed as `6a6a17b6`),
both members are done, and so is the collector. The objectui producer
stage 1 found is carried by objectui#10790, not by this card.

- **M-type — done.** `FilterConditionSchema` now asks the comparand-TYPE
face (`normalizeFilterComparandTypes`) read-only, after the
comparand-shape face, inside the one judge stage 1 built
(`reportQueryFaceRefusals`,
`packages/spec/src/data/filter-save-door-refusals.ts`). A plain object
where a single value belongs, a `Map`, a class instance, a function, a
Symbol, `undefined` and a bigint beyond ±2^53 are refused on save — as
the comparand, an implicit-equality comparand or a list member — at
every reach the save doors have, and nothing the face passes is refused.
- **M-widget — done.** `DashboardWidgetSchema.filter`,
`ReportSchema.runtimeFilter` and `JoinedReportBlockSchema.runtimeFilter`
declare the analytics-carrier filter the two dataset carriers declare,
so each judges the slots INSIDE a nested relation the way the analytics
`where` door does. objectstack-ai#20207's refinement was inline and module-private in
`dataset.zod.ts`; it moved verbatim into
`packages/spec/src/ui/analytics-carrier-filter.ts` (not in the `ui`
barrel), byte-neutral for `Dataset` (measured below). In `report.zod.ts`
only the two `runtimeFilter` lines (and the import) change.

## Zone 2, measured

### 1. Re-probe (base `origin/main` `17bd3187`; spec doors from `src`,
the analytics door from `service-analytics` `src` over a fresh spec
`dist`)

| member | FilterCondition | Dataset.filter | Measure.filter |
Widget.filter | Report.runtimeFilter | JoinedBlock.runtimeFilter | type
face (top) | analytics door |
|:--|:--|:--|:--|:--|:--|:--|:--|:--|
| `{ stage: { $eq: { a: 1 } } }` | ACCEPT | ACCEPT | ACCEPT | ACCEPT |
ACCEPT | ACCEPT | refuse 400 | refuse 400 |
| `{ stage: { $in: [{ a: 1 }] } }` | ACCEPT | ACCEPT | ACCEPT | ACCEPT |
ACCEPT | ACCEPT | refuse 400 | refuse 400 |
| `{ stage: { $eq: Map } }` | ACCEPT | ACCEPT | ACCEPT | ACCEPT | ACCEPT
| ACCEPT | refuse 400 | refuse 400 |
| `{ stage: Map }` | ACCEPT | ACCEPT | ACCEPT | ACCEPT | ACCEPT | ACCEPT
| refuse 400 | refuse 400 |
| each of the four under `{ acct: … }` | ACCEPT | ACCEPT | ACCEPT |
ACCEPT | ACCEPT | ACCEPT | accept (not descended) | refuse 400 |
| `{ acct: { stage: { $in: ['won', null] } } }` | ACCEPT | refuse |
refuse | ACCEPT | ACCEPT | ACCEPT | accept | refuse 400 |
| `{ acct: { region: ['a'] } }`, `{ acct: { region: { $eq: ['a'] } } }`
(objectstack-ai#20080) | ACCEPT | refuse | refuse | ACCEPT | ACCEPT | ACCEPT | accept
| refuse 400 |
| controls: `$gt: { $field }`, `$gt: Date`, `'{current_user_id}'`, `{
acct: { region: 'NA' } }` | ACCEPT | ACCEPT | ACCEPT | ACCEPT | ACCEPT |
ACCEPT | accept | accept |

After (src, same probe): every M-type cell is refused at the top level
on all six carriers, at the slot (`stage.$eq`) or the member
(`stage.$in.0`); every nested M-type and M-widget cell is refused on all
five analytics carriers (`widgets.0.filter.acct.stage.$in.1`,
`runtimeFilter.acct.stage.$in.1`,
`blocks.0.runtimeFilter.acct.stage.$in.1`); bare `FilterConditionSchema`
still accepts the nested cells (the face's reach); every control is
accepted.

### 2. One judge

The type face is asked inside `reportQueryFaceRefusals`, the function
both walks call, exactly where stage 1 asks the shape face — so it
reaches the shared walk (`checkFilterConditionComparands`) and the
analytics carriers' nested walk at once, with no second walk. The judge
raises ONE issue per slot, the first the query doors give in their order
(shape face → type face → flag rule; `parseFilterAST`, the engine seam
and `normalizeWhereComparands` all run them in that order). At the top
level and in the combinators, where a face refuses a slot, the shared
walk's own `objectstack-ai#19514` `$icontains` and `objectstack-ai#8793` preset arms stay silent on
that slot. Inside a nested relation on an analytics carrier they do not:
those two arms still judge nested slots beside the carrier walk's faces,
so a nested `$icontains` with a type-refused comparand, or a nested
one-bound `$between` of a preset name, carries two issues (the review
measured 303 such cells new at the head). No verdict moves either way;
the changeset and the entry say exactly this.

What the type face "judges only at request time" was measured: nothing.
The face is context-free (`context` is a message prefix only). The
request-time values are `{ $field }` references (the face steps around
them), `Date`s (accepted), and `{placeholder}` strings such as
`{current_user_id}` / `{today}` — strings at save time, resolved by
`resolveWhereTokens` only AFTER both faces have run on the engine seam.
Each is pinned accepted-and-kept (§4 controls), plus a bigint within
2^53, which the save door keeps as written (the face would narrow it on
a query).

Two classifications follow the type face rather than restating it: a
field value is a comparand unless it is a PLAIN object (prototype
`Object.prototype` / `null`), so `{ stage: new Map() }` reaches the face
instead of being walked as an empty nested relation; and the whole field
entry is shown to the face first, so a spec it classifies as a `{ $field
}` reference is stepped around whole, as the face does (pinned as a
control).

### 3. The dashboard and report carriers — extraction, byte-neutral

- The move is its own commit (`dfa424f6`), verbatim; `DatasetSchema` /
`DatasetMeasureSchema` call `analyticsCarrierFilter()` as before.
- `z.toJSONSchema` of `DatasetSchema`, `DatasetMeasureSchema`,
`DashboardSchema` and `DashboardWidgetSchema`: sha256
`246850f2b9efdf1b…` at base `17bd3187`, at the extraction commit, and
after the widget carrier — byte-identical. With `ReportSchema` and
`JoinedReportBlockSchema` added, the six projections hash
`35ba34f964bb8fd6…` both before the report fold (`6a0cfb05`) and after
it. The parse probe is identical base vs extraction.
- `dropped-refinements.baseline.json`: the `ui/Dataset` and
`ui/DatasetMeasure` rows are unchanged. The merge of `6a6a17b6`
conflicted only in this ledger's measured header; main's side was taken
and the branch's sites re-added from build-schemas' own printed
"corrected entries" (no hand-picked site): `ui/DashboardWidget`
`filter`, `ui/Dashboard` `widgets.element.filter`, `ui/Report`
`runtimeFilter` and `blocks.element.runtimeFilter`,
`ui/JoinedReportBlock` `runtimeFilter`, and the same three positions in
the four installed-package envelopes — +17 sites, 0 removed,
`measured.droppedRefinementSites` 605 → 622.
- Pin: §7 of the parity test asserts the carrier projects to exactly
`FilterConditionSchema.optional()`'s JSON Schema, and that the widget
filter and the report `runtimeFilter` keep their published descriptions.

### 4. The enumerating pin

`filter-save-door-face-parity.test.ts`, extended, not duplicated:

- §1 `queryFacesRefuse` now asks all three rules. The operator arms
still derive from `FieldOperatorsSchema`'s keys, and a new assertion
requires the type face to judge EVERY declared operator over the battery
(its own test reconciles its scalar/list split against the same
vocabulary). The battery gained the type face's shapes and neighbours
(Map, class instance, function, Symbol, `{ $field: 5 }`, `{}`, bigints
within and beyond 2^53, lists holding a plain object / Map / `undefined`
/ big bigint, a plain-object `$between` bound, bigint pairs).
- §5 runs the operator × comparand table and the implicit slot, one and
two hops down, on every analytics carrier — dataset `filter`, measure
`filter`, dashboard widget `filter`, report `runtimeFilter`, joined
block `runtimeFilter` (the two report rows were `EXPECTED_OPEN` until
the fold and now sit in `CARRIERS`), plus a pin that the carrier list is
exactly those five.
- §6 walks the type face's own conformance table
(`FILTER_COMPARAND_TYPE_CASES`): every `door-refusal` row is refused on
save; every `matches` / `compiles` row is accepted AND kept as written.

### 5. Producer census (narrowing), with lit controls — 0 hits

| corpus | object in a scalar slot | object list member | bigint literal
| `undefined` under an operator | `new X` under an operator | nested
relation holding a list / operator map in a filter |
|:--|:--|:--|:--|:--|:--|:--|
| objectstack `examples/**` @ `eaf7a925` | 0 | 0 (control: `$in` lists
3) | 0 | 0 | 0 | 0 (control: filters with an operator-map first entry
14) |
| objectstack non-test `packages/**` @ `eaf7a925` | 25 raw, all prose /
driver `case` labels / the type face's own table (control: `$field` in a
scalar slot 67) | 26 raw, all prose / `$field` members / `{placeholder}`
strings (control 238) | 3, prose | 31, prose (control: `null` 83) | 6,
`new Date` / the table | 0 (control 162) |
| objectui @ pin `f8a9d0fb05` | 0 (control 1) | 0 (control 12) | 0 | 1,
a comment | 1, a refusal message | 0 (control 13) |
| cloud `main` @ `96eb092fbf` | 0 (control 2) | 1, a comment (control
11) | 0 | 0 | 0 | 0 (control 14) |

Plus a runtime walk of every value under a `filter` / `runtimeFilter` /
`where` / `having` / `relatedListFilter` key in the loaded example
stacks, old door vs new door on each: `app-crm` 8, `app-todo` 15,
`app-multi-package` 0, `app-showcase` 20 (its metadata modules; its
config needs connector builds) — 0 refused by the new door alone. Lit
control: a planted `{ stage: { $eq: { a: 1 } } }` and a planted nested
`$in` null member fire the detector in every run. No ADR-0087 D2
conversion: nothing to convert.

## The words (changed or new refusal text)

A type-face cell reads the face's own sentence less its ` at where.SLOT`
clause — nothing restated:

> `Filter comparand is a plain object ({"a":1}), which no driver can
compare. A comparison value must be a string, number, bigint, boolean,
null or Date. Refusing rather than guessing: the backends disagreed on
this input (crash / zero rows / silently edited query). The filter was
NOT applied, and an unapplied filter would have returned the UNFILTERED
result set.`

at `filter.stage.$eq`, or at the member (`filter.stage.$in.1`).
`undefined` gets the face's own sentence (`Filter comparand is
undefined. { key: undefined } cannot be told apart from an omitted key,
… Write the null predicate — {"$eq": null} / {"$ne": null} — or omit the
key. …`), a bigint beyond 2^53 its (`Filter comparand is the bigint …n,
whose magnitude exceeds 2^53 — …`). The clause removed is the one the
face was handed (`where` plus this slot), so nothing is parsed out of
the text; if the face ever spells its location differently, the whole
message is reported location included, and §2's "no ` at where.`" pin
goes red.

Nested cells on the widget and report carriers print the same sentence
as their top-level form (stage 1 and objectstack-ai#20207's rule): e.g.
`widgets.0.filter.acct.stage.$in.1` and `runtimeFilter.acct.stage.$in.1`
carry the enforced `$in` slot's null-member sentence.

## Behaviour changes, each pinned

| change | pin |
|:--|:--|
| type-face cells refused on save, top level + combinators, every
`FilterCondition` carrier | parity §1 (all positions), §3 (six
carriers), §6 |
| type-face cells + stage-1 cells + objectstack-ai#20080 lists refused INSIDE a
relation on the widget filter and both report `runtimeFilter`s | parity
§5 (all five carriers) |
| one issue per slot at the top level and in the combinators, shape →
type → flag; the `$icontains` / preset arms silent on a face-refused
slot there (not inside a relation on an analytics carrier) — a dedupe,
no verdict moves; stated at that reach in the changeset | parity §6 "one
slot, one issue" (top level) |
| `POST /analytics/dataset/query` `selection.runtimeFilter` and `POST
/analytics/query` `where` with `{ stage: { $eq: { a: 1 } } }` / `{
stage: { $in: ['won', { a: 1 }] } }`: `400 INVALID_FILTER` → `400
VALIDATION_FAILED` located on the member |
`packages/rest/src/analytics-filter-refusal-envelope.test.ts`
`AT_THE_DOOR` rows + the sibling-schema control |
| request-time values accepted and kept | parity §4 |

## Pin sweep

① Every refusal code and message this touches was grepped repo-wide. The
type face's text is unchanged (only called). The HTTP-door code move has
two routes and both are pinned in the rest file above; no rest / runtime
test sent a type-face cell through a schema door before (grep over
`packages/**` tests outside spec: the plain-object / Map / `undefined` /
bigint comparand hits live in the analytics door's, the drivers',
objectql's engine and read-scope suites, which call the faces directly,
not a schema). Spec pins whose words could move — a flag with an object
/ `undefined` comparand, `$icontains` with a type-refused comparand, a
preset endpoint on a malformed `$between` — have no existing pin. ② The
flipped rest rows assert the substance: status 400, `VALIDATION_FAILED`,
exactly one `details.fields[]` entry at the member, the sentence, and no
` at where.`.

## Tests

Final head `830a071a` (merges `origin/main` `dfd8e398`, then corrects
the dedupe sentence of the changeset and the semantic entry — text
only), everything through `scripts/pm/os-verify-lock.sh`, `VERDICT
command-exit 0`:

- `@objectstack/spec` at `830a071a`: build 0; `check:generated` 0 ("All
15 generated artifacts are up to date"); `typecheck` 0; full suite 587
files / 17031 passed / 1 todo; the parity pin alone 153 passed.
`check-adr-0087-registration` 0 (`[BREAKING+bang+clause-②-narrowing]
registered
filter-comparand-types-and-widget-nested-slots-refused-at-save`) and
`check-changeset-no-major` 0 at `830a071a`.
- At `e9f93902` (review head): spec full suite 585 files / 16988 passed
/ 1 todo.
- At `3d9621a8` (the fold plus the ledger, before the second `main`
merge, which touches no file of this diff's behaviour): spec 585 / 16955
passed; consumer closures built (exit 0); `service-analytics` 129 files
/ 3041 passed; `lint` 111 / 4297; `rest` (`--project local`) 202 / 3664
passed / 1 skipped. `rest` typecheck 0 at `54b99f3c` (the rest file is
unchanged since). The two new rest rows ran by name (`a plain object
where a single value belongs → 400 VALIDATION_FAILED, located on the
member`, `a plain object as an $in member → …`).
- Ablation, through `scripts/ablation-replace.mjs` (WRAP), each anchor
x1 → x0 on disk with the blob changed, each restore proven blob == HEAD
blob and `git diff HEAD` empty (script trap restores on EXIT/INT/TERM).
Parity pin:
- at `3d9621a8`, 153 tests — **report-carrier arm**:
`ReportSchema.runtimeFilter` carrier stripped
(`analyticsCarrierFilter().unwrap().optional()`) → 18 failed;
`JoinedReportBlockSchema.runtimeFilter` stripped → 18 failed; restored →
153 passed (`unwrap` markers on disk after restore: 0);
- at `54b99f3c`, 152 tests — type face off → 48 failed; widget carrier
off → 18; shared walk's Map / class-instance classification off → 6;
nested walk's classification off → 3; one-issue-per-slot `continue` off
→ 1 (direction: MORE diagnostics — two issues at `name.$icontains`);
restored → 152 passed.
  - No leg needs a build: the pin imports spec `src`.
- Gates at `830a071a`: `dispatch-gates --commands` derived 90 on the
actual paths (the 89 of the review head plus `node
scripts/check-issue-citations.mjs`, which main's `7338efe0` now runs
locally); 88 exit 0 — `check:doc-formula-expressions` and
`check:lean-entry-closure` first answered exit 3 (their `formula` /
`objectql` builds were absent in the re-created worktree) and exit 0
after that closure was built under the lock;
`check:dual-build-cjs-loads` and `check:type-check-debt` exit 3
(PREREQUISITE NOT MET: whole-repo build) = NOT MEASURED; `--ran` with
recorded codes: 90 accounted, 88 run, 2 NOT MEASURED.
- ESLint, narrowed and proven, at `830a071a`: `eslint --no-inline-config
--format json` over the 10 changed `.ts` files (count read from the
JSON) → 0 errors, 0 warnings. The population is `eslint.config.mjs`,
which "never enables type-aware linting (no `parserOptions.project`, no
typed `@typescript-eslint` rules) for ANY file", so the diff cannot move
a verdict on an untouched file. Repo-wide `pnpm lint` is CI's.
- `main` moved after `830a071a` too (2 commits at the gate derivation,
none touching what this answer derives from, as `dispatch-gates` read
it).
- NOT MEASURED: `objectql`, `metadata-protocol`, `runtime` and example
suites (no fixture carries a refused shape through a schema door by the
sweep above; CI runs them).

## Hand-written docs the drift check named (github-actions comment
`5860213966`)

Each re-read against this PR's behaviour — the type-face refusals at
save, the widget / report `runtimeFilter` nested-relation refusals, and
the analytics routes' code move for the type face's JSON cells:

| page | verdict | why |
|:--|:--|:--|
| `content/docs/api/data-api.mdx` | unchanged | Its `INVALID_FILTER`
sentences are about the data routes' `?filter` (the engine's normalizer,
not a `FilterConditionSchema` parse), and the `/analytics/query` section
says only that `where` is the `FilterCondition` `find()` accepts — still
true; no sentence names the code a type-face cell gets. |
| `content/docs/api/error-catalog.mdx` | unchanged | `INVALID_FILTER` /
`VALIDATION_FAILED` are defined generically; no sentence claims the
analytics routes answer `INVALID_FILTER` for a plain-object comparand. |
| `content/docs/data-modeling/analytics.mdx` | unchanged | The widget
`filter` and report `runtimeFilter` examples (`$nin` list,
`{current_quarter_start}` placeholder) are accepted by the new doors;
the placeholder paragraph and the "one author-facing shape" section stay
true; nothing says a nested-relation filter saves on those carriers. |
| `content/docs/protocol/objectql/query-syntax.mdx` | unchanged | Its
`FilterConditionSchema` / field-reference / relation-traversal text
stays true: `{ $field: 'col' }` is still accepted, and the page makes no
claim about a plain-object, `Map` or `undefined` comparand passing
validation. |

No release page (`content/docs/releases/**`) names these shapes; none
was touched.

## Acceptance notes

- **Request doors keep two codes for one family.**
`DatasetSelectionSchema.runtimeFilter` /
`AnalyticsQueryRequestSchema.where` carry the shared reach, so a
top-level refused slot answers `VALIDATION_FAILED` at the schema door
while the same slot inside a relation answers `INVALID_FILTER` from the
analytics normalizer. Both 400, both located; not a save door, so not
this collector's. Noted, not filed (carrier: none).
- **File surface**, as amended by the seat (`5859432781`):
`report.zod.ts` (the two `runtimeFilter` carriers only); and, accepted
as the order's own mechanism, `data/filter-save-door-refusals.ts` (stage
1's judge), `ui/analytics-carrier-filter.ts` + `ui/dataset.zod.ts` (Zone
2.3's extraction),
`packages/rest/src/analytics-filter-refusal-envelope.test.ts` (the
HTTP-door pin).
- `GlobalFilterOptionsFromSchema.filter` (a dashboard's options source)
is an engine query, not charted through the analytics door, so it keeps
the shared reach.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:ui size/l tests tooling

Projects

None yet

2 participants