Skip to content

fix(service-analytics)!: refuse a caller-supplied analytics member that is not a column reference at the door - #21190

Merged
objectstack-fleet[bot] merged 6 commits into
mainfrom
claude/issue-21177-analytics-caller-content-admission
Oct 1, 2026
Merged

objectstack-fleet[bot] merged 6 commits into
mainfrom
claude/issue-21177-analytics-caller-content-admission

Conversation

@objectstack-fleet

@objectstack-fleet objectstack-fleet Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #21177

Clause-②: no (narrowing)

What

The analytics doors evaluated caller-supplied content at query time that neither the object-level nor the field-level read admission could judge. After merging main, this PR is narrowed to the residual that main does not yet judge.

The /analytics/query half is superseded

main landed #21156 (as #21173): a caller-named member that is neither a declared cube member nor a column reference is refused at the /analytics/query door in every tier, through the single field-read judge (PERMISSION_DENIED / 403 — "one judge, one shape, no new error code"). This PR therefore:

The residual: a dataset's own field text

The remaining gap is content main does not judge: the dimension and measure field text of a dataset sent to POST /api/v1/analytics/dataset/query. The service compiles that dataset into a cube whose members read as declared, so a dimension's or measure's field that is a raw expression resolves to a declared cube member — and #21156 leaves a declared expression member to the field-level gate, which stands down with no security service and on an object its reader answers undefined for. In those tiers the expression reached the native statement as written.

This PR judges the dataset's own field text at the dataset door, before the dataset is compiled and before any strategy runs, through main's existing judge (assertCallerMembersJudgeable / fieldReadUnjudgeableError): PERMISSION_DENIED / 403, naming the member and never the expression text, for every caller (admin included) and whether or not a security service is wired. No new error code and no second admission module — caller-content-admission.ts is removed.

What is left to the existing gates, unchanged:

Inline and saved alike. The route hands this door an inline dataset (body.dataset) and a saved one alike: it loads body.datasetName from metadata and calls the same queryDataset, and the build probe calls it with a saved dataset too. The service cannot tell the branches apart, so the refusal is uniform, which is the safer reading. A saved dataset whose field is an expression is refused the same way as an inline one. No shipped dataset carries a non-column field (examples/app-crm, examples/app-showcase, examples/app-todo and platform-objects were read; the showcase done_rate is a derived measure, not a field expression). Refusing such a field when it is authored belongs to the dataset schema's own retirement of expression fields, a separate change.

This aligns the caller-supplied dataset surface with ADR-0021's "zero raw expressions".

What stays answerable

Every dataset (inline or saved) and cube query that names columns, relationship paths and declared members — the whole legitimate author and query surface — is unchanged. A plain-column inline dataset, a saved plain-column dataset and a dataset registered through the configuration door are all still answered. Only a dataset field whose text is a raw expression is refused, inline or saved; the BREAKING note in the changeset covers both.

Scope / serial

Tests

Review

Security boundary. The maintainer has authorized landing; the first merge-queue build failed on the semantic overlap with #21156 described above, and this push carries the narrowing. Its draft/ready state is unchanged by this push, and it is not queued or armed for auto-merge here — the owning seat re-enqueues once CI is green on this head.

Thread receipts (round 5)

read 2026-10-01T17:23Z.


Generated by Claude Code

claude added 2 commits October 1, 2026 14:30
…t is not a column reference at the door

A member a caller supplies at query time — an inline dataset dimension/measure
field or own filter, or a member an authored cube does not declare — that is not
a plain column reference (a field, a relationship path ending in one, or a count)
is refused INVALID_FIELD / 400 at the analytics door, before any strategy runs,
for every caller and whether or not a security provider is wired. It is an
invalid request, not a permission verdict; a declared cube member whose own
definition is an expression stays the field gate's verdict. Aligns the
query-time surface with ADR-0021's "zero raw expressions".

New caller-content-admission module beside the field-read admission, wired at the
service door (queryDataset before compile; callCtx for the cube-query doors),
with unit and door-level tests.

Claude-Session: https://claude.ai/code/session_01MRdbfpy4sQT8bUjmMhxsN7
Co-authored-by: Claude <noreply@anthropic.com>
…caller-expression path

#21177 refuses a caller-supplied inline dataset field that is not a column
reference at the analytics door, so a raw `translate(...)` dimension can no longer
reach the real driver from caller content. The first block now pins that door
refusal (400 INVALID_FIELD) beside a positive control; the #16019 relay stays
covered by block 2 (a throwing double) and the driver declaration by driver-sql's
own sql-driver-16019 test.

Claude-Session: https://claude.ai/code/session_01MRdbfpy4sQT8bUjmMhxsN7
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation tests tooling labels Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/service-analytics, touching 4 documentable anchor(s).

⛔ 1 release-owned page(s) name something this change touched. These are read-only:

  • content/docs/releases/v17/17-5.mdx (via AnalyticsService (symbol, a top-level class))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 10 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 862f12c0b9b36a6192f6ffe55f4052fa7dbaf34a → packageMentionDocs.

Which tree this was computed on

This run read content/docs from c3663627264748d3580544ebba9518af85b1e6ea — the merge of head aea35ada9cbd0e1d9a642120f9a8b6ff519de6c4 into base 862f12c0b9b36a6192f6ffe55f4052fa7dbaf34a, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin c3663627264748d3580544ebba9518af85b1e6ea && git checkout c3663627264748d3580544ebba9518af85b1e6ea
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 862f12c0b9b36a6192f6ffe55f4052fa7dbaf34a aea35ada9cbd0e1d9a642120f9a8b6ff519de6c4 && git checkout -B drift-repro 862f12c0b9b36a6192f6ffe55f4052fa7dbaf34a && git merge --no-ff aea35ada9cbd0e1d9a642120f9a8b6ff519de6c4

node scripts/docs-audit/affected-docs.mjs --json 862f12c0b9b36a6192f6ffe55f4052fa7dbaf34a

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 862f12c0b9b36a6192f6ffe55f4052fa7dbaf34a → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Landing note: this is a permission-boundary change (human floor), so the PR stayed draft until the maintainer decided. The maintainer authorized the landing on #21177, selecting verbatim 「绿了就转 ready + 进队列(推荐)」 in Claude Code session session_01MRdbfpy4sQT8bUjmMhxsN7 on 2026-10-01. They then gave a standing instruction for this session, verbatim: 「绿了就应该进队列合并啊,为什么一直问我」. CI on head 5428f7e627 is 31 success / 3 skipped / 0 failed. The PR is now ready with auto-merge armed, and the merge queue re-validates on the merged generation.


Generated by Claude Code

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 36885164397 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Test Core (4/6) — 失败步骤: Run this shard's tests

    @objectstack/service-analytics:test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'NativeSQLStrategy' > 'no security pro
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    @objectstack/service-analytics:test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'NativeSQLStrategy' > 'admin — provide
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    @objectstack/service-analytics:test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'NativeSQLStrategy' > 'non-admin — pro
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    @objectstack/service-analytics:test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'ObjectQLStrategy' > 'no security prov
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    @objectstack/service-analytics:test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'ObjectQLStrategy' > 'admin — provider
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    @objectstack/service-analytics:test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'ObjectQLStrategy' > 'non-admin — prov
      ↳ 失败原因: @objectstack/service-analytics:test: AssertionError: expected Error: [Analytics] Access denied: member … { …(4) } to match object { code: 'INVALID_FIELD', …(3) }
    @objectstack/service-analytics:test:  FAIL  src/__tests__/field-read-admission-gate.test.ts > [#20965] the field-level read gate — a member that names no field is refused, never stood down > 'NativeSQ
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    @objectstack/service-analytics:test:  FAIL  src/__tests__/field-read-admission-gate.test.ts > [#20965] the field-level read gate — a member that names no field is refused, never stood down > 'ObjectQL
      ↳ 失败原因: @objectstack/service-analytics:test: AssertionError: expected Error: [Analytics] Access denied: member … { …(4) } to match object { code: 'INVALID_FIELD', …(2) }
    
  • Temporal Conformance (live PG + MySQL) — 失败步骤: Run the non-SQL temporal backends under the skewed process zone

    packages/services/service-analytics test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'NativeSQLStrategy' > 'no securit
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    packages/services/service-analytics test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'NativeSQLStrategy' > 'admin — pr
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    packages/services/service-analytics test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'NativeSQLStrategy' > 'non-admin 
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    packages/services/service-analytics test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'ObjectQLStrategy' > 'no security
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    packages/services/service-analytics test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'ObjectQLStrategy' > 'admin — pro
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    packages/services/service-analytics test:  FAIL  src/__tests__/caller-content-admission-door.test.ts > [#21177] caller-content admission — the /analytics/query door > 'ObjectQLStrategy' > 'non-admin —
      ↳ 失败原因: packages/services/service-analytics test: AssertionError: expected Error: [Analytics] Access denied: member … { …(4) } to match object { code: 'INVALID_FIELD', …(3) }
    packages/services/service-analytics test:  FAIL  src/__tests__/field-read-admission-gate.test.ts > [#20965] the field-level read gate — a member that names no field is refused, never stood down > 'Nat
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    packages/services/service-analytics test:  FAIL  src/__tests__/field-read-admission-gate.test.ts > [#20965] the field-level read gate — a member that names no field is refused, never stood down > 'Obj
      ↳ 失败原因: packages/services/service-analytics test: AssertionError: expected Error: [Analytics] Access denied: member … { …(4) } to match object { code: 'INVALID_FIELD', …(2) }
    

↳ 失败原因 是判读的关键:超时(Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言(AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

⚠️ 断言这一侧有一类例外,判据是断言在测什么,不是它是不是 AssertionError。 断言的对象是产品行为(一个值、一个形状、一次拒收)⇒ 照上面读:真实的行为改变,去查,⛔ 不要重排掉;
断言的对象是这次实验自身的有效性前提(跑完的耗时、负载下的先后、任何只在时间预算内才成立的条件)⇒ 它跟超时是同一类,同样对负载敏感,重排一次是合法的判别手段。
识别是机械的:断言的消息或它比较的值本身点名了一段时长、一个时间戳、一个耗时计数。实测过的一对 —— AssertionError: SecurityPlugin.init() ran: expected false to be true 测的是产品行为(真回归);
AssertionError: this run took over a second, so second-precision stamps could have differed too: expected 1006 to be less than 1000 测的是实验前提:它守护的那条不变式当时是绿的,同一个 head 原样重排一次即成功。
穿着 AssertionError 外衣的时间测量,仍然是时间测量。(⛔ 这只改「怎么读一次红」,不改「哪些测试可以重排」——后者由别处管。)

跨 PR 相同签名(24h,按失败测试文件聚合):

  • src/__tests__/caller-content-admission-door.test.ts — 24h 窗口内只有本 PR 撞到过,暂不汇总(再有一个不同 PR 撞到就会自动开汇总 issue)。
  • src/__tests__/field-read-admission-gate.test.ts — 24h 窗口内只有本 PR 撞到过,暂不汇总(再有一个不同 PR 撞到就会自动开汇总 issue)。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 1 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

…d field expression at the analytics door

Narrows #21177 to the residual main does not judge after #21156 (#21173)
landed. The /analytics/query door half — a caller-named non-column member —
is superseded by #21156's single judge (PERMISSION_DENIED / 403), so this
drops the duplicate handling of that class and reverts the #20965 pin on
field-read-admission-gate.test.ts to main.

The residual is an INLINE (caller-POSTed) dataset's own dimension/measure
`field` text: the service compiles the dataset into a cube whose members read
as declared, so a `field` that is a raw expression resolved to a declared
cube member and was left to the field-level read gate, which stands down with
no security service and on an object its reader answers `undefined` for. In
those tiers the expression reached the native statement as written.

The dataset's own `field` text is now judged at the dataset door, before
compile and before any strategy runs, through main's existing judge
(assertCallerMembersJudgeable / fieldReadUnjudgeableError): PERMISSION_DENIED
/ 403, naming the member and never the expression text, for every caller
(admin included) and with or without a security service. No new error code
and no second admission module; caller-content-admission.ts is removed. The
dataset's filter, the selection's runtime filter and cube-query members are
lowered into the compiled query and already judged on the query path, so they
are unchanged; a registered dataset's own field text is author text and stays
with the existing gates.

Claude-Session: https://claude.ai/code/session_01MRdbfpy4sQT8bUjmMhxsN7
Co-authored-by: Claude <noreply@anthropic.com>
…lared as an accept-set narrowing

The dataset door now refuses an inline dataset whose dimension or measure
`field` is not a column reference, where it was evaluated before: an
accept-set narrowing. Declare it `Clause-②: no (narrowing)` with the
ADR-0087 disposition and the BREAKING banner, and raise
@objectstack/service-analytics to `minor` under the launch-window
convention for narrowings.

Claude-Session: https://claude.ai/code/session_01MRdbfpy4sQT8bUjmMhxsN7
Co-authored-by: Claude <noreply@anthropic.com>
…taset's field too, and pin the saved branch

The `/analytics/dataset/query` route's `body.datasetName` branch loads a
saved dataset from metadata and calls the same `queryDataset`, and the build
probe does too, so the door judges a saved dataset's own `field` text exactly
as an inline one's. Behaviour is unchanged; the docblock, the door test's
header and the changeset said a registered dataset never reaches this door,
which holds only for one registered through the configuration door and
queried by cube name.

- analytics-service.ts: correct the docblock (every branch that supplies the
  dataset; author-time refusal is the dataset schema retirement's job) and
  rename the private method to assertDatasetFieldsJudgeable.
- changeset: the BREAKING note covers saved datasets with a non-column
  field; no shipped dataset carries one.
- rest: pin the saved branch through the real route and driver — a saved
  dataset with a non-column field is refused 403 with nothing executed, and
  a saved plain-column dataset is still answered.

Claude-Session: https://claude.ai/code/session_01MRdbfpy4sQT8bUjmMhxsN7
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 1, 2026 17:33
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Re-landing note: the merge-queue run on the previous head failed because of a semantic overlap with #21173 (#21156) on main. That overlap is reconciled in rounds 5 and 6; the PM review record is on #21177. The PR stays ready. Auto-merge is re-armed on head aea35ada9c, so the PR enters the merge queue only after its required checks pass. This follows the maintainer's standing instruction for session session_01MRdbfpy4sQT8bUjmMhxsN7, verbatim: 「绿了就应该进队列合并啊,为什么一直问我」.


Generated by Claude Code

@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 1, 2026
Merged via the queue into main with commit ce4e205 Oct 1, 2026
42 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-21177-analytics-caller-content-admission branch October 1, 2026 18:03
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Oct 7, 2026
…a column reference (objectstack-ai#21220) (objectstack-ai#21240)

Fixes objectstack-ai#21220
Clause-②: yes

Dispatched by the PM claim `5938507454` (PM loop round 1, `domain:spec`
seat 1), on the triage direction `5938409101`. An ADR-0021 dataset's
`dimensions[].field` and `measures[].field` now take the
column-reference accept set the cube members they compile to already
hold since objectstack-ai#20943 (PR objectstack-ai#20998), from ONE shared declaration. A non-column
value is refused at parse, at `dimensions.N.field` / `measures.N.field`,
with a prescription naming the ADR-0021 form. The runtime door from PR
objectstack-ai#21190 is untouched and stays as defence in depth. The changeset carries
the `(narrowing)` arm, the BREAKING banner at `minor`, and the ADR-0087
marker `registered dataset-member-field-expression-refused,
dataset-count-measure-empty-field-removed`.

## Patch round 2 — contract review `5940617829`, item ①.6

The review found one lossless sub-shape that the first round sent to D3
only: a dataset measure `{ aggregate: 'count', field: '' }`. It parses
on the base, and the objectstack-ai#21190 door skips it (its `!== ''` guard). On
SQLite's native path it compiled to `COUNT()` and answered 200. Its
producer is Studio's dataset inspector. This round:

- **New D2 conversion `dataset-count-measure-empty-field-removed`** in
`MAJOR_18_CONVERSIONS` (`order: 54`, inserted at its identifier's sort
position, defined directly above `elementFilterRemoved`). `main` landed
`form-field-public-picker-removed` at 53, so this entry takes the next
free number.
- It drops `field` from a `count` measure whose `field` is exactly `''`.
Without the key, `compileDataset` emits `sql: m.field ?? '*'`, which is
`COUNT(*)`: the row count.
- Its mechanics are measured from the precedent
`time-default-utc-suffix-dropped`, not recalled: `toMajor: 18`,
`retiredFromLoadPath: true` (ADR-0087's ratified pre-GA policy for a
lossless repair), and `retiredAfter: '17.5.0'` (the spec's current
version, the same value the precedent carries).
- It uses the shared `stripKeys` helper and emits one notice per removed
key.
- Its fixture is disjoint. The controls are left as stored: a dimension
`''`, a `sum` over `''`, a count over `'*'`, a count with no `field`,
and a count over a column.
- **Scope: only `count` + `''`.** A non-count measure with `''`, a
dimension with `''` and every expression have no working row or no
mechanical rewrite, so they stay D3-only. A padded value has no known
producer and is out of scope.
- **The D3 entry links the conversion** with `conversionIds:
['dataset-count-measure-empty-field-removed']`, the way
`18.time-default-zone-refused.ts` links its conversion. Its `reason` now
says what is true: the door refuses an expression, and it never judged
`''`. It also says that D2 carries the `count` + `''` repair and D3 the
rest. `acceptanceCriteria`, the `STEP18_RATIONALE` fragment, the
changeset, the two ledger notes and the `dataset.zod.ts` comment are
corrected to match. `registry.ts` was regenerated by
`gen:migration-registry`.
- **Pins.** The new
`src/conversions/dataset-count-measure-empty-field-removed.test.ts`
covers four things on a STORED row, through
`applyConversionsToStoredItem('dataset', row)`:
- The key is dropped, with one notice per measure, and the row then
parses.
  - Every control is the same reference.
  - The replay is idempotent.
- The conversion is registered under major 18, retired from the load
path, and linked from the D3 entry.

The table-wide fixture replay in `conversions.test.ts` covers before →
after.
- **What a NEW save does.** The write path parses with the current
schema and replays no conversion, so a new Studio save of `field: ''` is
still refused at save with the prescription to omit the key. The
producer-side change stays objectui's. A row already stored that way is
repaired on load at every stored-row seam.

## What changes

**`@objectstack/spec`**

- **One pattern.** The new module
`packages/spec/src/data/analytics-column-reference.ts` declares the
column path once (a bare identifier, then zero or more `.identifier`
hops). It sits outside the `data` barrel, like
`ui/analytics-carrier-filter.ts`, so it is not published API. It exports
two anchored forms built from that one source string:
  - `ANALYTICS_COLUMN_REFERENCE`: the path, or `'*'`.
  - `ANALYTICS_COLUMN_PATH`: the same path without the `'*'` arm.
- **Cube layer unchanged.** In `data/analytics.zod.ts`,
`CUBE_MEMBER_SQL` is now that same `RegExp` object: `const
CUBE_MEMBER_SQL = ANALYTICS_COLUMN_REFERENCE`. The declaration stays in
this file on purpose. ADR-0021's 2026-10-01 note links to
`analytics.zod.ts#CUBE_MEMBER_SQL`, and ADRs are a governed surface this
PR does not edit. The cube members' JSON-Schema `pattern` is
byte-identical; the new pin asserts it.
- **Dataset layer.** In `ui/dataset.zod.ts`:
- `DatasetMeasureSchema.field` uses
`.regex(ANALYTICS_COLUMN_REFERENCE)`. Admitted: a column, a relationship
path, or `'*'`. A count may still omit `field`.
- `DatasetDimensionSchema.field` uses `.regex(ANALYTICS_COLUMN_PATH)`,
so a dimension also refuses `'*'` (see measurement 3).
- Both are `.regex()`, not refinements, so the published JSON Schema
carries each as a `pattern`. `dropped-refinements.baseline.json` is
untouched.
- The refusal code is `invalid_format`. Each prescription opens with the
contract sentence and names ADR-0021.
- The measure prescription names the measure `filter` form and `derived:
{ op, of: [...] }`, with the 0–1 ratio scale.
- The dimension prescription says a CASE bucket becomes a field of the
object.
- The two `describe()` texts now say "never a SQL expression".
`content/docs/references/ui/dataset.mdx` is regenerated from them.
- **ADR-0087.**
- New D3 entry
`migrations/entries/semantic/18.dataset-member-field-expression-refused.ts`.
`registry.ts` was regenerated by `gen:migration-registry` and never
hand-edited inside the markers.
- One hand-written `STEP18_RATIONALE` fragment, inserted at the id's
sort position with `order: 58`. Round 1 used 57. After the round-2 base
merge it takes 58, because 57 is allocated to the in-flight PR objectstack-ai#21244's
fragment. Neither list requires unique orders:
`step18-rationale-merge.test.ts` models two fragments sharing one
`order` and only asserts a positive integer, and `main` already holds
two 56s. So whichever of the two PRs lands first, neither re-orders.
- One D2 conversion, `dataset-count-measure-empty-field-removed`, for
the one lossless sub-shape (a `count` measure's `field: ''`; see Patch
round 2). The D3 entry carries the rest: an expression has no mechanical
rewrite into a column.
  - No `RETIRED_KEYS_BY_MAJOR` row: no key left the shape.
- **Liveness.** The `dataset` ledger rows `dimensions.field` and
`measures.field` stay `live`. Each is re-verified on 2026-10-01, with
the narrowing recorded in its `note`.
- **Guide.** `content/docs/data-modeling/analytics.mdx` gains one "Key
rules" bullet saying that `field` is a column reference.

**Ratchets, as expected for a value narrowing.** The `api-surface`,
`authorable-surface`, `json-schema.manifest`, `export-origins` and
`declaration-map` artifacts are byte-identical. `spec-changes.json` and
the upgrade guide stay at protocol 17, so major-18 entries do not
project yet, and both checks are green.

## The PM's mechanism assumptions, measured

1. **Confirmed.** `dataset.zod.ts:125` (dimension, required) and `:189`
(measure, optional) were bare `z.string()` at the base `d6d6e872`.
`CUBE_MEMBER_SQL` was a module-private `const` at
`analytics.zod.ts:240`.
2. **Exporting `CUBE_MEMBER_SQL` would move the public surface.**
`packages/spec/src/data/index.ts` re-exports the whole module with
`export * from './analytics.zod'`, so an exported `CUBE_MEMBER_SQL`
becomes a new `@objectstack/spec/data` export and needs
`gen:api-surface`. I took the non-public module instead.
`check:api-surface`, `check:export-origins` and `check:declaration-map`
are green with zero changes to their artifacts.
3. **`'*'` on a dimension: measured, and refused.** Readings come from
`POST /api/v1/analytics/dataset/query` with today's spec, through the
real REST route, a real `AnalyticsService` and a real better-sqlite3
`SqlDriver`, using a temporary probe test that was deleted afterwards
(the tree is clean). The ObjectQL-strategy column bridges
`executeAggregate` straight to `SqlDriver.aggregate`, not through the
ObjectQL engine.

   | dataset member `field` | native-SQL strategy | ObjectQL strategy |
   |---|---|---|
| dimension `'*'` | 500 `DATABASE_ERROR` (`SELECT * AS ... GROUP BY *`)
| 500 `DATABASE_ERROR` (`groupBy: ['*']`) |
   | dimension `''` | 500 | 500 |
| count measure `''` | 200 (SQLite accepts the `COUNT()` it compiled to)
| 500 |
   | count measure `'*'` (control) | 200 | 200 |
   | sum measure `'*'` | 500 (`SUM(*)`) | 500 |
   | padded `' amount'` (sum) | 200 | 200 |
| padded dimension `' industry'` | 200 | 200, **dimension column
silently missing from the rows** |
| expression `amount * 2` | 403 `PERMISSION_DENIED` (PR objectstack-ai#21190's door) |
403 |

A `'*'` dimension is never answered: it compiles to grouping by every
column, which is not an axis. So the dataset dimension takes the same
path pattern without the `'*'` arm. That is one pattern source with one
stated restriction, not a second pattern; the pin proves the dimension's
published `pattern` equals the cube's with only the `\*|` arm removed.

⚠ **Flagged, not silently chosen.** The triage line reads "exactly the
`CUBE_MEMBER_SQL` accept set" for both keys. This PR narrows the
dimension one step further, as the dispatch's mechanism item 3 invited
and the card's own pin wording ("`*` (on a measure)") suggests. The cube
`DimensionSchema.sql` still admits `'*'`, per ruling D's execution
parameters, and is untouched here.
4. **Census, repo-wide, with a lit control.** A scan of every `field:`
value in tracked files that mention a dataset and
`dimensions`/`measures`, including `packages/**` tests,
`content/docs/**` and `skills/**`.
- **Lit control.** Column-reference values hit in every area, and the
scanner sees them: `examples` 116, `content` 88, `skills` 15,
`platform-objects` 6, `service-analytics` 587, `spec` 423, `lint` 282,
`rest` 92.
- **Non-column dataset `field`s found:** only the fixtures that PR
objectstack-ai#21190 wrote on purpose to drive its door: `rest`
`analytics-16019-driver-declared-fault.test.ts` (`translate(...)`,
`lower(name)`) and `service-analytics`
`inline-dataset-field-admission-door.test.ts` (an expression constant
and a template). Both were re-pinned (next section).
- **Zero** in the examples, `platform-objects`, the hand-written docs
and the published skills. Every other non-column literal the scan caught
is not a dataset field (driver-sql and protocol prose, filter paths,
`$field` prose in a skill).
- **The build as judge.** The full suites of `spec`, `lint`,
`service-analytics`, `metadata-protocol` and `rest` are green on the
narrowed contract.
- Nothing in `skills/**` teaches an expression `field`, so no Tier H
follow-up is owed.
5. **D2 for one sub-shape, D3 for the rest** (corrected in round 2; the
first round said "D3 only").
- **Expressions: D3 only.** A stored dataset with an expression `field`
already answered 403 at the dataset door. On the REST route it is now
refused one step earlier, at the route's own `DatasetSchema` parse,
which the route runs on the inline and the saved branch alike. An
expression has no mechanical rewrite.
- **Correction.** This item said that no stored row worked before. That
is false by this PR's own probe table: a `count` measure with `field:
''` answered 200 on SQLite's native path, and the door never judged it.
- **The repair.** That sub-shape gets the D2 conversion
`dataset-count-measure-empty-field-removed`, which every stored-row
rehydration seam replays (`applyConversionsToStoredItem`, e.g.
`metadata-protocol`'s `convertStoredItemDetailed`). The runtime door is
still reachable for a dataset handed to `queryDataset` unparsed: the
build probe's dashboard-widget path (`metadata-protocol`
`build-probes.ts`) passes the stored row as read.

## Fixture triage (two consumer tests the narrowing turns red; both
re-pinned, not loosened)

- **`service-analytics` `inline-dataset-field-admission-door.test.ts`**
built its expression fixtures with `DatasetSchema.parse`, which now
refuses them. The fixtures are now built UNPARSED, the shape a
pre-narrowing stored row has, through `storedDatasetWith`. The controls
still parse. One new case asserts that the contract refuses both
fixtures at `dimensions.0.field` / `measures.0.field`. All 4 provider
tiers x 2 strategies of the 403 door pins are unchanged and green.
- **`rest` `analytics-16019-driver-declared-fault.test.ts`.** The route
parses every dataset first, so its inline and saved expression cases now
answer `400 VALIDATION_FAILED`, where they answered `403
PERMISSION_DENIED`.
- Both cases are re-pinned to the `400`, plus `invalid_format` at the
path inside `detail`, the driver never called, and no expression text
echoed.
- The statement-leak check now reads SQL keywords as the strategies emit
them (upper case), because the prescription itself says "Group by the
column itself" in prose. It was case-insensitive before, when the body
carried no prose.
- The docblocks state the new layering and the reverse-verification
direction (measured below).

## Tests

All runs are at head `0d5e446e` (after merging `origin/main` at
`3ddd3d0c`) unless stated otherwise. Filter direction: each package's
own suite, no consumer sweep.

- **`@objectstack/spec`**
  - `vitest run --project local`: 597 files, 17468 passed, 1 todo.
- The new `src/ui/dataset-field-column-reference.test.ts` has 11 cases.
- The cube precedent pin `cube-member-sql-column-reference.test.ts`
stays green, with its `'*'`-on-a-cube-dimension case unchanged.
- **`@objectstack/service-analytics`** `vitest run`: 162 files, 3741
passed, 45 skipped.
- **`@objectstack/lint`** `vitest run`: 119 files, 5502 passed.
- **`@objectstack/metadata-protocol`** `vitest run`: 200 files passed, 3
skipped; 2973 tests passed, 19 skipped.
- **`@objectstack/rest`** `vitest run --project local`: 257 files, 4858
passed, 316 skipped.
- **Typecheck:** `pnpm --filter PKG typecheck` exit 0 for
`@objectstack/spec` (`tsc` + `check:scripts-typecheck` +
`check:test-typecheck`), `@objectstack/service-analytics` (its
`tsconfig` includes all of `src`, so the edited `__tests__` file is in
the program) and `@objectstack/rest` (`tsc` + `check:test-typecheck`).
- **`@objectstack/spec` `--project repo`, the relevant files:**
`step18-rationale-merge`, `conversions-major18-merge`,
`liveness/evidence`, `liveness/proof-registry`,
`retired-key-migrate-sentence`, `file-description`, `root-index`,
`export-list`, `category-title`, `schema-tree-freshness`, `escape-mdx`
and `references-banner`. 12 files, 294 passed.
- **Lint, a declared narrowing.** `eslint --no-inline-config --format
json` over the 8 changed lintable files at `0d5e446e` gave 8 files, 0
errors, 0 warnings.
- **Population:** from `eslint.config.mjs`, the
`**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}` block. The other changed files
are `.md`, `.mdx` and `.json`.
  - **File count:** read from the JSON output.
- **Invariance:** the config never enables type-aware linting (no
`parserOptions.project`, no typed rules), so this diff cannot move a
verdict on an untouched file.
  - The whole-repo `pnpm lint` is CI's.

## Round 2 readings (final head `fc4e91c0`; `origin/main` `3dc33b2d`
merged through `os-regen-merge.sh`)

These readings were taken after a container restart. The restart cut a
first round-2 gate run short at `2e57fa29`. Every reading below was
re-taken at `fc4e91c0`, the pushed head.

- **Build.** `turbo run build` over the closures of spec, cli,
service-automation, metadata-protocol, rest and client-react: 59/59
tasks.
- **`@objectstack/spec`**
- `vitest run --project local`: 597 files, 17465 passed, 1 todo. The
counts moved with `main`'s merge.
- That includes the new
`src/conversions/dataset-count-measure-empty-field-removed.test.ts`, the
table-wide fixture replay in `conversions.test.ts` and
`retired-after.census.test.ts`.
- `--project repo`: the same 12 relevant files as round 1
(`step18-rationale-merge` and `conversions-major18-merge` among them),
294 passed.
- **Consumers that read the conversion table.**
- `@objectstack/cli` `meta.report-order.test.ts` (unit tier): 16 passed.
- `@objectstack/service-automation`
`decision-overlapping-edge-conditions.pin.test.ts`: 22 passed.
- `@objectstack/metadata-protocol` full suite (it hosts the stored-row
seam): 200 files passed and 3 skipped; 2973 tests passed and 19 skipped.
- **Not re-run this round.** `rest`, `service-analytics` and `lint`:
this round's diff does not reach them (spec only), and their round-1
readings stand.
- **Lint, a declared narrowing.** `eslint --no-inline-config --format
json` over the 10 changed lintable files at `fc4e91c0` gave 10 files, 0
errors, 0 warnings. The population and invariance are as in round 1.
- **Gates.** `dispatch-gates --commands` at `fc4e91c0` derived the same
115 families. All were run with exit codes recorded, and `--ran`
reconciled them as "115 derived, 115 run, 0 NOT-MEASURED, 0 UNRUN".
  - `check:generated`: 15/15 up to date.
  - `check:migration-registry`: exit 0.
- `check:adr-0087-registration`: it reads `registered
dataset-member-field-expression-refused,
dataset-count-measure-empty-field-removed`, both new here.
- `check:skill-examples` and `check:dual-build-cjs-loads` both exited 0
this time. Both had been NOT MEASURED in round 1 for want of built
packages.

## Ablations

Each ablation ran from committed state, disk-verified through
`scripts/ablation-replace.mjs`. Each restore was proven by blob hash
equal to HEAD, an empty `git diff HEAD`, and a clean status. The
predicted direction was "turns red" in all four, and that is what was
observed.

| leg | mutation | under mutation | restored |
|---|---|---|---|
| A1 | measure `field` pattern admits anything | new pin: 7 failed / 4
passed (every measure refusal, door, pattern and defineStack case red;
the dimension and accept cases green) | 11 / 11 |
| A2 | dimension takes `ANALYTICS_COLUMN_REFERENCE` (admits `'*'`) | 3
failed / 8 passed (the dimension refusal case on `'*'` and the two
pattern pins) | 11 / 11 |
| A3 | `ANALYTICS_COLUMN_PATH` admits anything, then `@objectstack/spec`
rebuilt | `ablation-dist-preflight`: marker in 18 built files. `rest`:
the 2 re-pinned cases red, `expected 403 to be 400` (the route's parse
passes the expression to the service door, the direction its docblock
predicts); 6 green. `service-analytics` door test: the contract case
red, 20 green | rebuilt; marker absent from all 230 built files; tree
clean |
| A4 (round 2, at `fc4e91c0`) | the new conversion matches nothing (its
`field !== ''` guard reads a value no row carries) | 2 failed / 239
passed: the `conversions.test.ts` fixture pin
`dataset-count-measure-empty-field-removed: before → after, emits 2
notice(s)` and the stored-row pin are red; the controls are green | blob
`75f4166c` == HEAD; `git diff HEAD` empty; status clean |

No ablation file is left in the tree.

## Gates

`node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack
--commands`, run at `0d5e446e` with no paths, derived 115 families.

- All 115 were run, each with its exit code recorded before any pipe.
- `--ran` reconciled them: "115 derived, 114 run, 1 NOT-MEASURED, 0
UNRUN", every row carrying its recorded exit code.
- **NOT MEASURED: `check:dual-build-cjs-loads`.** Reason: it exited 3
(`PREREQUISITE NOT MET`) because 44 packages outside this diff's build
closure have no `dist/`, and only a full monorepo build supplies them.
This diff changes no package entry, export map or build config. CI runs
it on the full build.
- **`check:skill-examples`** first exited 3 for want of a built
`@objectstack/client-react`. After building that package it exited 0 at
`0d5e446e`: 259 examples type-check.
- Every other family exited 0. That includes:
- `check:generated`: all 15 artifacts up to date against a stamp-matched
dist.
- `check:adr-0087-registration`: at the first round's head it read
`registered dataset-member-field-expression-refused (new here)`.
  - `check:changeset-no-major` and `check:empty-changeset`.
- `check:liveness`, `check:migration-registry` and
`check:doc-authoring`.
  - `check:cross-package-test-inputs` and `check:nul-bytes`.

## Acceptance notes

- **File surface, declared.** The claim named `dataset.zod.ts`,
`analytics.zod.ts` "only as far as sharing needs", the ADR-0087 entry
and registry, the retirement kit, pins and one changeset. Four paths go
beyond that, each for the stated reason:
- The new non-public module `data/analytics-column-reference.ts`: the
sharing change itself, which avoids a public export.
- The two consumer test files the narrowing turned red: fixture triage,
re-pinned rather than loosened.
- One bullet in `content/docs/data-modeling/analytics.mdx`: the skill's
docs row.
- **The REST door's answer moves from 403 to 400 for an expression
`field`.** The route's existing `DatasetSchema.parse` refuses it first,
as `VALIDATION_FAILED` naming the path. The changeset says so. The
service door's 403 is unchanged.
- **Studio producer (objectui, outside this repo).**
- **What happens.** `DatasetDefaultInspector.tsx` at the pinned
`31971ff1e` seeds a new dimension row as `{ name: '', field: '', type:
'string' }` and a new measure row as `{ name: '', aggregate: 'sum',
field: '' }`. A plain count measure left with a blank Field box is saved
as `field: ''`. That parsed before. Its query answered 500 on the
ObjectQL path (the SQLite native path happened to accept `COUNT()`).
- **After this PR** a new save of that shape is refused at
`measures.N.field`, with the prescription to omit the key. A row already
stored that way is repaired on load by the D2 conversion
`dataset-count-measure-empty-field-removed`.
- **The producer half** is to omit `field` when the box is blank. It is
reported to the seat, not edited here.
- **Measured, not acted on.** Neither item is filed from here; both are
in the report.
- A `sum` (or any non-count aggregate) over `'*'` parses on a dataset
measure and answers 500 on both strategies. That is the count-only `'*'`
boundary the `analytics_cube` ledger already assigns to objectstack-ai#21000's family.
- A padded dimension `field` answered 200 with the dimension column
missing on the ObjectQL bridge. It is now refused at parse. A stored
padded row would still reach that path through the build probe; no
producer of one is known.

Authored by `session_01UtnxvdiN376GF3sgXwAw4d` (rounds 1 and 2).

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

2 participants