Merge remote-tracking branch 'origin/main' into claude/issue-20946-by-name-flow-read

Author: claude

.changeset/20701-import-row-not-null-and-unique-door-answer.md

@@ -0,0 +1,27 @@
+---
+'@objectstack/rest': patch
+---
+
+fix(rest): an import row for a NOT NULL refusal or a unique conflict answers what the create door answers (#20701)
+
+**`POST /api/v1/data/:object/import` and the async import job — a NOT NULL
+refusal.** When the database refuses a row because a NOT NULL column has no
+value (for example a field declared `storage: { notNull: true }` and not
+`required`, which the engine's own check lets through), the committed row
+now fails with `code: 'required'`, `field` set to the field, and the sentence
+`POST /api/v1/data/:object` gives for it ("f is required"). It used to fail with
+the database's own code (`SQLITE_CONSTRAINT_NOTNULL` on SQLite) and no `field`.
+The create door answers `400 VALIDATION_FAILED` with a `required` finding for
+the field; the row reports that finding the way it reports a `required` field
+the engine refuses itself, so no database dialect's code reaches the row.
+
+**A unique conflict.** A committed row that repeats a unique value keeps
+`code: 'UNIQUE_VIOLATION'` and its `field`, and now carries the create door's
+sentence, "A record with this f already exists", in place of the engine's longer
+sentence (which the create door returns as `developerMessage`).
+
+The async job's rows, read from `GET /api/v1/data/import/jobs/:jobId/results`,
+change the same way. The row takes these answers from the same mapper as the
+create door, as it already does for a missing database column. No key is added
+to the row. The dry run still previews such rows as `ok`, because it checks the
+metadata and does not judge `storage.notNull` or uniqueness.

.changeset/20701-import-row-schema-drift-door-answer.md

@@ -16,7 +16,6 @@ fail with the database's own code and text (for example `SQLITE_ERROR` and
 `GET /api/v1/data/import/jobs/:jobId/results`, change the same way.
 
 The row now classifies a write error through the same mapper as the create door,
-and takes that answer when it is `INVALID_FIELD`. Rows for a unique conflict or a
-NOT NULL failure are unchanged. The dry run still cannot see a missing column,
+and takes that answer when it is `INVALID_FIELD`. The dry run still cannot see a missing column,
 because it checks the metadata and not the table, so it previews such a row as
 `ok`.

.changeset/20863-orphan-package-binding-refused.md

@@ -0,0 +1,18 @@
+---
+'@objectstack/metadata-protocol': minor
+---
+
+fix(metadata-protocol)!: a flow saved through the metadata door naming, as its base, a package this deployment has not installed is refused, instead of being stored bound to a package that does not exist (#20863)
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) No metadata changes shape and nothing an author wrote is renamed or removed, so `objectstack migrate meta` has nothing to rewrite. What moves is which flow saves the metadata door accepts: a save that names a package no installed package holds is now refused. -->
+
+**BREAKING**: shipped as `minor` under the launch-window convention. `PUT /api/v1/meta/flow/:name` answered `200` and stored the flow live when the save named, as the package the flow belongs to, a package id this deployment has never installed. It now answers `422 WRITABLE_PACKAGE_REQUIRED`, and nothing is written, served or registered.
+
+**What changed.** The one authoring rule every flow write door asks (`tenantAuthoredWriteRefusal`) now also judges the base a save names. After the locked-base refusal and before the provenance check, a named base must be a package the registry holds as installed: a code package, an installed package, or a tenant's own writable base created through the package door. That is the same registry read the metadata write path already resolves a base against, so no second list of packages is kept. The refusal does not depend on what the definition carries: a save with no provenance of its own and a save whose provenance names that same missing package were both stored before, and both are refused now. It applies on a single-kernel host and on an environment kernel alike.
+
+- The code is `WRITABLE_PACKAGE_REQUIRED` / `422`, the one ADR-0070 D1 already uses for a runtime create whose base is missing or read-only. The refusal names the package id the save sent.
+- **Unchanged:** a flow saved without naming a package; a flow saved into an installed package; the locked-base refusal of a shipped flow, which still answers first; every other metadata type, whose saves keep their old handling; the `/automation` create, update and clone doors, which name no package; and the server-stated rewrites of stored rows (the stored-metadata migration and package duplication), which the rule does not judge.
+
+**What to send instead.** Save the flow into a package this deployment has installed (the package list shows them, and a base that does not exist yet is created first through the package door), or save the flow without naming a package.

.changeset/20901-form-view-subform-columns-canonicalized.md

@@ -0,0 +1,11 @@
+---
+'@objectstack/spec': patch
+---
+
+fix(spec): a stored form view whose subform grid columns use the `field` spelling is respelled to `name` on the way in, as a relationship field's `inlineColumns` already are (#20901)
+
+**`@objectstack/spec`**
+
+- **New ADR-0087 conversion `form-view-subform-columns-canonicalized` (protocol 18, retired from the authoring path).** A form view's `subforms[].columns` accepted any value through 17.5.0 and now takes the inline grid column contract, which refuses `{ field: 'x' }` with the prescription naming `name`. The conversion rewrites that entry as `{ name: 'x' }`, every other key kept, wherever a form view travels as data at rest: a stored `view` row (its `form`, each `formViews` entry, a form view item's `config`, a flattened form overlay), an assembled manifest's `viewItems`, and `os migrate meta --from 17`, which lists the edit. A built artifact whose declared protocol floor is 17.5.0 or lower is converted too, not refused. An entry that already carries `name` is left alone, including one that carries both `field` and `name`: the parse names both keys, and the author picks one. It is the same respelling `field-column-lists-canonicalized` applies to a relationship field's `inlineColumns`, and both entries run one shared rule.
+- **Authored sources are unchanged:** `defineStack` and `objectstack validate` do not replay a retired conversion, so a source that writes `field` on a subform column is still refused with the prescription. Write `name`.
+- **Two step-18 migration entries now read true.** `inline-grid-column-currency-scale-refused` no longer says a column declaring no `type` keeps its `scale`: over a `currency` field of the child object, `defineStack` refuses it, under `inline-grid-column-identity-only-currency-scale-refused`, which the entry now names. `form-view-subform-columns-closed` names this conversion as the one mechanical edit on its carrier.

.changeset/20932-predicate-guard-comparand.md

@@ -0,0 +1,17 @@
+---
+'@objectstack/plugin-security': minor
+---
+
+fix(plugin-security)!: a field the caller may not read is refused as a cross-field comparand exactly as it is refused as a filter key (#20932)
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) a refusal of a QUERY at the security layer's field predicate guard: a query that names a field the caller's field-level permissions hide as a cross-field comparand now answers the 403 PERMISSION_DENIED the same field already answers as a filter key. No authorable key, spelling, export or stored metadata shape moves: FieldReferenceSchema, every filter shape and every object definition parse as before, and @objectstack/plugin-security exports nothing new and nothing less (collectConditionFields, collectQueryFields and assertReadableQueryFields keep their signatures). There is nothing for objectstack migrate meta to rewrite, since what changes is which caller may run a query, not what any metadata says. The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id covers a permission verdict and this diff adds none (not registered / already-registered); and the change is runtime behaviour, not a declaration (not runtime-interface-only / type-surface-only). -->
+
+**BREAKING for queries that compare a column against a field the caller may not read.**
+
+**What changed.** The security layer refuses a query that filters, sorts, groups or aggregates by a field the caller's field-level permissions hide, with `403 PERMISSION_DENIED`: which rows answer would disclose the value that the field mask withholds from the result. A cross-field comparand (`FieldReferenceSchema`, "compare against another column of the same row") reads the field it names in the same way, and it is now collected into the same set and judged by the same rule. A hidden field named as a comparand, in any position the filter grammar admits for one, in `where`, `having` or a per-aggregation `filter`, answers the same `403 PERMISSION_DENIED`, in the same words, as the same field written as a filter key. This covers `engine.find`, `findOne`, `count`, `aggregate` and the bulk `update` / `delete` predicate, and every route that reaches them. Before, such a comparison was answered.
+
+**What is not affected.** A comparand naming a field the caller may read answers as before. A system context, and a caller with no permission sets, are unaffected. Row-level policies may still compare against fields the caller cannot read, because they are applied after the guard. A comparand the filter grammar refuses is still refused; when it names a hidden field, that refusal may now be the `403` rather than `400 INVALID_FILTER`, as it already was for a hidden filter key.
+
+**If a query stopped answering for some users,** it compares against a field those users may not read. Grant that field's read permission to the users who need it, or compare against a field they can read.

.changeset/20933-analytics-relationship-path-admission.md

@@ -0,0 +1,53 @@
+---
+'@objectstack/service-analytics': minor
+---
+
+fix(service-analytics)!: an object an analytics query reads through a relationship path is admitted and row-scoped exactly as a declared join to it is, on both strategies (#20933)
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) No authorable key, export or stored shape is removed or renamed. The change refuses, or row-scopes, what the native-SQL strategy read from an object reached through an undeclared relationship path, the way a declared join to the same object already was; there is nothing for `objectstack migrate meta` to rewrite. -->
+
+**BREAKING for analytics queries on a SQL deployment that read a related object through a relationship path the cube does not declare.**
+
+**What changed.** The analytics door admits and row-scopes one object set
+before either strategy runs. It held the cube's base object and the joins the
+cube declares (`joins`, or a dataset's `include`). An object reached through a
+relationship path the cube does not declare was not in it, although both
+strategies read that object: a dotted member of an inferred cube, an authored
+member whose `sql` walks a relationship the cube's `joins` does not list, or a
+dotted member the query names itself. Every such object is now in the set, so
+`POST /api/v1/analytics/query`, `POST /api/v1/analytics/sql` and
+`POST /api/v1/analytics/dataset/query` treat it exactly as a declared join:
+
+- a related object the caller may not read answers `403 PERMISSION_DENIED`,
+  naming that object, before any statement runs;
+- the caller's row scope on the related object is applied, so related rows
+  outside it are not read. On the native-SQL strategy a base row whose related
+  record is outside the scope drops out of the answer, as it already did for a
+  declared join; the ObjectQL strategy still groups such rows as restricted;
+- a related-object scope the native-SQL strategy cannot compile routes the
+  query to the ObjectQL strategy, as it already did for a declared join.
+
+Each hop of a multi-hop path is judged on its own object, resolved the way the
+field-level gate resolves it: the join the cube keys by the path, or else the
+relationship name itself.
+
+**What is not affected.** A query through a related object the caller may read
+answers as before, within the caller's row scope. A system context, and a
+caller with no permission sets, are unaffected, as on the data API. A
+deployment with no security service applies no object-level check, as on the
+data API.
+
+**Refusals that change form.** On the ObjectQL strategy a related object the
+caller may not read was already refused; it now answers the analytics door's
+refusal rather than the engine's, the same one a declared join gets. A filter,
+a time window or a two-hop path through such an object moves from
+`400 INVALID_FIELD` to that `403`. A relationship path whose relationship name
+is not itself an object name was never served by either strategy; for a caller
+the object-level check applies to, it now answers `403 PERMISSION_DENIED`
+naming that relationship.
+
+**If a widget stopped answering for some users,** it reads a related object
+those users may not read. Grant read access on that object to the users who
+need it, or build the widget on objects they can read.

.changeset/20935-analytics-masked-field-not-queryable.md

@@ -0,0 +1,37 @@
+---
+'@objectstack/service-analytics': minor
+---
+
+fix(service-analytics)!: a field the caller is served masked is refused as a group key, an aggregate input, a filter or a sort key on every analytics face, whichever strategy serves the cube (#20935)
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) No authorable key, export or stored shape is removed or renamed. The change refuses analytics queries that group, aggregate, filter or sort by a field the caller may only see masked, which the engine already refuses on the data API and on the ObjectQL strategy, so there is nothing for `objectstack migrate meta` to rewrite. The one public-surface addition is a new optional service hook. -->
+
+**BREAKING for analytics queries on a SQL deployment that group, aggregate, filter or sort by a field the caller may only see masked.**
+
+**What changed.** The field-level gate on `POST /api/v1/analytics/query`,
+`POST /api/v1/analytics/sql` and `POST /api/v1/analytics/dataset/query` judged
+each member by the caller's readable fields. A field whose `maskingRule` applies
+to the caller is readable (its values are served masked), so the gate admitted
+it, and the native-SQL strategy then grouped or filtered by the stored value.
+The gate now also asks which fields the caller may query on, and refuses a
+member naming a masked field with `403 PERMISSION_DENIED`, in the words the
+engine uses for the same field. The ObjectQL strategy and the data API already
+refused these queries.
+
+**What is not affected.** A caller who holds the capability that lifts a
+field's masking rule queries the field as before. A system context is
+unaffected. A query that names no masked field answers as before.
+
+**New hook.** `AnalyticsServiceConfig.getQueryableFields(object, context)`
+supplies the answer. `AnalyticsServicePlugin` wires it to the `security`
+service's `getQueryableFields`. When that service predates the method, or
+answers "no answer", the plugin treats every field that declares a
+`maskingRule` as not queryable, for every caller. A host that
+constructs `AnalyticsService` itself with `getReadableFields` and without
+`getQueryableFields` is warned once at construction.
+
+**If a widget stopped answering for some users,** it groups or filters by a
+field those users see masked. Give the users who need it the capability the
+field's `requiredPermissions` names, or build the widget on fields they can query.

.changeset/20935-plugin-security-queryable-fields.md

@@ -0,0 +1,5 @@
+---
+'@objectstack/plugin-security': minor
+---
+
+The `security` service implements `getQueryableFields(object, context)` (#20935). A field is in the answer exactly when a query naming it as a filter, a sort key, a group key or an aggregate input passes the engine's field guards: the answer is read from the one field map the predicate guard and the aggregate-input guard now share (permission sets, field grants, the `requiredPermissions` check, the on-behalf-of delegator intersection, and every field whose masking rule applies to the caller). The two guards refuse exactly what they refused before.

.changeset/20935-security-service-queryable-fields.md

@@ -0,0 +1,11 @@
+---
+'@objectstack/spec': minor
+---
+
+`ISecurityService` (`@objectstack/spec/contracts`) gains an optional `getQueryableFields(object, context)`: the field names field-level security lets the caller filter, sort, group or aggregate by on the object, the query-side twin of `getReadableFields` (#20935).
+
+Clause-②: yes (widening)
+
+- It is the exact complement of the fields the engine's field guards refuse when a query names them as a filter, a sort key, a group key or an aggregate input. It is a subset of `getReadableFields`, and the two differ by exactly the fields the caller is served masked: a field whose `maskingRule` applies to the caller is readable (served, its value replaced) and not queryable.
+- It fails soft like `getReadableFields`: `undefined` means no answer, `[]` means no field is queryable. A system context gets every field.
+- It is optional. A consumer checks `typeof svc.getQueryableFields === 'function'`. When the method is missing, or answers `undefined`, the consumer must treat every field that declares a `maskingRule` as not queryable, whoever the caller is. Falling back to `getReadableFields` alone would admit exactly the masked fields.

docs/NORTH-STAR.md

@@ -40,6 +40,7 @@
 - records · 写入规则在服务端兑现,撞车响亮、后悔得了、改过什么看得见 · records-forms · validation-rule-type-matrix, object-hook-lifecycle, delete-behavior-matrix, concurrent-edit-conflict, record-edit-undo, field-history-tracking
 - records · 记录上的动作与协作:按钮在声明的位置、参数契约在派发时兑现、讨论与 @ 提醒 · records-forms · action-location-matrix, action-param-widgets, upload-guard-blocks-confirm, record-discussion-mentions
 - records · 搜得到:跨字段、字段限定、权限一致、拼音、即时新鲜;全局搜索与命令面板同一条路 · search · cross-field-object-search, field-scoped-narrowing, rls-both-personas, pinyin-flag-both-sides, freshness-and-empty, console-global-search, command-palette-navigation
+- records · 共享选项集:多个对象复用同一份选项,客户按组织追加值而不改应用源码 · records-forms · picklist-shared-across-objects, picklist-org-append
 - access · 行级与字段级权限两边都对:受限成员只看自己的,该只读的只读、该看不见的不回给前端 · access-security · rls-both-sides, scope-depth-asymmetry, fls-mask-and-strip
 - access · 增删改查逐格兑现,改完权限立刻换脸,自查接口与服务端一致 · access-security · crud-permission-matrix, permission-matrix-edit-loop, me-permissions-aggregation-parity
 - access · 写入路径挡得住:只读剥离、伪造与转移 owner 被拒、默认可见度只能收紧 · access-security · write-path-guards, owd-save-gate