Merge remote-tracking branch 'origin/main' into claude/issue-20987-analytics-contains-membership

Author: claude

.changeset/19518-picklist-kind.md

@@ -11,7 +11,7 @@ Clause-②: yes (widening)
 
 - **The kind.** `PicklistSchema` — `{ name, label, description?, options }`, where `options` is the field option shape (`SelectOptionSchema`) reused as is. Authored in a package as `*.picklist.ts` (`definePicklist`) or `defineStack({ picklists })`. It is a registered kind (`MetadataTypeSchema`, `DEFAULT_METADATA_TYPE_REGISTRY`, `getMetadataTypeSchema('picklist')`) that loads before `object`. It is package-owned, so a runtime create or a per-organization overlay is refused.
 - **The reference.** `Field.select({ picklist: 'industry' })` adds a `picklist` key to `FieldSchema`. It is valid on the option types only (select, radio, multiselect, checkboxes, tags). A field that declares both `picklist` and `options` is refused at `options`, with a prescription. The functional-completeness predicate counts a `picklist` reference as the field's option source.
-- **The served shape.** `PicklistServedFieldSchema` declares what a client reads for a picklist-bound field: the resolved `options` next to the `picklist` that names the list. This release does not resolve the reference. Until the runtime does, a picklist-bound field is served without options, and the liveness ledger grades the key `planned` and warns an author who writes it.
+- **The served shape.** `PicklistServedFieldSchema` declares what a client reads for a picklist-bound field: the resolved `options` next to the `picklist` that names the list. The runtime resolves the reference onto that served field; see the picklist runtime entry of this release.
 - **Extensions.** `defineStack({ picklistExtensions: [{ extend, options }] })` adds options to a picklist that another package owns. It can only add; removing or renaming a value stays with the owning package.
 - **Translation.** `TranslationData` gains `picklists.<name>.{ label?, options: { value: label } }`. `translatePicklist` translates a served picklist item. `translateObject` gives a picklist-bound field the list's option labels, and a field-level `options` entry still wins over them.
 - **Studio type label.** `@objectstack/platform-objects` carries the `picklist` type's label and description in its metadata-forms translation bundles (en, zh-CN, ja-JP, es-ES).

.changeset/19519-picklist-runtime.md

@@ -0,0 +1,17 @@
+---
+'@objectstack/objectql': minor
+'@objectstack/metadata': patch
+'@objectstack/spec': patch
+---
+
+feat(objectql): the runtime resolves a field's `picklist` onto its served options, validates writes against the resolved list, and merges `picklistExtensions` additively
+
+Clause-②: no
+
+- **Load.** `defineStack({ picklists })` and `defineStack({ picklistExtensions })` now register, from a manifest and from a nested plugin, through the same registration seam as every other collection. The compiled-artifact door registers `picklists` as `picklist` items, so `GET /meta/picklist` serves them on an artifact boot.
+- **Merge.** A picklist's options are its own, followed by the options every `picklistExtensions` entry adds. A value the list already carries is refused with `422 INVALID_METADATA`, which names both declarations, whichever of the two registered first. The later declaration never replaces the earlier one. A package that registers again replaces its own extension. Uninstalling a package removes the values it added.
+- **Serve.** A field with `picklist: 'NAME'` is served with the resolved options written onto it and `picklist` kept (`PicklistServedFieldSchema`), on every object read, including objects stored in `sys_metadata`. The list's translations (`picklists.NAME.options.VALUE`) relabel those options per request locale. An option marked `default: true` in the list fills an omitted field on insert, as an inline option does, and the import template reads it the same way.
+- **Unknown name.** A packaged field that names a picklist no loaded package declares fails the boot at `kernel:ready` with `INVALID_METADATA`, and so does a `picklistExtensions` entry that extends such a list. The error names every such field or extension and the package that declared it. After boot, an artifact registered through the `manifest` service is checked before any of it registers. A field whose list does not resolve is served with no options and accepts no value.
+- **Write validation.** The write door judges a picklist-bound field against the resolved options, and its refusal names the picklist. The wire code stays `invalid_option`. The validation message catalog gains three message keys for this (`invalid_option_picklist`, `invalid_option_value_picklist`, `invalid_option_picklist_unresolved`) in en, zh-CN, ja-JP and es-ES. They change the message text only, never the wire.
+- **Writing the served body back.** The served body carries `picklist` and `options` together. Writing it back through the metadata door is still refused, with the prescription to drop `options`, as `FieldSchema` declares. Nothing strips it on the write side.
+- **Ledger.** `field.picklist`, the `picklist` kind's rows and `translation.picklists` are `live`. `field.picklist` no longer carries `authorWarn`, so `os lint` / `os validate` stop warning an author who writes it.

.changeset/20821-plan-deferred-ddl-reads.md

@@ -0,0 +1,29 @@
+---
+'@objectstack/driver-sql': patch
+---
+
+fix(driver-sql): `os migrate plan` on a database that does not exist yet no longer prints `DATABASE_ERROR` for the tables whose DDL it deferred (#20821)
+
+`os migrate plan` (and the boot of `os migrate apply`) runs with the SQL driver's DDL deferred: the driver records every table as pending `create_table` and creates none of them. The same boot then reads `sys_metadata`, `sys_metadata_activation` and `sys_migration`. On a new database those tables do not exist yet, so each read was refused, and each refusal printed a line like this on the driver's warn channel (stderr by default):
+
+```text
+[sql-driver] DATABASE_ERROR — the backend refused a read on 'sys_metadata' (SQLITE_ERROR) ... no such table: sys_metadata
+```
+
+Nothing was wrong: every reader already answers from the refusal, and the plan lists the same tables as pending creates. A dry run on a new database printed six of these lines.
+
+The driver now sends such a refusal to the logger's `debug` channel instead of `warn`, when all three hold:
+
+- this driver has DDL deferred;
+- the refused statement targets a table whose DDL this driver deferred;
+- the shared `isMissingTableError` predicate from `@objectstack/types` recognises the refusal as that table being missing.
+
+The default logger has no `debug`, so the line is not printed. The refusal is still thrown to the caller with the same envelope (`DATABASE_ERROR`, status 500), and the plan's output is unchanged.
+
+What still warns:
+
+- every other refusal on a deferred driver, such as a malformed statement on a table that exists;
+- a missing table that the driver did not defer, such as a table nothing in the boot declares;
+- every refusal once the deferred DDL has been applied, or on a driver that never deferred any.
+
+There is nothing to migrate.

.changeset/20890-dataset-dimension-json-stored-refused.md

@@ -0,0 +1,17 @@
+---
+"@objectstack/lint": minor
+---
+
+fix(lint)!: `os validate`, `os build` and `os lint` refuse a dataset dimension over a JSON-stored field, the group key the analytics door already refuses at query time
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) a refusal of a grouping TARGET at authoring time: a dataset dimensions entry whose field resolves to a declared structured-JSON field (json, composite, repeater, record, location, address, vector) or multi-value field (multiselect, checkboxes, tags, or a select, radio, lookup, user, file or image declared multiple: true). It is the authoring leg of the analytics door that already refuses every query grouping by such a column with 400 INVALID_FIELD, and that door's own changeset declared this category for this surface. No authorable key, spelling, export or stored shape moves: DatasetSchema keeps parsing every dimension, no stored row is read or rewritten, and which scalar part of a document, or which member of a list, an author meant to group on is not something a ledger entry can rewrite. The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id covers a grouping target, and dataset-measure-aggregate-field-type-refused says in its own reason that a field used as a DIMENSION is untouched (not already-registered); and the change is a rule verdict, not a declaration (not runtime-interface-only or type-surface-only). -->
+
+**BREAKING**: metadata that passed `os validate`, `os build` and `os lint` can now fail, and so can a runtime dataset save (Studio, REST `/meta`, MCP), which runs the same rule. A dataset dimension is a group key, and the analytics door refuses a query that groups by a JSON-stored column with `400 INVALID_FIELD` before any SQL is built, so such a dimension could be declared but never served. The new rule `dimension-json-stored-field-refused` (gating, `error`) refuses it where the author writes it. It ships as `minor` under the launch-window convention for accept-set narrowings. No export is removed. The package entry exports the new id as `DIMENSION_JSON_STORED_FIELD_REFUSED`, beside `MEASURE_AGGREGATE_FIELD_TYPE_REFUSED`, and the `rule` member of `DatasetMeasureAggregateFinding` gains it.
+
+**What is refused.** A dimension whose `field` resolves, on the dataset's object or across its join chain, to a field declared with a structured-JSON type (`json`, `composite`, `repeater`, `record`, `location`, `address`, `vector`) or a multi-value declaration (`multiselect`, `checkboxes`, `tags`, or a `select`, `radio`, `lookup`, `user`, `file` or `image` declared `multiple: true`). The two classes are `@objectstack/spec/data`'s `STRUCTURED_JSON_TYPES` and `isMultiValueField`, the predicates the analytics door reads.
+
+**What an author sees now.** The finding names the dataset, the dimension, the field, the object that declares it and its declaration, and says the analytics door refuses every query that groups by it. It names the route: group by a field that stores one scalar value, storing the part of the document you group on in a field of its own; for a multi-value field, filter by one member with `$contains` in a record query, one query per member. It is located at `datasets[N].dimensions[M].field`, name-keyed on the runtime wire.
+
+**Unchanged.** A dimension over any other field, a single-value `select` or `lookup` included; a dimension whose field does not resolve (`dataset-field-unknown` reports that) or declares no type; a dataset over an object this stack does not define; measures, filters and every other position. A cube dimension (`analyticsCubes`) is not judged: no authoring rule reads cubes.

.changeset/20890-dataset-distinct-multiple-refused.md

@@ -0,0 +1,17 @@
+---
+"@objectstack/lint": minor
+---
+
+fix(lint)!: a dataset `count_distinct` measure over a field declared `multiple: true` is refused by `measure-aggregate-field-type-refused`, as the compile leg and the engine already refuse it
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: not-required (already-registered dataset-measure-aggregate-field-type-refused) the registered entry carries this family's hand-migration, an aggregate the field accepts with count as the one that stays for a JSON-stored field, and the count_distinct narrowing over the JSON-stored types already rides it. A select, radio, lookup, user, file or image field declared multiple: true is the one JSON-stored shape the per-type table cannot see; the dataset compile leg and the engine's count_distinct door refuse it beside the table's row, and this is the authoring leg of that same pair, so it adds no surface of its own. -->
+
+**BREAKING**: metadata that passed `os validate`, `os build` and `os lint` can now fail, and so can a runtime dataset save, which runs the same rule. `measure-aggregate-field-type-refused` reads the field's declaration, not its type alone: `count_distinct` over a `select`, `radio`, `lookup`, `user`, `file` or `image` field declared `multiple: true` is refused, because that field is a list stored as JSON and no two backends compare such values for equality alike. The dataset compile leg already answers the pair `400 DATASET_INVALID`, and the engine's `count_distinct` door answers it `400 INVALID_FIELD`. It ships as `minor` under the launch-window convention for accept-set narrowings.
+
+**What an author sees now.** The finding names the measure, the field, the object and the declaration with its flag (`select` with `multiple: true`), and says the aggregate accepts its row's types, none of them with `multiple: true`. The hint names the aggregates the declaration does accept, read from the same predicate: `count`.
+
+**What to write instead.** `count` over the field, or `count_distinct` over a field that stores one scalar value. To count the records holding one member, filter by it with `$contains` in a record query.
+
+**Unchanged.** `count_distinct` over the same types without the flag; `count` over any field; every other aggregate, whose rows accept no multi-capable type and whose verdicts therefore do not move; and the skips the rule already had.

.changeset/20892-package-visibility-default-org.md

@@ -0,0 +1,22 @@
+---
+'@objectstack/spec': minor
+---
+
+feat(spec): `PackageSchema.visibility` defaults to `org` (was `private`), the create-time default every publish path already produced
+
+Clause-②: yes
+
+`PackageSchema` (`@objectstack/spec/marketplace`) filled an omitted `visibility` with
+`private`, but no path that creates a package ever reached that value: the cloud control
+plane gives a new package `org` when the create request omits the key, and
+`os package publish` used to send `org` itself. The declared default now matches what the
+runtime does: `org`, which makes a package published from one environment installable in
+the owner organization's other environments.
+
+- **What changes:** `PackageSchema.parse(row)` on a row with no `visibility` now returns
+  `visibility: 'org'`. A row that names `private`, `org` or `marketplace` is read exactly
+  as before, and any other value is still refused.
+- **What does not change:** the accepted values, and `CreatePackageRequestSchema.visibility`,
+  which stays optional with no default. A create request that omits the key reaches the
+  control plane without it, and the control plane's default applies.
+- **If you relied on the old default:** pass `visibility: 'private'` explicitly.

.changeset/20892-plugin-publish-visibility-when-asked.md

@@ -0,0 +1,25 @@
+---
+'@objectstack/cli': patch
+---
+
+fix(cli): `os plugin publish` sends `visibility` only when `--visibility` is passed, so re-publishing no longer moves a `marketplace` plugin to `private`
+
+The `--visibility` flag of `os plugin publish` defaulted to `private`, and the CLI always
+sent it in the package upsert (`POST /api/v1/cloud/packages`). That upsert is also the
+re-publish path, and the control plane updates an existing package's visibility whenever
+the request carries one. So re-publishing a new version of a `marketplace` plugin without
+repeating `--visibility marketplace` silently set it to `private`. `os package publish`
+already works this way; both commands now behave the same.
+
+The flag no longer has a default, and an omitted flag is an omitted key:
+
+- **Re-publish without the flag:** the package keeps its current visibility.
+- **First publish without the flag:** the control plane applies its own default (`org` on
+  ObjectStack Cloud), the default `PackageSchema.visibility` declares. Before this change
+  `os plugin publish` sent `private` here; pass `--visibility private` to keep that.
+- **With the flag:** unchanged. `--visibility private|org|marketplace` is sent and applied.
+
+The publish summary gains a `Visibility` line showing the control plane's answer. When the
+control plane does not report it and no flag was given, the line says
+`not reported by the control plane`. The "Re-run with --submit" hint now follows that
+answer too, so it also fires when a `marketplace` plugin is re-published without the flag.

.changeset/20901-step18-rationale-form-view-subform-columns.md

@@ -0,0 +1,10 @@
+---
+'@objectstack/spec': patch
+---
+
+fix(spec): protocol 18's migration rationale now covers the form view's inline grid columns and the identity-only currency `scale` refusal (#20901)
+
+**`@objectstack/spec`**
+
+- **`MIGRATIONS_BY_MAJOR[18].rationale` gains one fragment, `form-view-subform-columns-closed`.** It is the paragraph `os migrate meta --step` shows for the protocol 17 → 18 hop. The new sentences say that a form view's `subforms[].columns` now takes the strict `InlineGridColumnSchema` a relationship field's `inlineColumns` takes, that the conversion `form-view-subform-columns-canonicalized` respells a `{ field }` column as `{ name }` in stored rows and assembled artifacts while an author writing `field` is refused, and that `defineStack` refuses `scale` on a column that declares no `type` when its `name` is a `currency` field of a child object declared in the same stack (`inline-grid-column-identity-only-currency-scale-refused`).
+- Text only: no schema, conversion or migration entry changes, and `conversionIds` and `semantic` for step 18 are unchanged.