Merge remote-tracking branch 'origin/main' into claude/issue-20312-save-door-sdui-manifest

Author: claude

.changeset/18386-export-import-template.md

@@ -0,0 +1,37 @@
+---
+'@objectstack/rest': minor
+---
+
+feat(rest): `GET /api/v1/data/:object/export?template=true` answers an xlsx import template for the object (#18386)
+
+Clause-②: yes (widening)
+
+The export door takes one more query parameter, `template`. `template=true`
+answers an `.xlsx` workbook with no data rows; `template=false` answers the export.
+Without a `template` parameter the export is exactly as before, byte for byte.
+
+- **Columns.** Every field of the object except
+  those marked `system` or `readonly`, and `formula`, `summary` and
+  `autonumber` fields, in the order the object declares them. A `hidden` field
+  that can be written is a column. The seven columns the platform adds to every
+  object (`organization_id`, `created_at`, `created_by`, `updated_at`,
+  `updated_by`, `owner_id`, `owning_business_unit_id`) are never template
+  columns. A field the caller's field-level security does not let them edit is
+  left out. If the security service cannot say which fields the caller can edit,
+  the columns are narrowed by the fields the caller can read instead. The
+  instructions sheet then says so, and the `X-Export-Template-Projection`
+  response header reads `readable` instead of `writable` (`none` when no
+  field-level security applies). An explicit `?fields=` list is used as sent.
+- **First sheet.** The header row, with ` *` after each field that is required
+  and has no default value, and one example row to replace or delete. Select,
+  radio and boolean columns carry a dropdown.
+- **Second sheet.** One row per column: the field's API name, its type, whether
+  it is required, and the values the import accepts for it.
+- **Language.** The sheets are in Chinese for a `zh` request locale
+  (`?locale=` or `Accept-Language`) and in English otherwise.
+
+The same two permission checks as the export apply: an object that does not
+expose export answers `405`, and a caller without the export permission answers
+`403`. `template` with a value other than `true` or `false`, a `format` other
+than `xlsx`, or any of `limit`, `page`, `filter`, `search`, `searchFields`,
+`orderby` or `header` beside `template=true`, answers `400 VALIDATION_ERROR`.

.changeset/18386-plugin-security-writable-fields.md

@@ -0,0 +1,5 @@
+---
+'@objectstack/plugin-security': minor
+---
+
+The `security` service implements `getWritableFields(object, context)` (#18386). It uses the same permission sets, field grants, `requiredPermissions` check and on-behalf-of delegator intersection as the write gate. A field is in the answer exactly when a write naming it passes the field-level-security check. `getReadableFields` now shares that derivation, and its answers are unchanged.

.changeset/18386-security-service-writable-fields.md

@@ -0,0 +1,11 @@
+---
+'@objectstack/spec': minor
+---
+
+`ISecurityService` (`@objectstack/spec/contracts`) gains an optional `getWritableFields(object, context)`: the field names field-level security lets the caller write on the object, the write-side twin of `getReadableFields` (#18386).
+
+Clause-②: yes (widening)
+
+- It is the exact complement of the fields the write path's field-level-security gate refuses when a payload names them. Neither the object permission nor a field's own rules (`readonly`, `system`, a `formula`, `summary` or `autonumber` type) are part of the answer.
+- It fails soft like `getReadableFields`: `undefined` means no answer, `[]` means no field is writable. A system context gets every field.
+- It is optional. A consumer checks `typeof svc.getWritableFields === 'function'`. When the method is missing, the consumer may narrow by `getReadableFields` instead, and must say in its response that it did.

.changeset/20513-metadata-protocol-runtime-strings-state-the-decision.md

@@ -0,0 +1,30 @@
+---
+'@objectstack/metadata-protocol': patch
+---
+
+metadata-protocol refusals, hints and log lines no longer cite tracker numbers; each states the reason in words
+
+Clause-②: no
+
+Many messages the metadata protocol shows to authors, administrators and operators ended with an
+issue-tracker number where the reason belonged. The number goes, and where the sentence did not
+already say what was decided, it now does:
+
+- Refusals: `insertManyData` without an engine `insertMany` now names what that method is (the
+  partial-success batch insert, so a bad row neither fails the whole batch nor runs the good rows'
+  `beforeInsert` hooks twice); the unknown-metadata-type refusal says a plugin cannot declare a type
+  because `additionalTypes` was retired, having never been read; the stored non-canonical type
+  refusals on publish and revert say the `/meta` URL door now folds a type to its canonical spelling
+  before it writes, so such a row predates that.
+- The schedule-flow `organization_id` hint says why the author's value is the only source: the engine
+  fills only an organization the run resolved, and a schedule resolves none.
+- Log lines: the three `kernel:ready` "migration skipped" warnings now say what the migration that did
+  not run would have ensured; the history-counter abort says the old path took a failed read for an
+  empty table; the publish-closure degrade says the batch's own drafts are left out of the closure;
+  the cold-boot org-scoped audit calls the write refusal it points at declared-types-only. The
+  overlay, `sys_view_definition` and `sys_setting` index messages, the seed/API tenancy repair and its
+  receipt, the batch-row withhold and the object-existence gate's no-registry warning lose only the
+  citation, because their sentences already said it.
+- The live-MySQL testkit's isolation error loses its citation.
+
+Text only: no error code, field name, status or behaviour changes.

.changeset/20596-service-settings-provenance-anchors.md

@@ -0,0 +1,10 @@
+---
+'@objectstack/service-settings': patch
+---
+
+Provenance comments in `service-settings` were re-anchored
+
+Comment and docblock lines under `src/` that cited tracker numbers which no
+longer resolve on GitHub now cite the commit in this repository's history that
+decided the matter, and say in their own words what was decided. Comments
+only: no type, schema, export, log or refusal text, or runtime behaviour changes.

.changeset/20730-meta-packaged-base-dashboard-view.md

@@ -0,0 +1,20 @@
+---
+'@objectstack/rest': patch
+---
+
+fix(rest): an org's published edit to a packaged dashboard or view is what the `/meta` item and list reads serve, in every locale, instead of the packaged translation of the string it replaced
+
+An organization may edit a packaged dashboard or view in place and publish the edit. The metadata protocol's reads returned the edit, and `?layers=true` reported it as effective, but `GET /api/v1/meta/dashboard/:name`, `GET /api/v1/meta/dashboard`, `GET /api/v1/meta/view/:name` and `GET /api/v1/meta/view` served the bundle's translation of the string the package shipped. For example, a widget retitled `Total Users (edited)` on the platform's `system_overview` dashboard was served as `Total Users` to an `en` reader and as `用户总数` to a `zh-CN` reader.
+
+The translators in `@objectstack/spec/system` already let an edited string win over the bundle when they are handed the item as the package shipped it, and the metadata protocol already answers that item (`getPackagedDashboardBase`, `getPackagedViewBase`). The `/meta` reads handed it over for objects only. They now hand it over for dashboards and views too. The change is in the translation step that both `/meta` transports share, the REST server's routes and the runtime's HTTP dispatcher. A view is looked up by its full `<object>.<viewKey>` name.
+
+What a reader sees now:
+
+- An edited string is served as written, in every locale.
+- A widget or view the org left alone is still translated.
+- Resetting the overlay brings back the shipped string and its translation.
+- A dashboard or view with no org edit is served exactly as before.
+
+This fixes what the metadata reads serve, not yet what the console draws. The console built from objectui `db11afd4967c`, this repository's pin when the change was made, looks a dashboard's widget titles and a view's label up in the bundle again in the browser, so it still draws the packaged translation over an edit the server now serves: measured, the `system_overview` board shows `Total Users` / `用户总数` and a `zh-CN` view tab shows `进行中`.
+
+Nothing to migrate: no key, export or route changed.

.changeset/20773-etl-entry-syncconfig-schedule-deleted-in-17.md

@@ -0,0 +1,28 @@
+---
+'@objectstack/spec': patch
+---
+
+fix(spec): two ADR-0087 migration entries state what the tree does — `etl-pipeline-layer-retired` dates the `syncConfig.schedule` deletion to `@objectstack/spec` 17, and `driver-sql-unresolvable-where-column-refused` names the remote `aggregate()` refusals
+
+Clause-②: no
+
+**`etl-pipeline-layer-retired` (protocol 17).** The entry explains that connector-attached
+`syncConfig` has no reader outside `packages/spec`, and cites the same measurement that removed
+`syncConfig.schedule`. Its `replacement` text said that key was retired "in 18". It was deleted
+in `@objectstack/spec` 17 under ADR-0049 (first released in 17.5.0), as the note at the deleted
+position in `integration/connector.zod.ts` already says. The sentence now reads "the same
+measurement that deleted `syncConfig.schedule` in @objectstack/spec 17 under ADR-0049".
+
+**`driver-sql-unresolvable-where-column-refused` (protocol 18).** The entry named only a `where`
+column on `find()` / `findOne()` / `count()` and `INVALID_FILTER` / 400. On the remote face of
+`TursoDriver`, the `aggregate()` door answered `[]` for a missing column or a missing table. It
+now refuses as the local face does: `INVALID_FILTER` / 400 for a `where` column the table lacks, `INVALID_FIELD` / 400 for a
+`groupBy` or aggregation column the table lacks, and `DATABASE_ERROR` / 500 for an object whose
+table is absent. The entry's `surface` now names that door and those codes. Its remedy adds
+grouping and aggregating, and running schema sync so the object's table exists. Its acceptance
+criterion now also covers a report or dashboard that groups by, or aggregates over, a name the
+object has no column for.
+
+Text only: no entry id, conversion or matching logic changes, and `os migrate meta` rewrites
+exactly what it rewrote before. The generated migration registry, `spec-changes.json` and the
+protocol upgrade guide carry the corrected text.

.changeset/20805-formula-write-strip.md

@@ -0,0 +1,24 @@
+---
+'@objectstack/spec': minor
+'@objectstack/objectql': minor
+'@objectstack/service-automation': patch
+---
+
+A caller-supplied value for a `formula` field is stripped on every engine write path, in every context, and reported through `droppedFields` / `onFieldsDropped` under a new `reason`, `computed`; and `ObjectQL.validate` runs the write's own field doors, so a dry run built on it predicts what the write will do (#20805).
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) a change of runtime behaviour at the engine's write doors and its validate-only preview, plus one new arm on an OUTPUT enum. No authorable key, spelling, export or stored shape moves: `DroppedFieldsEvent` is an event the engine emits, never metadata an author writes, so widening its `reason` enum leaves every stored row and every authored file valid as it stands; `ObjectQL.validate` gains an optional listener and keeps its signature otherwise. The narrowing refuses, in the preview, a key the write already refused, so there is nothing for a ledger entry to rewrite: the payload was never writable. The other categories are closed on facts: the packages publish (not `unpublished`); no ADR-0087 id covers a write payload's keys or a strip's report (not `registered` / `already-registered`); and the change is runtime behaviour, not a declaration (not `runtime-interface-only` / `type-surface-only`). -->
+
+**BREAKING**: `ObjectQL.validate` now refuses a row that carries a key the object does not declare, exactly as `insert` and `update` refuse it: the call throws `INVALID_FIELD` / 400 naming the field. It used to answer `valid: true` for that row while the write it previews refused it, so the protocol's `validateData` and the import dry run built on it said "ok" for rows the commit then failed. It ships as `minor` under the launch-window convention; the widening half is the new `reason` arm.
+
+**A formula value is stripped, never refused.** A `formula` field is computed on every read and no driver has a column for it, so a full read returns the key and a record written back carries it: a form save, a flow's `update_record`, a `GET` then `PUT`. The key used to reach the driver, and the driver decided. On SQL drivers the whole write failed with the driver's own error (`SqliteError` "table … has no column named …", with no `status` and no `field`; the REST door relabelled it `400 INVALID_FIELD` "Unknown field" for a field the object declares). On the in-memory driver the value was stored as a shadow column nothing reads. Now the engine takes the value out before the defaults, the hooks and the other strips, completes the write, and reports one `{ reason: 'computed' }` event per call. That holds on `insert` (one row or a batch), `insertMany`, and `update` by id and by predicate, on every driver, and in every context, `isSystem` included: there is no column for any caller's value to land in. Measured on SQLite and the in-memory driver through `protocol.createData`, `POST /api/v1/data/:object`, `PATCH /api/v1/data/:object/:id` and `engine.update`: each now answers success with `droppedFields: [{ fields: ['doubled'], reason: 'computed' }]`, the stored row carries no such key, and the read still returns the computed value.
+
+- **`computed` is not `readonly`.** `isSystem` exempts the static `readonly` strip and does not exempt this one. A `formula` field also declared `readonly: true` is reported once, as `computed`.
+- **`strictReadonlyWrites` refuses it.** That option's coverage is derived from what `onFieldsDropped` reports, so a caller that passes it and sends a formula value now gets `ERR_READONLY_FIELD_REJECTED` with a `computed` drop in `drops`, and nothing is written, `isSystem` included. The refusal message names the reason and its remedy; a refusal without a `computed` drop reads exactly as before.
+- **Hooks are handed the payload that will be stored.** A `beforeInsert` / `beforeUpdate` hook no longer sees the formula key in `ctx.input.data`; `ctx.submitted` on update still carries the caller's submission as sent.
+- **Consumers of `DroppedFieldsEvent['reason']` must handle `computed`.** The contract requires a branch on `reason` to be exhaustive. In this release the strict refusal message (`@objectstack/objectql`) and the flow `create_record` / `update_record` step warning (`@objectstack/service-automation`) word it.
+
+**What `validate` runs now.** Before judging a row, `ObjectQL.validate` runs the write's own doors, by the same functions the write calls: the declared-field door (the refusal above), the computed-field strip, and the caller-write strips, under the write's `isSystem` gate (on `insert` mode the runtime-owned strip and the static `readonly` strip with its re-default; on `update` mode the static `readonly` strip, where a supplied `id` is the address the write binds and is never judged). What the write would drop is reported through a new optional `onFieldsDropped` listener on `validate`'s options, in the same events the write emits. One consequence for verdicts: a reference field declared static `readonly` is now stripped in the preview as it is on the write, so a validation rule that reads through it answers the same on both.
+
+**Unchanged.** A `summary` field keeps its column: a caller-supplied roll-up value is still stored as sent and overwritten by the next write of a child record. The REST layer's own handling of a missing column (schema drift) is unchanged.

.changeset/report-chart-matrix-placement-comment.md

@@ -0,0 +1,17 @@
+---
+'@objectstack/spec': patch
+---
+
+fix(spec): the `ReportSchema.chart` doc comment says the chart is drawn below the table of a `matrix` report with `columns`
+
+Clause-②: no
+
+The doc comment on `ReportSchema.chart` said the embedded chart is plotted above the report's
+table. That holds for a `tabular` or `summary` report, and for a `matrix` report without
+`columns`, which renders as a grouped table. A `matrix` report with `columns` renders as a
+cross-tab, and objectui's `DatasetReportRenderer` draws the chart below it. The comment now
+says so. It also drops a clause saying a chart on a `joined` report "parsed and plotted
+nothing": the schema refuses that key today, so the clause no longer described it.
+
+Doc comment only: the schema accepts and refuses the same reports, and no `.describe()` text
+or export changes.

content/docs/kernel/contracts/data-engine.mdx

@@ -315,6 +315,7 @@ The strips these two options cover are the engine's legal ones:
 | A TRUE `readonlyWhen` predicate (#3042) | `readonly_when` | `update` | none at the API boundary — every caller, `isSystem` included; a value a `beforeUpdate` hook derived or overwrote is not a caller write and is never stripped (#9107) |
 | Implicitly-readonly runtime-owned type (#5503 — `RUNTIME_OWNED_FIELD_TYPES`, today `autonumber`) | `readonly` | `insert` **and** `update` | `isSystem`, `preserveAudit` (#3493) |
 | Primary-key strip of a payload `id` the update dispatch already ruled is not an identifier (#6437) | `primary_key` | `update` | none |
+| A value for a `formula` field, which the engine computes on read and no driver has a column for (#20805) | `computed` | `insert` **and** `update` | none — every context, `isSystem` included |
 
 Of the two AUTHOR-DECLARED strips only `readonlyWhen` is insert-exempt at this seam
 (a conditional lock has no prior record on a create); the static `readonly` strip runs
@@ -332,7 +333,7 @@ interface DroppedFieldsEvent {
   fields: string[]; // caller-supplied fields that were dropped
   // why they were dropped — an OPEN vocabulary that grows with the write
   // path's legal strips; branch on it exhaustively, never with a binary test
-  reason: 'readonly' | 'readonly_when' | 'primary_key';
+  reason: 'readonly' | 'readonly_when' | 'primary_key' | 'computed';
 }
 ```