Merge remote-tracking branch 'origin/main' into claude/issue-22443-sys-file-public-scope-backfill

Author: claude

.changeset/22307-cold-boot-catalog-refusal.md

@@ -6,7 +6,7 @@ feat(objectql)!: a cold boot refuses a package-held position or permission-set n
 
 Clause-②: no
 
-<!-- adr-0087: not-required (no-migration-prescription) the refusal removes no key, export or field and changes the shape of no stored body; what an operator does about a refused name is rename or remove one of the two items, which no conversion can choose for them -->
+<!-- adr-0087: registered security-catalog-environment-overlay-refused -->
 
 **BREAKING** — an accept-set narrowing at boot, shipped as `major` on the v18 pre-release line (`.changeset/pre.json` is in `next` pre mode on `main`). A deployment whose environment catalog holds a position or permission-set name that a configured package also declares booted before this release and is refused at boot after it.
 
@@ -16,16 +16,16 @@ Clause-②: no
 
 **What an operator sees.** The kernel reports `Plugin com.objectstack.engine.objectql failed to start`, and the cause is the package door's envelope: `code: 'NAMESPACE_CONFLICT'` (`NAMESPACE_CONFLICT_CODE` is exported), `status: 422`, and `conflicts[]` with `{ catalogType, name, incomingPackageId, existingHolder: { kind: 'environment' } }`. The message names the package that declares each name and the environment catalog that holds it.
 
-**The upgrade shape.** A deployment fails to boot after this release when an active, environment-wide `sys_metadata` row of type `permission` or `position` (or the legacy plural `permissions` / `positions`, which the boot's load folds to the same types) has the name of a permission set or position that a configured package declares. That includes a row saved over a package-held name before the packaged locks refused such saves, whether or not the row was bound to the package, and a row over one of the platform security plugin's own permission sets (`member_default`, `admin_full_access` and the rest it declares).
+**The upgrade shape.** A deployment fails to boot after this release when an active, environment-wide `sys_metadata` row of type `permission` or `position` (or the legacy plural `permissions` / `positions`, which the boot's load folds to the same types) has the name of a permission set or position that a configured package declares. That includes a row saved over a package-held name before the packaged locks refused such saves, whether or not the row was bound to the package, and a row over one of the platform security plugin's own permission sets (`member_default`, `admin_full_access` and the rest it declares). **Run `os migrate security-catalog-overlays` before the first v18 boot**, with the flags and environment the deployment boots with: it lists exactly these rows, and with `--apply` deletes them.
 
-**The one-line fix: rename the item in the package, or rename or delete the environment's item, then restart.**
+**The one-line fix: before the first v18 boot, run `os migrate security-catalog-overlays` to list the rows, then `os migrate security-catalog-overlays --apply` to delete them; or rename the item in the package.**
 
 - **Before upgrading, for a permission set.** On the release you run now, a boot whose environment catalog overlays a package-declared permission set logs at `kernel:ready`: `[security] N package-declared permission set(s) are being shadowed by an environment overlay`, with the set names. Those are the permission sets this release refuses at boot. The audited **Discard Overlay** action on the set's record in Setup (`POST /api/v1/security/permission-sets/<id>/discard-overlay`, documented under "Declared ≠ enforced" on the Permission Sets page) removes the overlay and resyncs the set to the package's definition. So does `DELETE /api/v1/meta/permission/<name>`, which answers "Customization overlay deleted … reset to artifact default". Either works for the platform security plugin's own sets too, and neither needs direct database access. Positions have no such reading and no such action.
-- **After upgrading, for a package you can leave out.** Boot once without the package in the configuration, rename or delete the environment's item through the metadata API (`DELETE /api/v1/meta/permission/<name>`, `DELETE /api/v1/meta/position/<name>`), then add the package back.
-- **After upgrading, for a name the platform security plugin declares, or for any row the metadata API does not reach.** Back the database up, then delete the row in it. The rows that refuse the boot are the active, environment-wide ones of that name: `organization_id IS NULL` and `state = 'active'`, whatever their `package_id`, under the type or its legacy plural: `DELETE FROM sys_metadata WHERE organization_id IS NULL AND state = 'active' AND type IN ('permission', 'permissions') AND name = '<name>';` (for a position, `type IN ('position', 'positions')`). A draft row and an organization-scoped row are not loaded at boot and do not refuse it.
+- **After upgrading, for any refused name.** The same step: `os migrate security-catalog-overlays`, then `--apply`. It needs no server, so a deployment whose boot is refused can run it, and it covers permission sets and positions, a name the platform security plugin declares, and a row stored under a legacy plural.
+- **What the step deletes.** The rows that refuse the boot are the active, environment-wide ones of a package-held name: `organization_id IS NULL` and `state = 'active'`, whatever their `package_id`, under the type or its legacy plural. In SQL, for one permission-set name, the step's deletion is `DELETE FROM sys_metadata WHERE organization_id IS NULL AND state = 'active' AND type IN ('permission', 'permissions') AND name = '<name>';` (for a position, `type IN ('position', 'positions')`). The step runs it through the metadata write path, so each deleted row leaves a `sys_metadata_history` tombstone. A draft row and an organization-scoped row are not loaded at boot and do not refuse it.
 
-`DELETE /api/v1/meta/permission/<name>` and `DELETE /api/v1/meta/position/<name>` reach a row stored under `permission` or `position` only, one row per call. A row stored under the legacy plural `permissions` / `positions` is not reached: the call answers `200` that nothing was found and removes nothing. Where a name has two active rows, for example one bound to no package and one bound to the package, each call removes one. A plural-typed row is removed by Discard Overlay before upgrading (a permission set), or by the SQL above after upgrading, for any name.
+`DELETE /api/v1/meta/permission/<name>` and `DELETE /api/v1/meta/position/<name>` reach a row stored under `permission` or `position` only, one row per call. A row stored under the legacy plural `permissions` / `positions` is not reached: the call answers `200` that nothing was found and removes nothing. Where a name has two active rows, for example one bound to no package and one bound to the package, each call removes one. A plural-typed row is removed by Discard Overlay before upgrading (a permission set), or by `os migrate security-catalog-overlays --apply`, for any name.
 
-No `os` command deletes a `sys_metadata` row offline: `os meta delete` and `os data delete` call a running server. Nothing renames or removes either item automatically.
+`os migrate security-catalog-overlays` is the one `os` command that deletes these rows with no server running; `os meta delete` and `os data delete` call a running server. Nothing renames or removes either item automatically: the step deletes only under `--apply`, and adopts nothing.
 
 **What is NOT refused.** A stored definition under a built-in position name (`org_admin`, `everyone` and the other four): the platform declares its built-in positions itself, after this check, and the stored definition keeps answering first. The same package restarting with its own names. A package whose names the environment catalog does not hold, booting beside the environment's own items.

.changeset/22371-security-catalog-overlays-step.md

@@ -0,0 +1,24 @@
+---
+"@objectstack/cli": minor
+"@objectstack/objectql": minor
+"@objectstack/core": minor
+"@objectstack/runtime": minor
+"@objectstack/spec": minor
+---
+
+feat(cli): `os migrate security-catalog-overlays` lists, and with `--apply` deletes, the environment-wide rows a v18 cold boot refuses — run it before the first v18 boot
+
+Clause-②: yes (widening)
+
+- **What it is for.** A v18 cold boot refuses to start when the environment catalog holds a permission-set or position name that a configured package also declares (ADR-0048 N.3). A deployment upgraded from 17.x can carry such rows: a set or position saved in the environment over a package's name before the packaged locks refused that, or over one of the platform security plugin's own sets (`member_default`, `admin_full_access` and the rest). The server cannot clear them, because it does not start. This step is the offline remedy.
+- **What it lists.** Every active, environment-wide `sys_metadata` row (`organization_id IS NULL`, `state = 'active'`) of type `permission` or `position`, the legacy plurals `permissions` and `positions` included, whose name a configured package holds. That is the population the cold boot refuses: the list is computed from the engine's own reading of who holds a name, not from a copy of it. A draft row and an organization-scoped row are not loaded at boot, so they are never listed. Each row is shown with its stored type, its `package_id` binding and the package that holds its name.
+- **How it gets there without the refusal.** It composes the deployment as `os serve` does: the host config's plugins, the application or the compiled artifact, and the security plugin behind `serve`'s auth gate. It runs the kernel's first phase for them only, and boots without reading `sys_metadata` back into the registry, so the cold-boot check meets nothing and the boot comes up. No server starts. The security plugin's shipped sets are package-held names only where `serve` composes that plugin, so the step reads the environment `serve` reads. With `OS_AUTH_SECRET` (or `AUTH_SECRET` / `BETTER_AUTH_SECRET`) set, or on a development boot, the plugin is composed and its sets are held, unless the `auth` tier is off. The step takes `os serve`'s `--preset` and `--dev` with `serve`'s meaning, read through the rules `serve` reads them by: `--preset` names the tier preset the auth tier falls back to, and `--dev` composes the config's `devPlugins` and makes the boot a development one (as does `NODE_ENV=development`). The report says which way it went. **Run the step with the flags and environment the deployment boots with.**
+- **Preview by default.** The preview boots read-only, like `os migrate plan`. It writes nothing, creates no database file, and exits `1` when it lists a row: the next boot would be refused. `--apply` deletes the listed rows, after a `[y/N]` prompt or `--yes`, and prints one audit line per row: what was deleted, through which write path, or why not. It exits `0` when every listed row is gone. `--json` prints one document (`listed`, `deleted`, `failed`, `rows[]` with `outcome`, `securityPlugin` and `serveFlags`). `--force` gets past a busy SQLite file. `--database-url` / `$OS_DATABASE_URL` name the target.
+- **How it deletes.** A row stored under `permission` or `position` goes through the metadata protocol's own delete, the one `DELETE /api/v1/meta/:type/:name` uses. That writes a `sys_metadata_history` tombstone and a `sys_metadata_audit` row, with actor `os migrate security-catalog-overlays`. A row stored under a legacy plural is out of that door's reach, so it goes through the `sys_metadata` repository beneath it. That delete is addressed by the row's stored type, name, `package_id` and checksum, and writes the same history tombstone.
+- **The refusal names it.** The cold boot's `NAMESPACE_CONFLICT` message now points at `os migrate security-catalog-overlays` for the environment's rows, in place of "through the metadata API on a boot that leaves the package out of the configuration, or in the database". The envelope (`code`, `status`, `conflicts[]`) is unchanged.
+- **What it never does.** It adopts nothing: `managed_by` and `package_id` are never rewritten, and no row is renamed. A row the environment needs under its own name must be re-created under a name no package holds. A host config that exists and cannot be loaded is refused before any row is read, because the held names would be incomplete.
+- **New exports it is built on.**
+  - `@objectstack/objectql` exports `findPackageHeldSecurityCatalogNames(registry)`: every package-held permission-set and position name, with its holder packages. The cold-boot check now reads the same list.
+  - `@objectstack/core` exports the auth gate `os serve` and the step share. `resolvePlatformAuthComposition` answers whether the platform composes `AuthPlugin` and the security plugin beside it, or why not. With it come `resolveStackTiers`, `STACK_TIER_PRESETS`, `CAPABILITY_TO_TIER`, `resolveAuthSecret`, `isDevelopmentBoot`, `DEV_AUTH_SECRET_FALLBACK`, `stackSuppliesAuthPlugin`, `isHostKernelComposition` and the `PlatformAuthComposition` / `PlatformAuthSkipReason` types. `Serve.TIER_PRESETS` and `Serve.CAPABILITY_TO_TIER` are now handles over the core declarations, and `os serve` composes exactly what it composed before.
+  - `@objectstack/runtime`'s `createStandaloneStack` accepts `hydrateMetadataFromDb: false`, which skips the `sys_metadata` read-back. The default stays on.
+  - `@objectstack/spec` registers the ADR-0087 D3 entry `security-catalog-environment-overlay-refused` in its migration registry and `spec-changes.json`, so `os migrate meta --from 17` names the refusal and this step as its remedy. Its projection is the entry under protocol 18 in the upgrade guide (`docs/protocol-upgrade-guide.md`).

.changeset/22410-dev-banner-order.md

@@ -0,0 +1,11 @@
+---
+'@objectstack/cli': patch
+---
+
+fix(cli): `os dev` prints the server's ready banner whole, then the MCP connect block whole, instead of interleaving the two (#22410)
+
+Clause-②: no
+
+- **What a terminal showed.** `os dev` is two processes writing one terminal. The parent prints the MCP connect block (`🤖 MCP server — connect a coding agent:` with its Endpoint, Skill, Connect and Disable rows) when the `serve` child sends `objectstack:listening`, and the child sent that message before it printed its ready banner. Measured under a pty on the Build-with-Claude-Code tutorial project, 6 of 7 boots printed the block above or inside the banner. In 2 of them, banner lines landed between the block's rows, in one case the `➜ Console:` and `➜ MCP:` rows. A setup boot before those seven printed the `🔑 Dev admin` credential lines and the whole plugin section between `Skill` and `Connect`.
+- **What changed.** The child now sends `objectstack:listening` after its ready banner and boot diagnostics have printed. The parent prints the block on that message, so it always comes last, under the banner's `Press Ctrl+C to stop` row. The order follows from the sequence itself, with no timer: the banner is written to the terminal before the message is sent. After the change, every measured boot printed the banner whole and then the block whole.
+- **What did not change.** The message is still `{ type: 'objectstack:listening', port, url }`, carrying the port actually bound. The runtime state file is still written before either announcement, and `objectstack:seed-settled` still follows `objectstack:listening`. A program that spawns `os serve` with an IPC channel receives `objectstack:listening` slightly later than before: after the banner has printed instead of before.

.changeset/22477-flow-text-slot-dollar-root-refused.md

@@ -0,0 +1,43 @@
+---
+'@objectstack/spec': minor
+'@objectstack/lint': patch
+---
+
+A `{{ }}` hole in a flow text slot (a `notify` node's `title` and `message`, a `screen` node's `title` and `description`, a refusing `end` node's `message`) whose root is a `$` name the flow engine does not bind is refused. It is refused at the same doors, and by the same judge (`textSlotTemplateRefusal`), as a single-brace token: the node contracts, `registerFlow` and `objectstack validate`. `'By {{ $User.Id }}'` used to pass all three and send `'By '`.
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: registered flow-text-slot-unbound-dollar-root-refused -->
+
+**BREAKING**: an accept-set narrowing on a published authoring surface, shipped as `minor` under the launch-window convention for accept-set narrowings.
+
+**Why.** The hole grammar admits `$` in a name so that the engine's own variables have a spelling (`{{ $error.message }}`). That also makes `{{ $User.Id }}` a well-formed hole, but over a root no flow variable answers to. The text went out with the fragment missing, the run reported success, and nothing warned. Its single-brace spelling, `{$User.Id}`, was already refused with a remedy. The `$` names are reserved for the engine: a resume signal may not write one.
+
+**What is refused.**
+
+- A hole whose root is a `$` name other than the variables the engine binds: `$record`, `$runId`, `$flowName`, `$flowLabel`, `$error`, and a flat-graph `loop`'s `$loopItems` / `$loopIndex`.
+  - `NotifyConfigSchema`, `ScreenConfigSchema` and `EndConfigSchema` raise a `custom` issue at the slot's key.
+  - `registerFlow` refuses the flow, and a stored flow carrying such a hole is skipped at boot with a warn naming it.
+  - `objectstack validate` reports `expression-invalid` at `error`.
+- The remedy for `{{ $User.<path> }}` is the sentence `{$User.<path>}` gets: compute the value into a variable with an `assignment` node, whose value slot still reads that spelling, then write the variable as a hole. Any other root is named in the refusal, beside the variables the engine does bind.
+- A single-brace path token over such a root (`'Failed: {$caught.message}'`) is no longer prescribed the `{{ }}` spelling, which would be refused in turn; it gets the same remedy.
+
+**Unchanged.** `{{ $error.message }}`, `{{ record.name }}`, a node output `{{ lookup.result }}` and every hole over an engine-bound `$` variable. The template engine binds no new variable.
+
+**`@objectstack/lint`.** In a text slot, `flow-bare-dollar-reference` prescribes the hole for a bare `$X.y` written outside the holes only when the judge admits that hole. A bare `$User.Id` gets the judge's refusal and remedy instead of a `{{ $User.Id }}` the judge refuses.
+
+## FROM → TO
+
+| you wrote | write instead |
+|:--|:--|
+| `message: 'By {{ $User.Id }}'` | an `assignment` node first, `assignments: { by: '{$User.Id}' }`, then `message: 'By {{ by }}'` |
+| `errorVariable: '$caught'` with `message: 'Failed: {{ $caught.message }}'` | `errorVariable: 'caught'` with `'Failed: {{ caught.message }}'`, or keep the default `$error` and write `{{ $error.message }}` |
+
+**The one-line fix: compute a run-user value into a variable first, and name a variable the flow binds itself without the `$`.**
+
+**Who is affected, measured.** The last published spec, `@objectstack/spec@17.7.0` (npm `latest`), has no text-slot judge. Its `NotifyConfigSchema.title` / `.message`, `ScreenConfigSchema.title` / `.description` and `EndConfigSchema.message` are plain strings, so it accepts `'By {{ $User.Id }}'` in every one of these slots. Its single-brace interpolator substituted the inner `{ $User.Id }` token and left a literal brace on each side. This repository was measured with `git grep` over `examples`, `packages`, `skills`, `apps` and `content`: no flow text slot outside tests carries a `{{ $… }}` hole other than `{{ $error.… }}`. Deployed metadata and other repositories were not measured.
+
+### The kit
+
+- **The refusal.** `textSlotTemplateRefusal` in `automation/flow-text-slot-template.ts` reads one package-internal list of the `$` variables the engine binds. `@objectstack/service-automation`'s `text-slot-template.test.ts` scans that package's sources for every `$` variable they bind by name, and fails when the list misses one.
+- **The ledger.** The D3 semantic entry `flow-text-slot-unbound-dollar-root-refused` (protocol 18). There is no D2 conversion: what the hole was meant to read is not in the flow.