feat(spec,service-storage,client)!: one upload-scope vocabulary for the upload requests, the sys_file select, the upload doors and the SDK (#22470) (#22647)

Fixes #22470
Clause-②: yes (narrowing)

One upload-scope vocabulary, declared once in `@objectstack/spec` and
read by the two upload requests, the `sys_file` scope select, the two
upload-start doors and the SDK's `storage.upload`. A scope outside it is
now a caller error, answered `400 INVALID_REQUEST` naming the allowed
values, instead of the data engine's `invalid_option` relayed as `500
INTERNAL` with a message telling the operator to restore the data
engine.

Direction: triage `6080435724` as amended by `6081565553` (the
vocabulary is the `sys_file` select, not `StorageScopeSchema`), standing
per `6092602285`; the client half ruled option A in seat answer
`6095219171`.

## What changes

- **`@objectstack/spec`**: new `UploadScopeSchema` and type
`UploadScope` beside the upload request schemas (`api/storage.zod.ts`,
exported from `@objectstack/spec/api`): `user`, `tenant`, `private`,
`temp`, `attachments`. `GetPresignedUrlRequestSchema.scope` and
`InitiateChunkedUploadRequestSchema.scope` read it; the `user` default
and the description are kept. `StorageScopeSchema` is not touched.
- **`@objectstack/service-storage`, `SystemFile`**: the `scope` select
builds its options from `UploadScopeSchema.options`, in the enum's
order. The labels and the comment explaining `attachments` stay local,
in a label map keyed by `UploadScope`, so an enum member added without a
label fails `tsc`. The stored values and labels are unchanged (pinned
byte-equal to the old literal list).
- **`@objectstack/service-storage`, `registerStorageRoutes`**: the one
scope gate both upload-start handlers already asked
(`requireAcceptedUploadScope`, from the `public` retirement) now asks
`UploadScopeSchema.safeParse` and refuses everything else: any string, a
case variant, `null`, a number. It runs before the size gate, before any
row, URL or backend call. One gate, one status, one code. The former
`public`-only refusal is folded into it rather than left beside it,
keeping its message family and, for `public`, its `acl 'public_read'`
remedy. An omitted scope is still the default `user`, and a real engine
fault still answers `500`.
- **`@objectstack/client`**: `storage.upload` types its `scope`
parameter `UploadScope` instead of `string` (one parameter type, one
type import). `getPresignedUrl` and `initChunkedUpload` take the request
types and narrow with them.
- **ADR-0087**: D3 entry `upload-request-scope-closed`, `registry.ts`
regenerated. Changeset: spec `minor`, client `minor`, service-storage
`patch`, registered disposition. No `spec-changes.json` or upgrade-guide
regeneration is owed: `check:spec-changes` and `check:upgrade-guide`
project in memory and are green.
- **Generated**: the `api-surface`, `export-origins`, `declaration-map`
and `json-schema.manifest` shards for `api`, and the two reference pages
under `content/docs/references` (`gen:docs`).
`type-alias-convention.pin.test.ts` gains the isomorphic pin for
`UploadScopeSchema` (773 → 774, with its receipt), which
`check:spec-parsed-alias` reads as its registry.

## Evidence (head `3e8227f5df`, after merging `origin/main` at
`1b99388505` through `os-regen-merge.sh`)

All runs under `os-verify-lock`.

- **New pins.** `packages/spec/src/api/storage.test.ts`: the enum lists
the five in order and refuses `public`. Each request accepts all five,
keeps the `user` default, and refuses `avatars`, `public`, `User` and
the empty string at parse, with issue code `invalid_value` on path
`scope`.
`packages/services/service-storage/src/upload-scope-vocabulary.test.ts`
runs on a real `ObjectQL` over `SqlDriver` (sqlite in memory) with the
real `SystemFile` and `SystemUploadSession`:
- the select's values equal the enum's, in order, and the labels are
unchanged;
- `scope: 'avatars'` on each door → `400 INVALID_REQUEST`, the message
names the five values, and there are zero `sys_file` rows, zero session
rows, no presign and no backend initiate call;
  - `null`, `7` and `User` → `400`;
- control: every allowed scope, and an omitted one, → `200`, and the
engine stores it;
  - control: an engine insert fault on scope `user` → `500 INTERNAL`.
- **Suites.**
  - spec `--project local`: 642 files, 19157 passed, 1 todo.
  - spec `--project repo`: 54 files, 915 passed.
  - service-storage: 50 files, 837 passed.
  - client: 51 files, 653 passed.
  - typecheck exit 0 for spec, service-storage and client.
  - full workspace build: 72 of 72 tasks.
- **Ablation** (`scripts/ablation-replace.mjs`, wrap mode with trap
restore, at `3e8227f5df`). The refusal condition in
`requireAcceptedUploadScope` is replaced by `return true`: anchor 1 → 0,
marker 0 → 1 → 0.
  - Green leg: 54 of 54.
- Mutated leg: 3 failed, 51 passed. The `avatars` pin goes red with
`expected 500 to be 400`, which is the card's defect reproduced over the
real engine. The `null` / number / case pin goes red with `expected 200
to be 400`, and the `public` pin goes red.
  - Restored to blob `b355ea0a5b` == HEAD, with `git diff HEAD` empty.
- No build leg is owed: the pins import the door from source by relative
path.
- **Reverse verification of the type change.** Before the client edit,
`@objectstack/client` typecheck exited 2 with TS2322 (`string` not
assignable to the five-value union) in `storage.upload`, and
`@objectstack/client#build` failed its DTS step. So the rebuilt spec
declarations were what the consumer read. After the edit it exits 0.
- **Gates.** `dispatch-gates --commands` on the final diff derives 119
commands. All 119 ran at `3e8227f5df` with exit 0, including
`check:dts-closure`, `check:skill-examples`,
`check:dual-build-cjs-loads`, `check:i18n` (service-storage: 7 bundles
in sync) and `check:type-check-debt`. `--ran`: 119 derived, 119 run, 0
NOT MEASURED, 0 UNRUN. `check:generated`: 15 of 15 up to date after the
merge.
- **Not run locally**: integration layers and the repo-wide lint, which
CI owns.

## Reach (measured)

- **In-repo door callers.** The SDK `storage.upload` (default `user`),
two README examples passing `user`, and the dogfood upload sites: 7 send
`attachments` and 2 send `user`. The service-storage and organizations
tests send only vocabulary values. Nothing sends an off-vocabulary scope
through a door. `examples/**` and `apps/**`: 0.
- **objectui**, at the pin `20c6d351ad` and at main `12ff256313`: 2
adapter callers. The console sends no scope, and the record attachments
panel sends `attachments`. There are 0 calls to the SDK upload and 0
imports of the request types, so the Console Pin Gate is unaffected.
- **Console bundle**: not measured (no `packages/console/dist` in this
checkout).

## Acceptance notes

- The doors now also refuse an explicit `null` scope, which used to be
read as the default `user`. This agrees with the published schema, which
refuses `null` at parse. It is pinned, and ruled to stand in
`6095219171`. No measured caller sends `null`.
- No `STEP18_RATIONALE` fragment, which was ruled not owed.
- `sys_upload_session.scope` stays a free text column. It mirrors the
file row's scope, and only the chunked door writes it, after the gate.
Noted, not filed.
- The contract review is owed before the queue.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01VZqqwTj2wsihZEbfT6yyYN)_

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: objectstack-fleet[bot]

.changeset/22470-upload-scope-vocabulary.md

@@ -0,0 +1,40 @@
+---
+'@objectstack/spec': minor
+'@objectstack/service-storage': patch
+'@objectstack/client': minor
+---
+
+feat(spec,service-storage,client)!: one upload-scope list — the upload requests' `scope` and the SDK's `storage.upload` close to the new `UploadScope` enum, the `sys_file` scope select is built from it, and the upload doors answer any other scope with `400` naming the allowed values instead of `500 INTERNAL` (#22470)
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: registered upload-request-scope-closed -->
+
+**BREAKING** — an accept-set narrowing on a published request contract and on the SDK method that sends it, shipped as `minor` under the launch-window convention for accept-set narrowings, plus one new export.
+
+The presigned and chunked upload requests declared `scope` an open string, while the stored file record (`sys_file.scope`, a closed select) only ever took `user`, `tenant`, `private`, `temp` and `attachments`. Any other value reached the `sys_file` insert, the data engine refused it as an invalid option, and the upload door answered that caller error as `500 INTERNAL`, with a message telling the operator to restore the data engine.
+
+### What changes
+
+- **`UploadScopeSchema` / `UploadScope`** (`@objectstack/spec`, new, exported from `@objectstack/spec/api`): the upload-scope vocabulary, declared once — `user`, `tenant`, `private`, `temp`, `attachments`. It is a different list from `StorageScopeSchema`, which classifies a storage configuration and is not read by any upload.
+- **`GetPresignedUrlRequestSchema.scope` and `InitiateChunkedUploadRequestSchema.scope`** read it. The default stays `user`. A literal outside the list fails `tsc`, and a parse refuses it on the `scope` key.
+- **`client.storage.upload(file, scope)`** (`@objectstack/client`): the `scope` parameter is typed `UploadScope` instead of `string`, default `user` unchanged, so a scope outside the list fails `tsc` at the SDK call. `client.storage.getPresignedUrl` and `client.storage.initChunkedUpload` take the request types above and narrow with them.
+- **The `sys_file` scope select** (`@objectstack/service-storage`) takes its options from the enum, in its order, with the same labels. The stored values do not change.
+- **The presigned upload door and the chunked upload door** answer a scope outside the list — any string, a case variant, `null`, a number — with `400 INVALID_REQUEST`, naming the allowed values, before a file record, a session record, an upload URL or a backend upload exists. `public` is refused by the same gate and keeps its remedy (`acl: 'public_read'` on the stored file record). An omitted scope is the default `user`, as before. A real data-engine fault still answers `500`.
+
+### FROM → TO
+
+| before | what to write instead |
+| --- | --- |
+| an upload naming a scope outside the list (a key prefix such as `avatars`, a folder, a record path) | one of `user`, `tenant`, `private`, `temp`, `attachments`, or no `scope` for the default `user` |
+| `client.storage.upload(file, scope)` called with a `scope` typed `string` (`@objectstack/client`) | pass one of the five, or type the value `UploadScope` (`import type { UploadScope } from '@objectstack/spec/api'`) |
+| a caller passing a scope typed `string` into either upload request | type it `UploadScope` |
+
+**The one-line fix: send one of the five upload scopes, or none.** `attachments` is for a file whose referrers are record attachment rows (it is what orphan tombstoning reads); `temp` for a scratch file; otherwise `user` or `tenant`.
+
+**No stored file record moves**: no upload naming another scope ever succeeded, so no stored record carries one.
+
+### The kit
+
+- **D3 entry `upload-request-scope-closed`** carries the judgement no rewrite can make: which scope a file that was sent under a free name really belongs under. No D2 conversion: no metadata type carries the upload request, so `os migrate meta` has no authored source to rewrite.
+- **Liveness.** No ledger row: the liveness ledger walks metadata types, and no metadata type carries the upload request.

content/docs/references/api/storage.mdx

@@ -19,8 +19,8 @@ rather than proxying bytes through the API server.
 ## TypeScript Usage
 
 ```typescript
-import { CompleteChunkedUploadRequestSchema, CompleteChunkedUploadResponseSchema, CompleteUploadRequestSchema, FileDownloadUrlResponseSchema, FileTypeValidationSchema, FileUploadResponseSchema, GetPresignedUrlRequestSchema, InitiateChunkedUploadRequestSchema, InitiateChunkedUploadResponseSchema, PresignedUrlResponseSchema, RawUploadResponseSchema, UploadChunkRequestSchema, UploadChunkResponseSchema, UploadProgressSchema } from '@objectstack/spec/api';
-import type { CompleteChunkedUploadRequest, CompleteChunkedUploadResponse, CompleteUploadRequest, FileDownloadUrlResponse, FileTypeValidation, FileUploadResponse, GetPresignedUrlRequest, InitiateChunkedUploadRequest, InitiateChunkedUploadResponse, PresignedUrlResponse, RawUploadResponse, UploadChunkRequest, UploadChunkResponse, UploadProgress } from '@objectstack/spec/api';
+import { CompleteChunkedUploadRequestSchema, CompleteChunkedUploadResponseSchema, CompleteUploadRequestSchema, FileDownloadUrlResponseSchema, FileTypeValidationSchema, FileUploadResponseSchema, GetPresignedUrlRequestSchema, InitiateChunkedUploadRequestSchema, InitiateChunkedUploadResponseSchema, PresignedUrlResponseSchema, RawUploadResponseSchema, UploadChunkRequestSchema, UploadChunkResponseSchema, UploadProgressSchema, UploadScopeSchema } from '@objectstack/spec/api';
+import type { CompleteChunkedUploadRequest, CompleteChunkedUploadResponse, CompleteUploadRequest, FileDownloadUrlResponse, FileTypeValidation, FileUploadResponse, GetPresignedUrlRequest, InitiateChunkedUploadRequest, InitiateChunkedUploadResponse, PresignedUrlResponse, RawUploadResponse, UploadChunkRequest, UploadChunkResponse, UploadProgress, UploadScope } from '@objectstack/spec/api';
 
 // Validate data
 const result = CompleteChunkedUploadRequestSchema.parse(data);
@@ -224,7 +224,7 @@ const result = CompleteChunkedUploadRequestSchema.parse(data);
 | **filename** | `string` | ✅ | Original filename |
 | **mimeType** | `string` | ✅ | File MIME type |
 | **size** | `number` | ✅ | File size in bytes |
-| **scope** | `string` | optional (default: `"user"`) | Storage scope the new file is filed under (default user; attachments for the record attachments surface). Not an access setting: the upload doors refuse public, and a file is served without sign-in only when its stored file record carries acl 'public_read' (ADR-0104). The upload request carries no acl; every upload is stored private. |
+| **scope** | `Enum<'user' \| 'tenant' \| 'private' \| 'temp' \| 'attachments'>` | optional (default: `"user"`) | Storage scope the new file is filed under (default user; attachments for the record attachments surface). Not an access setting: the upload doors refuse public, and a file is served without sign-in only when its stored file record carries acl 'public_read' (ADR-0104). The upload request carries no acl; every upload is stored private. |
 | **bucket** | `string` | optional | Specific bucket override (admin only) |
 
 
@@ -240,7 +240,7 @@ const result = CompleteChunkedUploadRequestSchema.parse(data);
 | **mimeType** | `string` | ✅ | File MIME type |
 | **totalSize** | `integer` | ✅ | Total file size in bytes |
 | **chunkSize** | `integer` | optional (default: `5242880`) | Size of each chunk in bytes (minimum 5MB per S3 spec) |
-| **scope** | `string` | optional (default: `"user"`) | Storage scope the new file is filed under (default user; attachments for the record attachments surface). Not an access setting: the upload doors refuse public, and a file is served without sign-in only when its stored file record carries acl 'public_read' (ADR-0104). The upload request carries no acl; every upload is stored private. |
+| **scope** | `Enum<'user' \| 'tenant' \| 'private' \| 'temp' \| 'attachments'>` | optional (default: `"user"`) | Storage scope the new file is filed under (default user; attachments for the record attachments surface). Not an access setting: the upload doors refuse public, and a file is served without sign-in only when its stored file record carries acl 'public_read' (ADR-0104). The upload request carries no acl; every upload is stored private. |
 | **bucket** | `string` | optional | Specific bucket override (admin only) |
 | **metadata** | `Record<string, string>` | optional | Custom metadata key-value pairs |
 
@@ -497,3 +497,18 @@ const result = CompleteChunkedUploadRequestSchema.parse(data);
 
 ---
 
+## UploadScope
+
+Storage scope an uploaded file is filed under
+
+### Allowed Values
+
+* `user`
+* `tenant`
+* `private`
+* `temp`
+* `attachments`
+
+
+---
+

content/docs/references/index.mdx

@@ -1,7 +1,7 @@
 ---
 title: Protocol reference — every schema by module
 navTitle: Protocol Reference
-description: Every schema published by @objectstack/spec — 1515 schemas across 14 protocol modules
+description: Every schema published by @objectstack/spec — 1516 schemas across 14 protocol modules
 ---
 
 {/* ⚠️  AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
@@ -21,7 +21,7 @@ counts are sums of the rows they head. Regenerate with
 | Module | Pages | Schemas | Description |
 | :--- | ---: | ---: | :--- |
 | [AI Protocol](/docs/references/ai) | 12 | 68 | Agents, tools, skills, RAG and knowledge sources, model registry, conversations. |
-| [API Protocol](/docs/references/api) | 32 | 431 | REST contracts, endpoints, routing, realtime, batch, discovery. |
+| [API Protocol](/docs/references/api) | 32 | 432 | REST contracts, endpoints, routing, realtime, batch, discovery. |
 | [Automation Protocol](/docs/references/automation) | 13 | 70 | Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execution records. |
 | [Data Protocol](/docs/references/data) | 29 | 175 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. |
 | [Identity Protocol](/docs/references/identity) | 5 | 27 | Users and accounts, organizations, positions, SCIM provisioning. |
@@ -34,7 +34,7 @@ counts are sums of the rows they head. Regenerate with
 | [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. |
 | [System Protocol](/docs/references/system) | 34 | 275 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. |
 | [UI Protocol](/docs/references/ui) | 16 | 166 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. |
-| **Total** | **195** | **1515** | 14 protocol modules |
+| **Total** | **195** | **1516** | 14 protocol modules |
 
 ---
 
@@ -63,7 +63,7 @@ Agents, tools, skills, RAG and knowledge sources, model registry, conversations.
 
 ## API Protocol
 
-**Source:** `packages/spec/src/api/` · **Import:** `@objectstack/spec/api` · **32 pages, 431 schemas**
+**Source:** `packages/spec/src/api/` · **Import:** `@objectstack/spec/api` · **32 pages, 432 schemas**
 
 REST contracts, endpoints, routing, realtime, batch, discovery.
 
@@ -98,7 +98,7 @@ REST contracts, endpoints, routing, realtime, batch, discovery.
 | [`rest-server.zod.ts`](/docs/references/api/rest-server) | `BatchEndpointsConfig`, `CrudEndpointsConfig`, `CrudOperation`, `EndpointRegistry`, `GeneratedEndpoint`, `MetadataEndpointsConfig`, `RestApiConfig`, `RestServerConfig`, `RouteGenerationConfig` |
 | [`router.zod.ts`](/docs/references/api/router) | `ConflictResolutionStrategy`, `HttpMethod`, `RouteCategory`, `RouteDefinition`, `RouterConfig` |
 | [`sortability.zod.ts`](/docs/references/api/sortability) | `FieldSortability`, `ObjectSortability` |
-| [`storage.zod.ts`](/docs/references/api/storage) | `CompleteChunkedUploadRequest`, `CompleteChunkedUploadResponse`, `CompleteUploadRequest`, `FileDownloadUrlResponse`, `FileTypeValidation`, `FileUploadResponse`, `GetPresignedUrlRequest`, `InitiateChunkedUploadRequest`, `InitiateChunkedUploadResponse`, `PresignedUrlResponse`, `RawUploadResponse`, `UploadChunkRequest`, `UploadChunkResponse`, `UploadProgress` |
+| [`storage.zod.ts`](/docs/references/api/storage) | `CompleteChunkedUploadRequest`, `CompleteChunkedUploadResponse`, `CompleteUploadRequest`, `FileDownloadUrlResponse`, `FileTypeValidation`, `FileUploadResponse`, `GetPresignedUrlRequest`, `InitiateChunkedUploadRequest`, `InitiateChunkedUploadResponse`, `PresignedUrlResponse`, `RawUploadResponse`, `UploadChunkRequest`, `UploadChunkResponse`, `UploadProgress`, `UploadScope` |
 | [`versioning.zod.ts`](/docs/references/api/versioning) | `VersionDefinition`, `VersionNegotiationResponse`, `VersionStatus`, `VersioningConfig`, `VersioningStrategy` |
 | [`websocket.zod.ts`](/docs/references/api/websocket) | `AckMessage`, `CursorMessage`, `CursorPosition`, `DocumentState`, `EditMessage`, `EditOperation`, `EditOperationType`, `ErrorMessage`, `EventMessage`, `EventPattern`, `EventSubscription`, `PingMessage`, `PongMessage`, `PresenceMessage`, `PresenceState`, `PresenceUpdate`, `SimpleCursorPosition`, `SimplePresenceState`, `SubscribeMessage`, `UnsubscribeMessage`, `UnsubscribeRequest`, `WebSocketConfig`, `WebSocketEvent`, `WebSocketMessage`, `WebSocketMessageType`, `WebSocketPresenceStatus`, `WebSocketServerConfig` |
 

packages/client/src/index.ts

@@ -37,6 +37,7 @@ import {
   CompleteChunkedUploadRequest,
   CompleteChunkedUploadResponse,
   UploadProgress,
+  UploadScope,
   ListNotificationsResponse,
   MarkNotificationsReadResponse,
   MarkAllNotificationsReadResponse,
@@ -5189,7 +5190,7 @@ export class ObjectStackClient {
    * Storage Services
    */
   storage = {
-    upload: async (file: any, scope: string = 'user'): Promise<FileUploadResponse> => {
+    upload: async (file: any, scope: UploadScope = 'user'): Promise<FileUploadResponse> => {
         // 1. Get Presigned URL
         const presignedReq: GetPresignedUrlRequest = {
             filename: file.name,

packages/services/service-storage/src/objects/system-file.object.ts

@@ -1,6 +1,28 @@
 // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
 
 import { ObjectSchema, Field } from '@objectstack/spec/data';
+import { UploadScopeSchema, type UploadScope } from '@objectstack/spec/api';
+
+/**
+ * Display labels of the `scope` select, one per upload scope.
+ *
+ * The VALUES are not listed here: the select reads them off
+ * `UploadScopeSchema`, the one upload-scope list `@objectstack/spec` declares
+ * for the upload requests and the upload doors too (#22470), so the store, the
+ * request and the doors cannot drift apart. `Record<UploadScope, …>` makes a
+ * member added to that enum without a label here a compile error.
+ */
+const UPLOAD_SCOPE_LABELS: Record<UploadScope, string> = {
+  user: 'User',
+  tenant: 'Tenant',
+  private: 'Private',
+  temp: 'Temp',
+  // Files uploaded through the generic Attachments surface (#2727).
+  // Their only legitimate referrers are sys_attachment join rows, so
+  // this scope is the discriminator for orphan tombstoning (#2755) —
+  // field-attachment scopes above are never tombstoned.
+  attachments: 'Attachments',
+};
 
 /**
  * System File Object
@@ -62,19 +84,12 @@ export const SystemFile = ObjectSchema.create({
     // are rewritten to `user` by `backfill-sys-file-public-scope.ts`, the
     // operator step that must run before a copy of such a row can succeed.
     // Anonymous download is `acl: 'public_read'` and nothing else.
+    //
+    // The options are `UploadScopeSchema`'s values, in its order, each with
+    // its label above — the list the upload requests and doors read (#22470).
     scope: Field.select({
       label: 'Scope',
-      options: [
-        { label: 'User', value: 'user' },
-        { label: 'Tenant', value: 'tenant' },
-        { label: 'Private', value: 'private' },
-        { label: 'Temp', value: 'temp' },
-        // Files uploaded through the generic Attachments surface (#2727).
-        // Their only legitimate referrers are sys_attachment join rows, so
-        // this scope is the discriminator for orphan tombstoning (#2755) —
-        // field-attachment scopes above are never tombstoned.
-        { label: 'Attachments', value: 'attachments' },
-      ],
+      options: UploadScopeSchema.options.map((value) => ({ label: UPLOAD_SCOPE_LABELS[value], value })),
     }),
 
     bucket: Field.text({