feat(verify): the reference-value shape family, runnable as os verify --reference-shapes (#15953 slice 1)

.changeset/15953-verify-reference-shape-family.md

@@ -0,0 +1,30 @@
+---
+'@objectstack/verify': minor
+'@objectstack/cli': minor
+---
+
+feat(verify): the reference-value shape family — every client-writable reference field refuses a value that is not a record id (ADR-0104 D1), runnable as `os verify --reference-shapes` (#15953)
+
+Clause-②: yes
+
+A third derived proof family beside `runCrudVerification` and `runRlsProofs`. It needs no hand-written test: from the app's own objects it takes every field whose type is in `REFERENCE_VALUE_TYPES` (`lookup`, `master_detail`, `user`, `tree`), reads what the declaration promises from `valueSchemaFor(field, 'stored')` (a record-id string, or an array of them when the field is multi-valued), and asks the engine whether three wrong-shaped values are refused: a boolean, a number, and an embedded record object (the expanded read shape written back). A multi-valued field gets each one as an array element. Every probe is written on insert and on update, through two doors: the REST data routes as the signed-in caller, and the engine's own write as the system principal.
+
+A probe counts as refused only when the engine names the probed field in its refusal. A refusal that names something else is reported `unattributed`, so an engine that refused every write would not read as proven. A field is `proven` (every probe refused), `holes` (a probe was admitted), `failed` (a door faulted), `inconclusive` (a probe could not be attributed or written; the report says why) or `excluded` (declared `readonly` or `system`, which the caller does not write). `holes + failed` is the report's `hardFailures`.
+
+**`@objectstack/verify` — new exports.**
+
+- `runReferenceShapeProofs(stack, token, config, { doors? })` → `ReferenceShapeReport`. Run it on a fresh stack of its own: a door that admits a probe lands a row.
+- `deriveReferenceShapeCases(config)`, `referenceShapeProbeValue(kind, multiple, id)`, `formatReferenceShapeReport(report)`, `REFERENCE_SHAPE_PROBE_ID`.
+- Types: `ReferenceShapeCase`, `ReferenceShapeDoor`, `ReferenceShapeOperation`, `ReferenceShapeProbeKind`, `ReferenceShapeProbeResult`, `ReferenceShapeFieldStatus`, `ReferenceShapeFieldResult`, `ReferenceShapeReport`, `ReferenceShapeProofOptions`.
+
+```ts
+import { bootStack, runReferenceShapeProofs, formatReferenceShapeReport } from '@objectstack/verify';
+
+const stack = await bootStack(myApp);
+const report = await runReferenceShapeProofs(stack, await stack.signIn(), myApp);
+console.log(formatReferenceShapeReport(report));
+expect(report.summary.hardFailures).toBe(0);
+await stack.stop();
+```
+
+**`@objectstack/cli` — `os verify --reference-shapes`.** Opt-in, like `--rls`, on a fresh boot of its own. It is opt-in because the engine still admits wrong-shaped elements on a `multiple: true` `lookup` or `user` field (#22908), so an app that declares one would fail the run until the engine refuses them. A field with holes, or a door that faulted, counts in `hardFailures` and fails the run. The `--json` document carries the report as `referenceShapes` when the flag is given; without the flag the run and its document are unchanged.

content/docs/deployment/cli.mdx

@@ -1922,6 +1922,7 @@ only when the first passes.
 os verify                                      # Author-time rules, then CRUD round-trip fidelity
 os verify --app ./objectstack.config.ts --rls  # Also prove the cross-owner RLS invariant
 os verify --rls --multi-tenant --json          # Org-scoped boot, structured report
+os verify --reference-shapes                   # Also prove reference fields refuse non-id values
 ```
 
 1. **Author-time rules.** The rule registry `os validate` runs, over the stack
@@ -1943,7 +1944,14 @@ os verify --rls --multi-tenant --json          # Org-scoped boot, structured rep
    fidelity failure fails the run. With `--rls`, a second fresh boot proves the
    cross-owner invariant — a member must not write what it cannot read — for a
    probe persona and for one persona per declared position, and a hole fails
-   the run. `--multi-tenant` boots org-scoped so tenant-isolation RLS policies
+   the run. With `--reference-shapes`, another fresh boot proves that every
+   client-writable reference field (`lookup`, `master_detail`, `user`, `tree`)
+   refuses a value that is not a record id — a boolean, a number, an embedded
+   record object — on insert and update, through the REST door and the system
+   door; a field that admits one fails the run. It is opt-in because the engine
+   still admits wrong-shaped elements on a `multiple: true` `lookup` or `user`
+   field, so an app that declares one would fail until the engine refuses them.
+   `--multi-tenant` boots org-scoped so tenant-isolation RLS policies
    apply; a walled tenancy posture (`OS_TENANCY_POSTURE` set to `isolated` or
    `group`) asks for the same boot.
 
@@ -1970,8 +1978,8 @@ document carries depends on where the run ended:
   `path`, `message`, `hint`, plus `package` for a finding raised inside one
   package of the artifact), or the schema issues when the stack does not parse.
 - **Reached the runtime stage:** the report — `app`, `config`, `multiTenant`,
-  `crud`, `rls` (with `--rls`) and `hardFailures`, the count the exit status is
-  read from.
+  `crud`, `rls` (with `--rls`), `referenceShapes` (with `--reference-shapes`)
+  and `hardFailures`, the count the exit status is read from.
 - **Could not run:** `{ "error": "<sentence>" }`, plus `code` and `httpStatus`
   when the failure carries them.
 

packages/cli/src/commands/verify.ts

@@ -15,8 +15,11 @@ import {
   provisionRlsPositionPersona,
   declaredPositionNames,
   rlsProbeSecurity,
+  runReferenceShapeProofs,
+  formatReferenceShapeReport,
   type VerifyReport,
   type RlsReport,
+  type ReferenceShapeReport,
   type RlsProbeDescriptor,
   type RlsPositionPersonaInput,
 } from '@objectstack/verify';
@@ -79,17 +82,21 @@ export function resolveVerifyMultiTenant(flags: { 'multi-tenant'?: boolean }): b
  *      asserting runtime behavior the static gates can't see:
  *        - data fidelity: author → write → read → assert, per object/field type
  *        - authorization (--rls): "you can't write what you can't read" (#1994 class)
+ *        - reference-value shape (--reference-shapes, ADR-0104 D1): every
+ *          client-writable reference field refuses a value that is not a
+ *          record id, through the REST door and the system door
  *
  * Exits non-zero on either stage's failures so it drops straight into CI.
  */
 export default class Verify extends Command {
   static override description =
-    'Run the author-time rules os validate runs, then boot the app in-process and verify it through the real HTTP stack (CRUD round-trip fidelity + the cross-owner RLS invariant)';
+    'Run the author-time rules os validate runs, then boot the app in-process and verify it through the real HTTP stack (CRUD round-trip fidelity, plus the cross-owner RLS invariant and the reference-value shape proofs on request)';
 
   static override examples = [
     '<%= config.bin %> verify',
     '<%= config.bin %> verify --app ./objectstack.config.ts --rls',
     '<%= config.bin %> verify --rls --multi-tenant --json',
+    '<%= config.bin %> verify --reference-shapes',
   ];
 
   static override flags = {
@@ -101,6 +108,10 @@ export default class Verify extends Command {
       description: 'Also run the cross-owner RLS invariant (a fresh member must not write what it cannot read)',
       default: false,
     }),
+    'reference-shapes': Flags.boolean({
+      description: 'Also prove every client-writable reference field refuses a value that is not a record id (a boolean, a number, an embedded record), through the REST door and the system door (ADR-0104 value shapes)',
+      default: false,
+    }),
     'multi-tenant': Flags.boolean({
       description: 'Boot org-scoped (register the @objectstack/organizations plugin) so tenant-isolation RLS policies apply (also honors a walled $OS_TENANCY_POSTURE, and the legacy $OS_MULTI_ORG_ENABLED it falls back to)',
       default: false,
@@ -178,6 +189,7 @@ export default class Verify extends Command {
   private async runVerification(flags: {
     app?: string;
     rls: boolean;
+    'reference-shapes': boolean;
     'multi-tenant': boolean;
     json: boolean;
   }): Promise<void> {
@@ -278,6 +290,21 @@ export default class Verify extends Command {
       }
     }
 
+    // The reference-value shape proofs run on a fresh stack of their own, for
+    // the reason the RLS proofs do: a door that admits a probe lands a row,
+    // and the base records the family creates would collide with the other
+    // phases' on a unique field.
+    let referenceShapes: ReferenceShapeReport | undefined;
+    if (flags['reference-shapes']) {
+      const shapeStack = await bootStack(config, { multiTenant, hostRoot });
+      try {
+        const adminToken = await shapeStack.signIn();
+        referenceShapes = await runReferenceShapeProofs(shapeStack, adminToken, config);
+      } finally {
+        await shapeStack.stop();
+      }
+    }
+
     // Failure contract: a "real" runtime break the app's author must see.
     // A degraded RLS probe counts: the run reported verdicts it could not have
     // established, which is worse than no verifier at all.
@@ -293,16 +320,22 @@ export default class Verify extends Command {
       crud.summary.fidelityGaps +
       (rls?.totals.holes ?? 0) +
       (rls?.probe.degraded ? 1 : 0) +
-      (rls?.positionCoverage.notRun.length ?? 0);
+      (rls?.positionCoverage.notRun.length ?? 0) +
+      // A reference field that ADMITTED a wrong-shaped value, or a door that
+      // faulted on one. Inconclusive fields (the probe could not be attributed
+      // or written) are reported, not counted, exactly as CRUD's
+      // `needs-fixture` is.
+      (referenceShapes?.summary.hardFailures ?? 0);
 
     if (flags.json) {
       // `emitJson`, not `this.log`: stdout is reserved (see `run()`), so the
       // report has to leave through the real stream — and awaiting the write
       // drains a report larger than one pipe buffer before `this.exit` below.
-      await emitJson({ app: crud.app, config: absolutePath, multiTenant, crud, rls, hardFailures });
+      await emitJson({ app: crud.app, config: absolutePath, multiTenant, crud, rls, referenceShapes, hardFailures });
     } else {
       this.log(formatReport(crud));
       if (rls) this.log(formatRlsReport(rls));
+      if (referenceShapes) this.log(formatReferenceShapeReport(referenceShapes));
       this.log('');
       this.log(
         hardFailures > 0

packages/cli/test/verify-reference-shapes.test.ts

@@ -0,0 +1,130 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * PIN — `os verify --reference-shapes` runs the reference-value shape family
+ * (ADR-0104 D1) and counts what it finds in the exit status.
+ *
+ * ## What is pinned
+ *
+ *   - without the flag the run is what it was: no `referenceShapes` key in the
+ *     `--json` document, and the exit status reads CRUD (and RLS) only;
+ *   - with the flag, on a stack whose engine refuses wrong-shaped references
+ *     (a fresh datastore is born strict), every client-writable reference
+ *     field is `proven`, `hardFailures` stays 0 and the run exits 0;
+ *   - with the flag, on the same metadata under the posture that admits
+ *     wrong shapes (`OS_ALLOW_LAX_VALUE_SHAPES=1`, ADR-0104's declared escape
+ *     hatch), each field is reported with holes, `hardFailures` counts them and
+ *     the run exits 1 — the negative control, through the CLI.
+ *
+ * Spawned rather than run in-process: the exit status is the contract a CI
+ * gate reads, and only a real process has one.
+ */
+
+import { describe, expect, it, beforeAll, afterAll } from 'vitest';
+import { spawnSync } from 'node:child_process';
+import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { CLI, TSX, childEnv } from './helpers/serve-process.js';
+import { defineStackSource, linkSpec } from './helpers/define-stack-fixture.js';
+
+const STACK = {
+  manifest: {
+    id: 'com.example.verify-reference-shapes',
+    namespace: 'vrs',
+    version: '1.0.0',
+    name: 'Verify Reference Shapes',
+    type: 'app',
+    engines: { protocol: '^18' },
+  },
+  objects: [
+    {
+      name: 'vrs_account',
+      label: 'Account',
+      pluralLabel: 'Accounts',
+      sharingModel: 'private',
+      fields: { name: { type: 'text', label: 'Name', required: true } },
+    },
+    {
+      name: 'vrs_contact',
+      label: 'Contact',
+      pluralLabel: 'Contacts',
+      sharingModel: 'private',
+      fields: {
+        name: { type: 'text', label: 'Name', required: true },
+        account: { type: 'lookup', reference: 'vrs_account', label: 'Account' },
+      },
+    },
+  ],
+};
+
+interface Run {
+  status: number | null;
+  stdout: string;
+  stderr: string;
+}
+
+interface Doc {
+  hardFailures?: number;
+  referenceShapes?: {
+    results: Array<{ object: string; field: string; status: string }>;
+    summary: { proven: number; holes: number; hardFailures: number };
+  };
+}
+
+function runCli(dir: string, args: string[], env: Record<string, string> = {}): Run {
+  // Through tsx, so the child runs this checkout's `src/`.
+  const r = spawnSync(TSX, [CLI, ...args], {
+    cwd: dir,
+    encoding: 'utf8',
+    env: childEnv({ NO_COLOR: '1', OS_ALLOW_LAX_VALUE_SHAPES: undefined, ...env }),
+    maxBuffer: 64 * 1024 * 1024,
+  });
+  return { status: r.status, stdout: r.stdout ?? '', stderr: r.stderr ?? '' };
+}
+
+const docOf = (run: Run): Doc => JSON.parse(run.stdout) as Doc;
+
+describe('os verify --reference-shapes', () => {
+  let dir: string;
+  let plain: Run;
+  let strict: Run;
+  let lax: Run;
+
+  beforeAll(() => {
+    dir = mkdtempSync(join(tmpdir(), 'os-verify-reference-shapes-'));
+    writeFileSync(join(dir, 'objectstack.config.mjs'), defineStackSource(STACK));
+    linkSpec(dir);
+    plain = runCli(dir, ['verify', '--json']);
+    strict = runCli(dir, ['verify', '--reference-shapes', '--json']);
+    lax = runCli(dir, ['verify', '--reference-shapes', '--json'], { OS_ALLOW_LAX_VALUE_SHAPES: '1' });
+  }, 480_000);
+
+  afterAll(() => {
+    if (dir) rmSync(dir, { recursive: true, force: true });
+  });
+
+  it('leaves the run unchanged without the flag', () => {
+    expect(plain.status, `${plain.stdout}\n${plain.stderr}`).toBe(0);
+    const doc = docOf(plain);
+    expect('referenceShapes' in doc).toBe(false);
+    expect(doc.hardFailures).toBe(0);
+  });
+
+  it('proves the reference field on a strict stack, and exits 0', () => {
+    expect(strict.status, `${strict.stdout}\n${strict.stderr}`).toBe(0);
+    const doc = docOf(strict);
+    expect(doc.referenceShapes?.results).toEqual([
+      expect.objectContaining({ object: 'vrs_contact', field: 'account', status: 'proven' }),
+    ]);
+    expect(doc.referenceShapes?.summary).toMatchObject({ proven: 1, holes: 0, hardFailures: 0 });
+    expect(doc.hardFailures).toBe(0);
+  });
+
+  it('NEGATIVE CONTROL: reports the hole on a posture that admits wrong shapes, counts it, and exits 1', () => {
+    expect(lax.status, `${lax.stdout}\n${lax.stderr}`).toBe(1);
+    const doc = docOf(lax);
+    expect(doc.referenceShapes?.summary).toMatchObject({ proven: 0, holes: 1, hardFailures: 1 });
+    expect(doc.hardFailures).toBe(1);
+  });
+});

packages/core/src/security/second-object-read-exposure.pin.test.ts

@@ -221,6 +221,14 @@ const CALLERS: Record<string, Classification> = {
     kind: 'not-a-served-read',
     reason: 'validation-rule analysis and evaluation on the write path; the verdict is pass/fail and no target field is served',
   },
+  'packages/verify/src/reference-shape.ts': {
+    kind: 'not-a-served-read',
+    reason:
+      "the verification harness's reference-value shape family: the target only picks, from the ids of records the " +
+      'family itself created through the REST door, the id its embedded-record probe carries. It writes probe values ' +
+      "to the SOURCE object (the REST data routes, which judge that object's exposure, and the engine's system write) " +
+      'and reads no record of the target',
+  },
 };
 
 const SCAN_TIMEOUT_MS = 60_000;