Repository navigation
fix(metadata-protocol): a hydrated view expansion carries its container's tenant marker, so an unscoped kernel answers an expanded view as env_local does (#21511) - #21603
Conversation
…er's tenant marker On an unscoped kernel, registry hydration registered each expansion of a stored environment-wide view container without the tenant-authorship marker its container carries. An expansion of a package-bound container then read as a code artifact through the registry's bare-key fallback: the by-name read answered resettable and the layers read answered the expansion as its code layer, where env_local answered neither. hydrateExpandedViewItems now asks expandRuntimeViewContainer for tenant-authored expansions: stateTenantAuthorship first, then the expansion's own artifact envelope, the order the container gets. The registry-free reads are unchanged. Claude-Session: https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi Co-authored-by: Claude <noreply@anthropic.com>
…pansion-tenant-marker
📓 Docs Drift CheckThis PR changes 1 package(s): 1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 2 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 11 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 50b8fc75a83028a9624ad1de6b39e7592a69eab3 && git checkout 50b8fc75a83028a9624ad1de6b39e7592a69eab3
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin b610eabf721672ab621dd1f45e2d1dafbf76a740 09033d877d6160e270b03192581264d80c2fad5d && git checkout -B drift-repro b610eabf721672ab621dd1f45e2d1dafbf76a740 && git merge --no-ff 09033d877d6160e270b03192581264d80c2fad5d
node scripts/docs-audit/affected-docs.mjs --json b610eabf721672ab621dd1f45e2d1dafbf76a740
|
…rch narrowing, owned by the door and called by the reader seam; cross-field and deep family reads refused (objectstack-ai#21619) Fixes objectstack-ai#21544 Clause-②: yes (narrowing) The generic data door now owns the stored-metadata family's one filter-field collector and its one default-search narrowing. Both are exported module functions, and the door and the in-process reader-context seam (`@objectstack/runtime`) call the same two. The card's measure-first step found a door gap: two filter shapes read a family column at the generic data door without the family's refusal ever seeing them. Per the ruling, the door therefore adopts the stricter collector in this landing, and under the card's raise rule the card is p1. Measurement below. ## The door-gap measurement (measured before any collector was chosen) Measured at `b610eabf72` with a composed kernel (ObjectQL, a default datasource, HTTP server, platform objects, auth, security, sharing, REST and the dispatcher) and the administrator signed in. The family credential was stored by the production writer (`PUT /meta/datasource/NAME`). Every request went through three doors: `POST /api/v1/data/OBJECT/query` (HTTP), the in-process door (`protocol.findData`) and the seam (`serveStoredMetadataReadsThrough` over the engine). Both family tables were covered, on `driver-sqlite-wasm` and on `driver-memory`. | shape at the door | request (body of `POST /api/v1/data/sys_metadata/query`) | column read | answer before this PR | |---|---|---|---| | positive control: direct reference | `{"where":{"metadata":{"$ne":"zzz"}}}` | `metadata` | 400 `INVALID_FIELD` (family refusal) | | **cross-field comparand** | `{"where":{"type":"datasource","name":{"$ne":{"$field":"metadata"}}}}` | `metadata` | **200, 1 row**; the `$eq` twin answers 0 rows; `type` `$lt` `$field metadata` answers every row and `$gt` answers none (SQL). The family column is evaluated. | | **cross-field comparand** | same, with `checksum` (and `previous_checksum` / `change_note` on `sys_metadata_history`) | the hash column / the note | **200**, partitioned the same way (SQL) | | **cross-field comparand in an aggregation filter** | `{"groupBy":["type"],"aggregations":[{"function":"count","alias":"n","filter":{"name":{"$ne":{"$field":"metadata"}}}}]}` | `metadata` | **200**, `n` = 1 (the `$eq` twin gives `n` = 0) | | **direct predicate below the depth backstop** | the body filter `{"metadata":{"$contains":"STORED_CREDENTIAL"}}` wrapped in 33 nested one-armed `$and` levels | `metadata` | **200, the row** for the stored credential, **0 rows** for a wrong guess, **the row** for a prefix. A credential oracle on both drivers, both tables, over HTTP and in-process, in `where` and in an aggregation filter. At 32 levels the same filter is refused (control). | | dotted key | `{"where":{"metadata.x":"zzz"}}` | none | 400 `INVALID_FIELD` from the door's dotted-path verdict (`param: where`). Moot: never evaluated. | | `having` | `having` naming `metadata` / `checksum`, or `{ "$field": … }` to one | none | 400 `INVALID_FILTER`: the engine judges every `having` name against the aggregated row's columns. Grouping by a family column is refused, and `min` / `max` of the `textarea` / `text` family columns is refused by the engine's aggregate-field-type door. Moot. | The two bold shapes are the door gap. The cross-field one is a SQL-driver reach. On `driver-memory` the reference is not resolved at all (see Acceptance notes). The depth one is a reach on both drivers. ## What this changes ### Public surface: `@objectstack/metadata-protocol` (additive, `minor`) - `collectStoredMetadataFilterFields(object, query): string[]` is the family's one filter-field collector. It returns every column a read query's filters read, across `where`, the engine's `filter` alias and each `aggregations[i].filter`. That means each key's head plus each cross-field `{ $field }` comparand's head, at any depth: the walk is iterative and cycle-safe, with no depth backstop. Anything beneath an unrecognised `$` key is read as a condition, and a `FilterArray` is lowered first. It is `[]` outside the family. It does **not** read `having`, whose names are the aggregated row's. - `narrowStoredMetadataSearch(object, query, schema, wireSpelling?): string[] | undefined` is the family's one default-search narrowing. It moved, unchanged in its answers, from the door's private method of the same name: - an explicit list naming the body or a hash column is refused under the caller's wire spelling; - a default search returns the narrowed field set for the caller to run as `searchFields`; - an emptied set is refused; - `undefined` means "run as is". - `type StoredMetadataSearchSchema` is the definition slice the narrowing reads. ### Accept-set changes: the generic data door, `sys_metadata` / `sys_metadata_history` only This applies to `GET /api/v1/data/:object`, `POST /api/v1/data/:object/query` and in-process `findData`. 1. A cross-field comparand naming the body, `checksum`, `previous_checksum` or `change_note` changes from **200 to 400 `INVALID_FIELD`**, with `param: filter` and `field` set to the column, before the engine is asked. This covers `where`, `$not` and an aggregation filter. 2. A family-column predicate, key or comparand, nested more than 32 combinators deep changes from **200 to 400 `INVALID_FIELD`**, in the same envelope. 3. Some shapes were refused before and stay refused, with a different refusal, in-process only. An unrecognised `$` key wrapping a family column, an array-form aggregation filter naming one, and a malformed `$field` reference to one were the engine's `INVALID_FILTER`. They now get the family's `INVALID_FIELD`. Over HTTP, the array-form aggregation filter is still refused earlier by REST validation. Unchanged: - **Every other object.** The collector answers `[]` outside the family, so the door collects nothing there. - **Dotted keys.** The dotted-path verdict still answers first. - **Every search answer** (explicit list, default narrowing, emptied set). These are pinned identical at the door and the seam. ### `@objectstack/runtime` (`patch`) The seam (`stored-metadata-reader-seam.ts`) changes as follows: - `narrowFamilySearch` is **deleted**, along with the `filterHeadFields` wrapper and the `@objectstack/plugin-security` `collectConditionFields` import. The seam now calls the door's two exports. - `count` runs the query the guard returns (H2). The seam's accept set is unchanged: everything it refused before it still refuses. Two seam refusals change code, from the engine's `INVALID_FILTER` to the family's `INVALID_FIELD`: an unrecognised `$` key wrapping a family column, and a malformed `$field` reference to one. ## Readings on the dispatch's hypotheses - **H1 (dotted / cross-field), confirmed in part.** The dotted half is moot at the door: the dotted-path verdict refuses it. The cross-field half is a measured gap. The depth backstop was a second gap the measurement found. - **H2 (`count` discards the guard's return), confirmed and corrected.** `count` now consumes the guard's return. The pin: a default search through `count` arrives narrowed. - **H3 (`having`), moot.** See the table. The collector deliberately does not read `having`: an aggregation alias spelled like a family column is a legitimate count, and a parity control pins it as run. - **H4 (the door's own answers unchanged), confirmed.** - The existing door suites pass unchanged: `objectstack-ai#21207` search, hash and note, `objectstack-ai#21120` body. - The parity table pins explicit list, default search and emptied set identical at the door and the seam. - **H5 (other `collectConditionFields` readers).** After this PR its one reader is `@objectstack/plugin-security`'s own FLS predicate guard (`collectQueryFields` / `assertReadableQueryFields`). That guard does not judge the family. `@objectstack/runtime` still depends on `@objectstack/plugin-security` through `security/resolve-execution-context.ts`. No `plugin-security` edit. ## Tests - New in `packages/metadata-protocol/src/protocol.data-door-stored-metadata-filter-reads.test.ts`: - collector cases (16 shapes on both tables, plus non-family, cycle and shared-node pins); - narrowing cases; - door pins: 6 gap shapes × 5 family columns, each refused with `code` / `status` / `param` / `field` / `object` and the engine never asked; - scalar-column and non-family controls. - New in `packages/runtime/src/stored-metadata-reader-seam.test.ts`: the door / seam parity table. It is one table of 24 cases: 7 search cases (explicit list ×4, default ×2, emptied); 14 collector cases (direct, dotted key ×2, cross-field comparand ×3, dotted reference, list reference, `$not`, unknown `$` key, depth 33 ×2, aggregation filter ×2); and 3 controls. Each case runs through `findData` and through the seam, and the two outcomes must be equal. There is also a narrowed-default pin and the `count` pin. - `pnpm --filter @objectstack/metadata-protocol exec vitest run`: 208 files passed, 3 skipped; 3266 tests passed, 19 skipped (at `5513190ca5`, after merging `main`). - `pnpm --filter @objectstack/runtime exec vitest run --project local`: 318 files passed; 4514 passed, 19 skipped (at `5513190ca5`). `--project repo`: 3 files, 751 passed (at `9be38c5987`). - At the final head `0ac5749662`: - the three family door suites: 99 passed; - the seam, reader-contexts, body-writes and boundary suites: 86 passed. - `typecheck`: both packages green (at `5513190ca5`). `--listFiles` confirms both new tests are inside each package's tsc program. ## Reverse verification (both at `be99ceee6a`, through `scripts/ablation-replace.mjs`, mutation and restore proven on disk) 1. **The seam reverted to its own narrowing and collector.** The file was swapped to its base blob `27efc3412999`; on disk, `narrowFamilySearch` = 1, `collectConditionFields` = 3 and the new collector = 0. Result: **2 red** (the parity case for an unrecognised `$` key wrapping a body filter, and the `count` pin) and 41 green. Restored with `git checkout HEAD -- PATH`: blob == HEAD `51fe56bb5e73`, `git diff HEAD` empty. 2. **The door's collector reverted** to the ingress key collector expression. On disk the ablation marker = 1 and the new call = 0. Result: **exactly the 30 door-gap pins red** (6 shapes × 5 columns); the 69 others stayed green (collector and narrowing units, controls, the existing `objectstack-ai#21207` / `objectstack-ai#21120` door suites). Restored: blob == HEAD `fa64d2b26cd9`, `git diff HEAD` empty. Both are src-resolved: each subject is imported by relative path, so no `dist/` leg was needed. ## Gates (at `0ac5749662`) - `node scripts/pm/dispatch-gates.mjs --commands` derived 64 families; all 64 were run and exited 0. Reconciled with `--ran`: 64 run, 0 NOT-MEASURED, 0 UNRUN. - Two needed a step first: - `check:dual-build-cjs-loads` printed `PREREQUISITE NOT MET` before a full `turbo build`, then exited 0. - `check:engine-double-contract` flagged the new door doubles' `findOne`. `findData` never reads `findOne`, so the doubles carry none, and the pinned ledger is untouched. - Lint is a proven narrowing. ESLint ran with `--no-inline-config --format json` on the 6 touched TS files at `0ac5749662`: 6 files, 0 errors, 0 warnings. The checked population is read from `eslint.config.mjs` (`--print-config` per file). Type-aware linting is not enabled anywhere: no `parserOptions.project` or `projectService`, `project=null` per file. So this diff cannot move any untouched file's verdict. ## File surface - Declared: - the data door's read regions of `packages/metadata-protocol/src/protocol.ts`. The hydration region is untouched; PR objectstack-ai#21603 was merged in from `main`. - `packages/metadata-protocol/src/index.ts` - `packages/runtime/src/stored-metadata-reader-seam.ts` - tests in both packages - `.changeset/21544-door-narrowing-export.md` - Beyond the claim's list, one sentence of comment in `packages/metadata-protocol/src/metadata-redaction.ts`. It is the `storedMetadataBodyPredicateRefusal` docblock's parenthetical, which named the old collector as the source of `filterFields` and would have been false after this change. No code. - The changeset carries `Clause-②: yes (narrowing)` and an ADR-0087 `not-required (no-migration-prescription)` disposition marker. `check:adr-0087-registration` reads it. ## Acceptance notes (observations, not filed) - **`driver-memory` does not resolve a cross-field reference in `where`.** It compares against the literal reference object, so on a non-family object `{ "name": { "$eq": { "$field": "name" } } }` answered 0 rows (SQL: every row). It was measured at `b610eabf72` through the generic data door on a memory-backed composition. Public reach is not measured after `9a4182a752` (the in-memory engine is no longer a boot store), so it is not filed. Carrier: none. - **`count` / `count_distinct` over a family column is still served,** at the door and the seam alike. It discloses equality only, which the keyed content hash already serves per row. - **The ingress gate's `collectFilterFieldKeys` keeps its 32-level backstop for the existence question.** That question is not the family's. The backstop's reach on an unknown field nested below it is an unmeasured inference. --- _Generated by [Claude Code](https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Fixes #21511
Clause-②: no
What this changes
On an unscoped (control-plane) kernel, registry hydration registers every view a stored environment-wide view container expands, each under its own name.
hydrateOverlayIntoRegistryregisters the container with the tenant-authorship marker (stateTenantAuthorship, ADR-0010_provenance: 'org'), andhydrateExpandedViewItemsregistered the expansions without it. An expansion of a package-bound container therefore sat under its bare name, wearing that package's_packageIdand no marker, andSchemaRegistry.getArtifactItem's bare-key fallback took it for a view the package ships.This PR implements triage's ruling (comment 5964348889): each expansion inherits its container's authorship.
hydrateExpandedViewItemsasksexpandRuntimeViewContainerfor tenant-authored expansions (tenantAuthored: true).expandRuntimeViewContainerappliesstateTenantAuthorshipto each expansion BEFORE that expansion's own artifact envelope is merged over it. This is the order the container gets (mergeArtifactProtection(stateTenantAuthorship(data), envelope)), so where the container's own package ships a view of that name, the artifact's_provenance,_packageIdand_lockstill win (ADR-0010 §3.3).expandRuntimeViewContainer(the list read's expansion pass and the by-name read's step 1b, throughexpandStoredViewContainers) pass no such option and serve exactly what they served before.isArtifactBacked(whichresettablereads) and the layered read'scodearm already askisTenantAuthored; they now get the marker to read.Measured before the change
At base
a7ab047c, with the #21508 harness (showcaseHarness) and the card's probe (a containeros_qa_probeonshowcase_taskwithlistViews.in_progress, read asshowcase_task.os_qa_probe.in_progress):getMetaItem(...).resettablegetMetaItemLayered(...).codeenv_localcom.example.repairassets)falsenullenv_localfalsenulltruefalseAt the base the registry held the container as
{ name: 'os_qa_probe', _provenance: 'org' }and the expansion as{ name: 'showcase_task.os_qa_probe.in_progress', _packageId: 'com.example.repairassets' }, with no_provenance. For the package-less arm the expansion carried no_packageId, soresettablewas alreadyfalse. Itscodelayer was still wrong, through the layered read's runtime-onlygetItemarm, which drops only tenant-marked entries.After the change, both kernels give
env_local's answer for every member kind in both arms.The save door (ruling: no save-door rule change)
The fix changes what
isArtifactBackedanswers for an expanded name on the unscoped kernel, and the save door reads that predicate. The door's acceptance is pinned rather than assumed. After the container is saved, a write by the expanded name is accepted on both kernels and in both arms, stored once in the container's scope, and that row then answers the name on the by-name read and on the object door. The outcome is identical across the two kernels. The same probe was accepted on all four kernel and arm combinations at the base. Under reverse verification leg 1 below, the save-door pins stay green, so the acceptance does not move with the fix.Tests
packages/metadata-protocol/src/view-container-runtime-expansion.test.tsgets a new describe block,#21511 an expanded view of a stored container answers as tenant-authored on both kernels, with 15 tests in #21508's harness. It covers the package-bound and package-less environment-wide arms. Hydration never registers an organization-scoped row.codelayer on either kernel, and the unscoped kernel's whole answer equalsenv_local's (resettable,editable,deletable,lock,provenance,packageId,code). That is 10 tests.isCodeArtifactBody).showcase_taskcontainer. Its expansions,showcase_task.defaultandshowcase_task.in_progress, stay resettable with the packagedcodelayer on both kernels, and on the unscoped kernel the registered expansion keeps the artifact's envelope (_provenance: 'package',_packageId: com.example.showcase) over the marker.Reverse verification (both runs recorded)
Each run starts from the committed fix, and each leg restores with
git checkout HEAD -- ABSOLUTE_PATH. Each restore is proven by blob hash equal to the HEAD blob, an emptygit diff HEAD, and a cleangit status, all inside a script armed withtrap restore EXIT INT TERM. The subject is imported from source (./index.js), so nodist/is involved.Run 1, at
e8e00609(the fix commit, before merging main):protocol.tswas reverted to the base blob3ac2573f(git restore --source=a7ab047c). The landing was proven by the on-disk blob equalling the base blob; the base blob carries 0 occurrences oftenantAuthored: true, and HEAD carries 1. Result: 12 failed, 132 passed (144). The 10resettable/codepins failed withunscoped: no package ships it: expected true to be false(package-bound) andunscoped: no artifact, so no code layer: expected {…} to be null(package-less). The 2 marker pins failed withexpected undefined to be 'org'. The 2 save-door pins and the CONTROL stayed green.scripts/ablation-replace.mjs, with anchor 1→0, replacement 0→1, and blobb106f11e→5d6e6263. Result: 1 failed, 143 passed. Only the CONTROL failed:showcase_task.default: the artifact's envelope is merged over the marker, not under it: expected { _provenance: 'org' } … { _provenance: 'package' }.Run 2, at
09033d87(after mergingorigin/main6c5697df, which includes the landed #21545 aseb9ef791):24cd0629(6c5697df): 12 failed, 132 passed, with the same 12 tests and the same messages.Verification at
09033d87pnpm --filter @objectstack/metadata-protocol exec vitest run --maxWorkers=2: Test Files 207 passed, 3 skipped (210); Tests 3207 passed, 19 skipped (3226);VERDICT command-exit 0.pnpm --filter @objectstack/metadata-protocol typecheck(tsc --noEmit) exited 0. Its--listFilesreaches 210 of the package's 210 test files, including the edited test file.eslint --no-inline-config --format jsonover the two edited.tsfiles gives 2 files, 0 errors and 0 warnings. The changeset.mdis outside everyfilesglob ofeslint.config.mjs.--print-configshows noparserOptions.projectorprojectService(type-aware linting is not enabled), so this diff cannot move the verdict on any untouched file. The fullpnpm lintis CI's.node scripts/pm/dispatch-gates.mjs --commands(no paths) derived 64 families for this change set. 63 exited 0.pnpm check:dual-build-cjs-loadsis NOT MEASURED: it exited 3 (PREREQUISITE NOT MET), because it needs every package'sdist/and 67 had none. This diff changes no exports, entry points or build config.--ranreconciliation: 64 accounted, 63 run, 1 NOT MEASURED. Two first runs were prerequisite misses and were re-run green after the prerequisite was met.check-plugin-teardown-shape --self-testneeded its pinned fixture commit fetched into the shallow clone.check:lean-entry-closureneeded@objectstack/objectqlbuilt.Region and surface
protocol.tshunks: thestateTenantAuthorshipdocblock (its "ONE caller" sentence now names both callers),expandRuntimeViewContainer's options type and its merge line, andhydrateExpandedViewItems' call and docblock. The claim names thehydrateExpandedViewItemsregion.expandRuntimeViewContainersits in the same hydration block, and the stamp must go there so that it precedes each expansion's own envelope (the order above), without a second copy of the envelope rule. That is the one widening of the region, declared here. The save door and the data door's read region are not edited. #21545's hunks (landed aseb9ef791, merged here) are disjoint from these.Acceptance notes
codelayer was wrong at the base;resettablewas alreadyfalse(table above). Both values are pinned now.isArtifactBackednow answersfalse, as onenv_local. Reading the save path (not separately measured), the write intent it derives is therefore the runtime-only one rather than the artifact-override one. The ruling expects this ("layered by the corrected predicate on both kernels"). Acceptance is unchanged, as measured above./layersREST routes call.origin/mainat6c5697df. One later main commit (f6b75208, spec conformance-case notes and a lint test) is not merged. It touches none of this PR's files.Changeset:
.changeset/21511-expansion-tenant-marker.md,patchfor@objectstack/metadata-protocol.Generated by Claude Code