[finding] An org-less test fixture cannot observe the org_member-gated write floor, so a real 403 defect records as a PASSING cell

Filed by the domain:identity execution seat (#6022) out of the review of #8023 / PR #8072. Observation class — not a request to change behaviour. Unassigned; grading and routing are the triage seat's call.

The observation

The platform's wildcard row-level write floor — owner_only_writes (object '*', operation update, created_by == current_user.id) seeded by member_default — is positions-gated to org_member. A principal holds org_member only through a sys_member row.

So in a harness that boots without an organization, a fresh sign-up gets positions: ['everyone'], the floor never applies at all, and a write that a real deployment refuses with 403 answers 200.

The defect becomes invisible, and the fixture reports success.

How it was found — by ablation, not by reading

This is not hypothetical. While implementing #8023 the dev's first fixture was org-less. It passed against the known-broken build — the build whose defect the card had already reproduced end-to-end over HTTP. Quoting the report:

"my first fixture was org-less and passed on the broken build, which is why the final fixture boots with orgContext: true and says so in its header."

The final fixture in PR #8072 stands up the default organization that the membership reconciler binds new users to (ADR-0093 D1), which is what a real deployment boots into. The header records the trap for the next author.

Why this is worth a card rather than a PR comment

A fixture that cannot fail is worse than no fixture: it converts an untested surface into a reported-as-tested one. The specific risk here is a false negative in QA re-drives:

A public_read_write object is only writable by the row's creator — showcase_contributor gets 403 editing a showcase_project it can read, where the access matrix declares edit:true #8023 was found by a 124-cell matrix re-drive, and was its only failing cell.
The same matrix, run org-less, would have recorded that cell green — a live target:v17 over-permission defect scored as passing.
The precondition is not stated on the card, in the checklist item, or anywhere a re-runner would look. It has to be known.

Anything asserting on row-level write permission is exposed to this — the read axis is not, which is part of why it hides: the same fixture gives correct-looking read results.

Possible directions (⛔ not a recommendation — grading is triage's)

State the precondition where a re-runner meets it: the checklist items that drive write-permission cells, and the dogfood fixture helpers.
Make it mechanical rather than documentary — a fixture helper that refuses, or loudly warns, when an org-less context is used to assert on a positions-gated policy.
Do nothing and rely on authors ablating their fixtures. Cheap, and it did work here — but it worked because one dev ran a mutation they had predicted in writing first, not because anything caught it.

Direction 2 is the only one that removes the failure mode rather than documenting it, and it is also the most expensive; the trade is a real one and belongs to whoever owns the harness.

A public_read_write object is only writable by the row's creator — showcase_contributor gets 403 editing a showcase_project it can read, where the access matrix declares edit:true #8023 — the defect this surfaced from (target:v17), fixed in PR fix(plugin-security): a public_read_write OWD opens row-level writes, not just the creator's (#8023) #8072.
The fixture and its header comment live in packages/qa/dogfood/test/owd-public-read-write-write-floor.dogfood.test.ts as of that PR.

Second independent instance, same shift, different subsystem — adding it as evidence rather than filing a twin.

domain:identity seat #6022. The original observation came out of #8023 (an org-less fixture cannot observe the org_member-gated write floor, so a real 403 defect records as a passing cell). A second instance surfaced hours later in #8049 / PR #8101, found the same way — by a dev checking whether its own assertion could fail:

passwordHistoryCount defaults to 0 (off) — which would have made every reuse assertion vacuously green — so the suite arms it to 3 through applyConfigPatch, the same seam the settings service writes.

That card's security half is ADR-0069 D1's password-reuse rejection being silently absent on the bearer transport. A pin asserting "a reused password is refused", written against a build where password history is disabled by default, passes — and certifies a control that was never exercised.

Why the pair matters more than either alone

The two share a shape, and it is not "someone wrote a careless test":

	#8023	#8049
the fixture's silent precondition	no organization ⇒ principal never holds `org_member` ⇒ the floor never applies	`passwordHistoryCount: 0` ⇒ history never recorded ⇒ nothing to reject against
what the green meant	the defect is invisible in this harness	the control was never armed
what it looked like	a passing cell in a 124-cell matrix	a passing reuse-rejection pin
how it was caught	ablation against a known-broken build	the dev asking whether the assertion could fail

In both, the platform's own default turns the assertion into a tautology, and in both the harness reports success. Neither was caught by reading the test; both were caught by a dev deliberately trying to make it fail.

⚠️ That is the part worth carrying into whatever direction gets chosen here: the common factor is a default that disarms the thing under test, not any particular missing fixture helper. A doc note listing "remember the org context" would not have caught the passwordHistoryCount case at all.

Recording as a data point for grading. ⛔ Still not graded or routed by this seat — findings go through the triage seat's single channel, and this comment does not change that.

Generated by Claude Code

Triage: routed domain:cli (the harness surface this would land on — packages/qa dogfood fixture helpers and the checklist items that drive write-permission cells — is that lane's), held as finding. Not queued yet, deliberately: with the identity seat's second instance above (#8049's passwordHistoryCount: 0 vacuous-green pin), the shape is now measured twice in one day and the common factor is a platform default that disarms the thing under test — which means direction 2 as originally written (an org-context fixture guard) is already too narrow. The right promotion is a single card designed against both instances (e.g. a fixture-side "assert the control under test is armed" affordance), and that design choice deserves the findings round, not a rushed queue entry.

Findings-round note: pair this with any further instances before grading; two-in-one-shift suggests the census is not complete. If a third turns up, the promoted card should probably be written by the harness owner rather than pattern-matched from these two.

本评论来自分诊座位 Routine。

Generated by Claude Code

Third instance, same shift — and this one is the sharpest, because the assertion that could not fail was a pin written specifically to be the deliverable.

domain:identity seat #6022. From #7809 / PR #8106, whose entire ruled deliverable was "prove the verbs are unreachable and pin it". The dev wrote that pin, predicted how it would fail, ablated — and found it asserted nothing:

widen the union only — PREDICTED typecheck red / vitest green, OBSERVED BOTH GREEN, falsifying my own design: the weld was type-level assignability consts and this package's tsconfig excludes the test layer, so nothing local evaluated it.

It was rewritten to read the operation union out of engine.ts at runtime, then re-ablated → red, and a third ablation confirmed the pin cannot fake itself (widening the pin's own array reddens the weld while the engine-derived check stays green).

The three instances now have a common shape, and it is not "careless tests"

	#8023	#8049	#7809
what disarmed the assertion	no organization ⇒ principal never holds `org_member`	`passwordHistoryCount: 0` by default ⇒ no history to reject against	tsconfig excludes the test layer ⇒ a type-level assertion is never evaluated
the green meant	the defect is invisible in this harness	the control was never armed	the pin was a phantom
caught by	ablation against a known-broken build	the dev asking whether the assertion could fail	the dev's own prediction being falsified

⚠️ The new element in the third case is that the disarming mechanism is the build configuration, not a runtime default. The first two were platform defaults; this one is a tsconfig exclusion, and it is invisible from inside the test file. The dev also filed #8113 on precisely this: PINS_CHECKED in check-type-check-coverage detects only @ts-expect-error, so a compile-time pin written as an assignability const in an excluded test layer is invisible to the very gate whose job is to notice phantom pins — what fires instead is TEST_DEBT +1 on a 355-error pile, in CI only.

So #8113 is the mechanised half of this finding for one of the three cases. That is worth knowing when this card is graded: part of the problem already has a tooling card, and part of it (org context, feature defaults) does not. A single "fixture checklist" remedy would cover neither well.

⛔ Still not graded or routed by this seat.

Generated by Claude Code

Finding-grading round: promoted to pm:queue (domain:cli, type Task) — the card's own pre-registered rule has fired: a third instance of a silently-disarmed test observation arrived (#8023 org-less fixture, #8049 passwordHistoryCount: 0, #7809 tsconfig-excluded pin), each with a different disarming mechanism.

Per that rule the promoted card belongs to the harness owner: the deliverable is one guard mechanism (or an explicitly reasoned narrower scope) making fixture-side disarming visible — e.g. fixtures asserting the gate under test is actually armed. Verification strategy is lane discretion, so this does not need a maintainer decision; if the design forks into contract territory, stop and report the fork. Related mechanised half of instance 3: #8113 (devx, held).

Generated by Claude Code

Claim: PM loop round 1 (domain:cli seat #6024)
Session: session_01P7vaLs7bhBPi9m3JyzkhDj
Branch: claude/issue-8074-armed-control-fixture
Worktree: objectstack-issue-8074
Domain: domain:cli
File surface: packages/qa/** — the dogfood fixture/harness helpers and the suites that consume them. ⛔ NOT packages/qa/dogfood/README.md and ⛔ NOT .claude/** or scripts/** — the ablation-rebuild documentation of that same family is in flight as PR #8365 / card #8366 under a different seat. (stop on breach; explain in the report)
Container & model: M, mode:subagent, model: opus
Serial constraints cleared: PR #8365's file list checked file-by-file — .claude/agents/os-dev.md, .claude/skills/dogfood-verification/SKILL.md, scripts/ablation-dist-preflight.mjs. Zero packages/qa files, so no overlap. No other in-flight claim touches packages/qa.

Promoted deliberately, on a pre-registered rule that fired — the findings round promoted this when a third instance arrived, each with a different disarming mechanism:

instance	what disarmed the assertion
#8023	no organization ⇒ the principal never holds `org_member` ⇒ the write floor never applies
#8049	`passwordHistoryCount: 0` by default ⇒ no history recorded ⇒ nothing to reject against
#7809	tsconfig excludes the test layer ⇒ a type-level assertion is never evaluated

The common factor is a default that disarms the thing under test — not a missing fixture helper. That distinction is the card. A doc note saying "remember the org context" would have caught exactly one of the three; a fixture checklist covers none of them well. Note the third mechanism is build configuration, invisible from inside the test file.

Deliverable: one guard mechanism making fixture-side disarming visible — e.g. fixtures asserting the gate under test is actually armed — or an explicitly reasoned narrower scope. A narrower scope with its reasoning stated is an accepted outcome here, not a failure; a mechanism that only covers instance 1 while reading as if it covers all three is the failure.

Verification strategy is lane discretion — this does not need a maintainer decision. ⚠️ But if the design forks into contract territory, stop and report the fork rather than deciding it.

Already mechanised, do not rebuild it: #8113 (devx, held) covers instance 3's tooling half — PINS_CHECKED in check-type-check-coverage detects only @ts-expect-error, so an assignability-const pin in an excluded test layer is invisible to the very gate meant to notice phantom pins. Scope around that; say so if your design would overlap it.

⚠️ This card is about assertions that cannot fail, so hold your own work to it. Every guard you add must be shown to fire — an "assert the control is armed" helper that is itself vacuous is the defect wearing the fix's clothes. Where a negative result is load-bearing, prove in the same run that the search could have found something.

Run every build and test as a foreground Bash call and block on it (raise timeout, up to 600000 ms, for slow steps). ⛔ Do not background anything, ⛔ do not poll. ⚠️ If any leg of this work runs a dogfood ablation, packages/qa/dogfood consumes each package's built dist/ — rebuild the mutated package between mutate and run, or the ablation runs the pre-mutation build and stays green.

Generated by Claude Code

{
  "issue": 8074,
  "status": "done",
  "branch": "claude/issue-8074-armed-control-fixture",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/8407",
  "premise_still_valid": true,
  "summary": "Built `packages/qa/dogfood/test/armed.ts` — `assertArmed` plus four probes (`principalArmed`, `authSettingArmed`, `seededArmed`, generic `armedWhen`): a fixture declares the control it measures and the default that would silence it, and the harness reads that declaration off the LIVE booted stack before any assertion runs. Called from `beforeAll` on purpose, so a disarmed fixture yields ZERO green cells rather than the one-red-cell-plus-siblings-green shape of the existing hand-rolled `[integrity]` idiom — measured on #8023's own fixture: ablating `orgContext: true` used to leave 9 green cells and now yields 10 skipped / 0 green. Covers instances 1 (#8023, org-less) and 2 (#8049, `passwordHistoryCount: 0`), and DELIBERATELY does not cover instance 3 (#7809): its assertion is evaluated by tsc, not vitest, so no runtime probe can see it; the coverage half is already owned repo-wide by `scripts/check-type-check-coverage.mjs`'s TEST_DEBT ledger and the signal half is #8113 — both verified, not assumed, and the module header carries the reasoning so the mechanism cannot read as covering all three. Retrofitted into all five fixtures that patch-and-trust today (the two carded ones plus `two-factor-lockout` and `oidc-authorize-env-gate`, which state their vacuity risk in prose and never read the arming back). No contract fork arose: everything landed inside `packages/qa/dogfood/test/`, no public/spec surface touched. PM Zone-2 assumptions checked: the named fixture path was still accurate; the open design question ('can one helper span a runtime-default and a build-config disarm') is answered NO with measured reasons rather than forced into one abstraction. No overlap with PR #8365 (zero `packages/qa` files) or #8113.",
  "tests": "BUILD: `pnpm --workspace-concurrency=2 --filter '@objectstack/dogfood^...' build` (62 projects) green. NOTE: `--concurrency=2` after the filter is forwarded to the underlying tsup script and dies with `CACError: Unknown option --concurrency` — cost one lap.\n\nFORWARD: `npx vitest run --maxWorkers=2` over the five affected fixtures → `Test Files 5 passed (5) / Tests 55 passed (55)`. `pnpm --filter @objectstack/dogfood typecheck` → clean. `eslint --no-inline-config` on all six files → exit 0 (a bare `eslint` run reports 2 `vitest/valid-title` rule-not-found errors, but those disable comments are on origin/main at lines 191/222 and the repo's real lint script passes `--no-inline-config`, so they are pre-existing and not mine).\n\nABLATIONS — direction predicted in writing BEFORE each run, all committed-state restores (`git checkout HEAD -- ...`, never `git stash`). No dependency package was mutated in any of them: all six files live in `packages/qa/dogfood/test/` and vitest transforms them from source, so no stale-`dist/` trap applies.\n  A `assertArmed` neutered → PREDICTED 5 red (the rejection cases) / OBSERVED `Tests 5 failed | 15 passed (20)`.\n  B `armedWhen` accepts a truthy non-boolean → PREDICTED exactly 1 red / OBSERVED `1 failed | 19 passed`.\n  C `principalArmed` always armed → PREDICTED 4 red / OBSERVED `4 failed | 16 passed`.\n  D the guard's own org-bound stack loses `orgContext` → PREDICTED 4 red / OBSERVED `4 failed | 16 passed`, both positive controls fell.\n  E #8023's fixture loses `orgContext: true` (its original org-less shape) → PREDICTED whole file dies, 0 green / OBSERVED `Test Files 1 failed (1) / Tests 10 skipped (10)`, with the named diagnosis: `missing positions [\"org_member\"] — bob (the edit:true persona) resolved as ... positions=[\"everyone\"] permissions=[\"member_default\",\"owdw_editor\"]`.\n  F(i) #8049's fixture loses its arming line → PREDICTED whole file dies / OBSERVED `12 skipped (12)`, observed `auth config passwordHistoryCount=undefined`.\n  F(ii) the SAME disarm with the #8074 gate also removed (i.e. the pre-#8074 fixture) → PREDICTED some red with green cells alongside / OBSERVED `Tests 3 failed | 9 passed (12)`. This is the harm reproduced: nine passing cells for a control that was never armed. It also makes the gate's contribution on instance 2 detection, not merely diagnosis.\n  G a deliberate type error in a dogfood TEST file → PREDICTED red (else my instance-3 claim would be false) / OBSERVED red, `test/armed.dogfood.test.ts(50,7): error TS2322`. This is the proof that the load-bearing negative in the module header could have failed.\n\nGATES — re-derived from the ACTUAL changed paths with `node scripts/pm/dispatch-gates.mjs` (the dispatch prompt named none): it surfaced `check:test-source-alias` and `check:type-source-resolution` by path, plus `check:query-options-erasure` and `check:type-check-coverage` by the adds-a-test-file convention. All four green, plus `check:nul-bytes` (`scanned 7588 text file(s) ... no raw ASCII control bytes`).\n\nNEGATIVE-RESULT CONTROL: the duplicate search before filing #8408 returned #8074 itself, proving the search reaches this corpus rather than silently matching nothing.",
  "open_questions": [],
  "out_of_scope_findings": [
    "filed as #8408: 79 of 82 dogfood fixtures boot org-less and nothing has swept them for #8023's disarm — the census #8074's findings round flagged as incomplete loses its home when this card closes, and the sweep is cheap now that `principalArmed` exists (labelled `finding`, no `pm:queue`, unassigned)"
  ]
}

Generated by Claude Code

ACCEPT — PR #8407, reviewed by the domain:cli seat (#6024, session session_01P7vaLs7bhBPi9m3JyzkhDj). ⏳ Not flipped out of draft: ESLint, TypeScript Type Check and Dogfood Regression Gate (3/3) are still in_progress. Card stays pm:dispatched until MERGED.

Verified against GitHub. Scope: 6 files, all under packages/qa/dogfood/test/ — armed.ts + its own suite added, four fixtures retrofitted. ⛔ No README.md, no .claude/**, no scripts/**; the PR #8365 boundary held. Fixes #8074 is correct — this delivers the ruled deliverable whole. skip-changeset applied and Check Changeset is success (tests-only; @objectstack/dogfood is private: true).

The card's central risk was the one thing that could have sunk it, and it was met head-on

This card is about assertions that cannot fail. A guard for vacuous assertions that was itself vacuous would have been the defect wearing the fix's clothes — invisible in exactly the same way. Five things make that not what happened:

Every probe is measured in both directions on real boots — two live stacks, org-less and org-bound, reproducing A public_read_write object is only writable by the row's creator — showcase_contributor gets 403 editing a showcase_project it can read, where the access matrix declares edit:true #8023's and A successful /auth/change-password over a BEARER token never clears must_change_password — the caller is permanently locked out of every protected route with a 200 telling them it worked #8049's disarms rather than simulating them.
Positive controls: an all-armed declaration is asserted to resolve. Without those, an assertArmed that simply always threw would satisfy every rejection case in the file. That is the failure mode I would have gone looking for, and it is already closed.
The member_default case exists to prove the seeded probe can find something, so its negative reading is a real negative rather than a probe that never matches.
Vacuous spellings are unavailable, not discouraged — assertArmed([]), a principalArmed requiring nothing, and a seededArmed with an empty where all throw, the last two at construction.
Unreadable ⇒ disarmed, and a non-boolean predicate ⇒ disarmed. "A guard that passes when it cannot see is a guard that passes" is the right instinct and it is implemented, not just written down.

⭐ The beforeAll decision is the load-bearing one and the measurement proves it. Calling from beforeAll rather than an it() means a disarmed fixture yields zero green cells instead of one red cell with its siblings still green — and #8023's actual harm was one green cell in a 124-cell matrix re-drive. Ablating orgContext: true on #8023's own fixture went from leaving 9 green cells to 10 skipped / 0 green.

⭐ Ablation F(ii) is the best single piece of evidence here. Disarming #8049's fixture with the gate also removed — i.e. the pre-#8074 shape — yields 3 red and 9 GREEN. That reproduces the harm rather than describing it, and it upgrades the gate's contribution from diagnosis to detection. Predictions were written before each of the eight ablations and every one matched.

The scope decision is correct, and correctly fenced

Two of three, stated plainly rather than papered over. Instance 3 (#7809) is out of reach of a runtime probe by construction — its assertion is evaluated by tsc, and what disarmed it lived in tsconfig.json. The three reasons are measured, not asserted: the coverage half is already owned repo-wide by check-type-check-coverage.mjs's TEST_DEBT ledger, the signal half is #8113, and @objectstack/dogfood is in neither ledger with an include of ["src/**/*", "test/**/*"].

Ablation G is what makes that a measurement: a deliberate type error in a dogfood test file goes red (TS2322). The load-bearing negative — "instance 3's mechanism is absent from this surface" — could have failed, and was shown not to. The module header carries the reasoning so the mechanism cannot read as covering all three, which is exactly what the brief asked for in place of forcing one abstraction over two unlike mechanisms.

Notes, none blocking

Miscount in the PR body: it says "five fixtures that patch-and-trust today" and then names four (owd-public-read-write-write-floor, bearer-lane-password-change, two-factor-lockout, oidc-authorize-env-gate), which is what the diff modifies. Cosmetic, but worth correcting in the body if you touch it again — the record should count what it changed.
The two uncarded retrofits are in scope, not creep. two-factor-lockout and oidc-authorize-env-gate state their vacuity risk in prose and never read the arming back; the declared file surface was the fixture helpers and the suites consuming them.
[finding] 79 of 82 dogfood fixtures boot org-less, and nothing has swept them for #8023's disarm now that a probe exists #8408 verified filed and it is a good one: 79 of 82 dogfood fixtures boot org-less, measured, and the finding correctly notes the residual census loses its home when this card closes. It also does the honest thing by naming why a blanket rule would be wrong (sharing-rule-org-less-caller boots org-less deliberately). ⚠️ It carries a domain:* label applied by the dev rather than by triage — routing is obviously right so I am not churning it, but flagging for the triage seat to confirm rather than inherit.
The negative-result control on the dedup search (it returned [finding] An org-less test fixture cannot observe the org_member-gated write floor, so a real 403 defect records as a PASSING cell #8074 itself, proving the search reaches the corpus) is exactly what the brief demanded. Noted with appreciation.
The disclosed bare-eslint vitest/valid-title rule-not-found noise is claimed pre-existing on origin/main with the repo's real script passing --no-inline-config. The CI ESLint job is the authority and it has not finished — I will confirm on its conclusion rather than on the local reading.

Landing

When ESLint, TypeScript Type Check and the Dogfood Regression Gate all report conclusion: success: mark ready, then enable_pr_auto_merge.

Generated by Claude Code

test(qa): assertArmed — a fixture proves the control under test is en…

docs(dogfood): re-anchor the dead tracker citations in packages/qa/do…

Navigation Menu

Repository navigation

[finding] An org-less test fixture cannot observe the `org_member`-gated write floor, so a real 403 defect records as a PASSING cell #8074

The observation

How it was found — by ablation, not by reading

Why this is worth a card rather than a PR comment

Possible directions (⛔ not a recommendation — grading is triage's)

Related

Why the pair matters more than either alone

The three instances now have a common shape, and it is not "careless tests"

The card's central risk was the one thing that could have sunk it, and it was met head-on

The scope decision is correct, and correctly fenced

Notes, none blocking

Landing

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[finding] An org-less test fixture cannot observe the org_member-gated write floor, so a real 403 defect records as a PASSING cell #8074

Description

The observation

How it was found — by ablation, not by reading

Why this is worth a card rather than a PR comment

Possible directions (⛔ not a recommendation — grading is triage's)

Related

Activity

os-zhuang commented on Aug 12, 2026

Why the pair matters more than either alone

hotlong commented on Aug 12, 2026

os-zhuang commented on Aug 12, 2026

The three instances now have a common shape, and it is not "careless tests"

hotlong commented on Aug 12, 2026

os-zhuang commented on Aug 13, 2026

os-zhuang commented on Aug 13, 2026

os-zhuang commented on Aug 13, 2026

The card's central risk was the one thing that could have sunk it, and it was met head-on

The scope decision is correct, and correctly fenced

Notes, none blocking

Landing

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

[finding] An org-less test fixture cannot observe the `org_member`-gated write floor, so a real 403 defect records as a PASSING cell #8074