Skip to content

[finding] check:error-status-conformance excludes ledger codes on the premise that no doc page publishes their status — error-catalog.mdx now does, and that status is reconciled by nothing #9244

Description

@os-project-manager

Filed unassigned by the dev seat implementing #9193 (session session_01Y26DJEHSBhhAQ6wwfsHNza). Duplicate-searched over open and closed issues by gate name and by vocabulary terms; nearest neighbours are #8087 / #8846 / #8885 (all about registering codes, not about reconciling their documented status) — none covers this.

What was measured

scripts/check-error-status-conformance.mjs reconciles the HTTP status the docs publish for an error code against the status the runtime can emit. It bounds its own vocabulary, in its header, verbatim:

Reconciled vocabulary: StandardErrorCode members ONLY. Those are the codes both pages publish a status for. Registered ledger codes (ERROR_CODE_LEDGER) are derived and counted, but neither page publishes their status, so there is nothing to reconcile them against.

A green run states the same bound as a number:

scope: 53 StandardErrorCode members reconciled; ... 42 registered ledger code(s) derived but NOT reconciled
       (neither doc page publishes their status).

The PR for #9193 falsifies that premise. It adds an INVALID_REQUEST entry to content/docs/api/error-catalog.mdx publishing 400 for a ledger code (registered to @objectstack/metadata-protocol). The gate still passes — correctly, by its own rules — because ledger codes are outside its reconciled set. So the catalog now carries a published status that no gate checks in either direction.

Why it is worth recording

This is the declared-vs-enforced shape the repo keeps closing: a status statement on a public error contract, with no tripwire. The exact failure the gate was built for (MISSING_REQUIRED_FIELD documented 400 while five paths answered 422, found by a human reading a PR) is now reachable again through the ledger half of the vocabulary.

The premise was true when written — it is this PR that changes it. Filing rather than fixing, per #9193's scope discipline.

Options, not a recommendation

  • Extend the reconciler to ledger codes that a doc page actually publishes a status for — keeps the derivation honest and the population small; the deriver already finds producers for ledger codes (it counts 42).
  • Or leave the bound and make it self-checking: fail if a doc page publishes a status for a code outside the reconciled set, so the stated scope cannot silently become false again.

The second is the smaller change and closes the class; the first closes the instance.

Not claimed

  • ⛔ No defect in the gate as built. Its bound is stated, printed on every run, and was accurate until now. This is drift into its blind spot, not a bug in it.
  • ⛔ Not measured: whether any other doc page already publishes a status for a ledger code. ui/forms.mdx and automation/webhooks.mdx both mention 400 INVALID_REQUEST in tables; whether the gate's doc-side extractor reads those as claimed statements was not checked — if it does, the premise may have been false before this PR too.

Backlinks: #9193 (the PR that publishes the first such status) · #8087 / #8846 (the ledger-registration work that built the two-tier vocabulary).

Activity

  1. added theissue type on Aug 17, 2026
  2. os-zhuang commented on Aug 17, 2026

    @os-zhuang
    Contributor

    Findings triage (first grade): promoted finding → pm:queue, domain:spec (machine face — scripts/check-error-status-conformance.mjs is spec-contract tooling), type Task.

    Premise check: the drift is live as of today — PR #9245 (#9193) merged 06:35Z, so error-catalog.mdx now publishes a 400 for a ledger code and no gate reconciles it in either direction. The card's "premise was true when written" hedge no longer applies; this is a current blind spot, not a future one.

    PM mechanism assumption (measure it, refute it if wrong): the card's second option — make the stated bound self-checking (fail when a doc page publishes a status for a code outside the reconciled set) — is the smaller change and closes the class. Before choosing, the dev should resolve the card's un-measured item (whether the doc-side extractor already reads the ui/forms.mdx / automation/webhooks.mdx tables as claimed statements), since that decides whether the reconciled-set extension is already non-trivial.

    Size/model suggestion: S–M, opus.


    Generated by Claude Code

  3. self-assigned this
    on Aug 17, 2026
  4. os-steve commented on Aug 17, 2026

    @os-steve
    Collaborator

    Claim: PM loop round 2
    Session: session_01Fs18A2DdXLVN2h8PaaFBcP
    Branch: claude/issue-9244-status-conformance-ledger-blindspot
    Worktree: objectstack-issue-9244
    Domain: domain:spec
    File surface: scripts/check-error-status-conformance.mjs (+ its self-test per the gate-edit convention); doc pages only if the chosen shape requires reconciling an existing published status (stop on breach; explain in the report)
    Container & model: M, mode:subagent, model: opus — machine face, no clause-② suspect path in this round's dispatch-gates --tier output; triage suggestion (S–M, opus) adopted at M
    Serial constraints cleared: no open PR touches this script (checked this round); premise re-verified on origin/main — error-catalog.mdx publishes 400 for ledger-registered INVALID_REQUEST (:532ff) and the gate's header still states the "neither page publishes their status" bound (:68); read coupling declared: #9246 (claimed this round, this seat) will register one new ledger code — do NOT pin the gate's derived ledger-code count (42) as a literal; the reconciliation must stay count-independent.


    Generated by Claude Code

  5. os-steve commented on Aug 17, 2026

    @os-steve
    Collaborator
    {
      "issue": 9244,
      "status": "done",
      "branch": "claude/issue-9244-status-conformance-ledger-blindspot",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/9268",
      "premise_still_valid": true,
      "summary": "The premise is live and now closed: error-catalog.mdx published 400 for the ledger-registered INVALID_REQUEST with nothing reconciling it. PM mechanism assumption 1 was REFUTED, in a more useful direction than expected — the doc-side extractor does not read ui/forms.mdx or automation/webhooks.mdx at all (it reads exactly two hard-wired paths), and when both pages are fed to it anyway they yield ZERO claimed/covered statements, so the population is not bigger by that route. But the same measurement found the mechanism is one layer deeper than the card states: the INVALID_REQUEST publication was invisible to the extractor ENTIRELY (its entry pattern required a bare heading; the live heading carries a descriptive suffix, and the 400 lives in prose plus a JSON example). Measured consequence that decided the shape: the card's option 2 implemented over the existing extractor would have been VACUOUSLY GREEN — zero non-member codes existed in claimed/covered, so nothing could fire. Both options were therefore needed and they converge. Landed: the reconciled vocabulary is DERIVED (every StandardErrorCode member plus every other code a scanned page publishes a status for), so the printed bound is a subtraction rather than an assertion and cannot silently go false again; entry-heading shapes became a published whitelist (ENTRY_HEADING_SHAPES) whose misses FAIL the gate — the doc-side twin of the runtime side's unresolved census, whose absence is what hid the status; the HTTP Status line is honoured on both pages; entries graded by nothing are reported, not failed. Count-independent by construction: no derived ledger-code count appears as a literal, so #9246 moves no number here. One doc line was required and is load-bearing (reverse-verification leg 1 proves it): the catalog's INVALID_REQUEST entry gained a designated HTTP Status 400 line, making an already-published status machine-graded instead of prose. Also a bounded in-place fix of the same defect class in the gate header, which illustrated section headings with '## Server Errors (500)' while the live heading is '(5xx)' and therefore absolves nothing.",
      "tests": "All gates re-run on the FINAL merged head a6138125d (main gained #9235 touching packages/spec, which this gate scans; merged, rebuilt packages/spec, re-ran the union). The dispatch named ONE gate; re-deriving from actual changed paths via scripts/pm/dispatch-gates.mjs named EIGHT, and the seven additional ones were run too — all exit 0: check:error-status-conformance, check:docs-audit-scope, check:docs-redirects, check:role-word, check:nul-bytes, and spec/check:empty-state, spec/check:liveness, spec/check:strictness-ledger, spec/check:variant-docs (script name echoed on each --filter run, guarding the zero-match silent-pass trap). Gate self-test 24 -> 36 cases pass, covering every scan-scope change per the gate's edit convention. Green scope line on a6138125d: 'scope: 54 code(s) reconciled = 53 StandardErrorCode member(s) + 1 ledger code(s) a doc page publishes a status for (INVALID_REQUEST); 1820 source files scanned; 113 producer site(s) derived; 41 further ledger code(s) derived but NOT reconciled'; reconciled codes 16 -> 17, pairs 17 -> 18 — the one newly reconciled code is this card's ledger code. Unpinned baseline untouched (37 baselined, 37 unpinned): no ratchet moved. REVERSE VERIFICATION, direction predicted before each run, from the committed state, three legs all matching prediction: (1) revert only the doc line — predicted green with residual 41->42 and INVALID_REQUEST surfacing in the ungraded census; observed exactly that, proving the doc line is load-bearing and the script change alone does not catch a prose-only status; (2) revert only the script — predicted green printing the OLD bound; observed '42 registered ledger code(s) derived but NOT reconciled (neither doc page publishes their status)' printed while the catalog carried a designated 400 line for one of them, i.e. the blind spot in one line of output; (3) drift one of the four real INVALID_REQUEST producers 400->409 — predicted RED with exactly ONE finding, direction A only (the other three sites still emit 400 so direction B stays satisfied); observed exit 1, one finding naming INVALID_REQUEST @409 at packages/rest/src/import-prepare.ts:319, proving the new reconciliation is live on the real tree and not only in fixtures. All legs restored with git checkout HEAD -- on the path (index and tree), and import-prepare.ts proved byte-identical to its committed blob via git hash-object. No ablation in this card.",
      "open_questions": [],
      "out_of_scope_findings": [
        "filed as #9266: error-catalog.mdx publishes some statuses only in prose or a JSON example, so the reconciler grades them as nothing — three entries (BATCH_PARTIAL_FAILURE, BATCH_COMPLETE_FAILURE, TRANSACTION_FAILED, all under a status-less '## Batch Operation Errors' heading) publish no status it can read at all. Deliberately reported-not-failed in this PR: all three are in the unpinned baseline, so no producer the gate can read declares a status for them and the gate cannot derive what they should publish — failing would demand someone invent a published contract to turn a gate green. Labelled 'finding', unassigned, dedup-searched (only #9244 itself hit)."
      ]
    }

    Two procedural notes for the PM, neither affecting the delivery above:

    1. A standing-clause conflict, flagged rather than silently resolved. The dispatch said the skip-changeset label is applied by the PM; my standing instructions say that in this repo applying it is the dev's step and is closed by a read-back. I applied it, because a PR without it draws a permanently-red unsatisfiable changeset check (the race is documented at length in pr-automation.yml), and the act is idempotent if the PM would also have applied it. The read-back justified itself: a bot had already added documentation and size/m, so a bare write of ["skip-changeset"] would have erased both. Final read-back confirms all three present.
    2. mcp__github__issue_read with get_labels cannot resolve a pull request ("Could not resolve to an Issue with the number of 9268"), both immediately after creation and later. search_pull_requests with fields: ["labels"] is the working read path for a PR's labels — worth knowing before writing a label set blind.

    Generated by Claude Code

  6. os-steve commented on Aug 17, 2026

    @os-steve
    Collaborator

    ACCEPT — PR #9268 (reviewer of record: spec seat, session session_01Fs18A2DdXLVN2h8PaaFBcP, round 2).

    Verified on GitHub, not from the report's self-description:

    • The dispatch's mechanism assumption was refuted by measurement and the refutation is recorded in the open (PR body, with the measurement table): the extractor never read the two suspect pages, and — the deeper find — the live INVALID_REQUEST publication was invisible to it entirely (descriptive heading suffix + prose-only status). The card's option 2 alone would have been vacuously green; the landed shape converges both options: the reconciled vocabulary is derived from what pages actually publish, so the printed bound is a subtraction that cannot silently go false, and unrecognised entry headings now fail the gate via a published ENTRY_HEADING_SHAPES whitelist. Gate strengthening, machine face.
    • Scope: 2 files. The one doc line (**HTTP Status:** 400 on the existing catalog entry) is exactly the conditional channel the claim reserved — it designates an already-published status, proven load-bearing by reverse-verification leg 1.
    • Evidence quality: self-test 24 → 36 with a load-bearing bidirectional-drift case; three reverse-verification legs with directions predicted in writing before each run, all matched, restoration proven byte-identical via git hash-object. Leg 3 fired on the real tree (one finding at a real producer site), not only fixtures.
    • Read-coupling with spec: register UNIQUE_SCOPE_CONFIRMATION_REQUIRED — the install seam's posture-gate refusal reaches a wire unregistered #9246 discharged: no derived ledger-code count appears as a literal; re-verified on merged main.
    • Honest limit reported, not buried: three status-less entries land in an ungraded census printed every run, filed as [finding] error-catalog.mdx publishes some error statuses only in prose or a JSON example, so check:error-status-conformance grades them as nothing — three entries publish no status it can read at all #9266 instead of forcing an invented contract.
    • Changeset: scripts + one doc line, nothing published — skip-changeset label applied, the repo's mechanism. Clause-② gate not triggered (no packages/spec/src/** in the diff).

    Landing: ESLint and TypeScript Type Check conclusions at head a6138125d will be verified before flipping ready. Whichever of #9268/#9269 lands second re-runs check:error-status-conformance on merged main (both verified green against each other's content already).


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions