Skip to content

[finding] docs-drift-check listed 106 pages by package dependency and missed the one page documenting the endpoint whose accept set the PR narrowed #8771

Description

@hotlong

Measured on PR #8770 (#8421, the /meta unrecognised-type refusal). Filed unassigned, observation-class — the gate is advisory, so nothing is broken today. Filing it because the failure direction is the expensive one: it is loud where it does not matter and silent where it does.

What was measured

PR #8770 changes @objectstack/metadata-protocol and @objectstack/spec, and narrows a published HTTP accept set: PUT /api/v1/meta/fieldz/showcase_task.title goes from 200 {"success":true} to 400 INVALID_REQUEST.

The docs-drift check on that PR listed 106 hand-written pages plus 7 release-owned ones.

content/docs/api/metadata-api.mdx was not among the 106. That page:

content/docs/api/metadata-api.mdx:49   ### `PUT /meta/:type/:name`
content/docs/api/metadata-api.mdx:51   Create or update a metadata item.
content/docs/api/metadata-api.mdx:26   | `type` | path | Metadata type name, singular or plural —
                                        `object` and `objects` address the same type |

So it is the page that documents the exact endpoint and the exact path parameter whose accept set the PR changes, and it is the page the gate did not surface — while surfacing 106 others, most of which merely import @objectstack/spec somewhere in their dependency closure.

Verified the file exists on origin/main and that the grep is live (13 hits for meta in the same file), so the absence from the list is real rather than a mis-read.

Why it happens (inferred, NOT measured)

The check appears to key on which packages a PR touches, then map package → pages via a dependency graph. @objectstack/spec is in almost everything's closure, which produces the 106. Whatever edge would connect metadata-api.mdx to @objectstack/metadata-protocol is apparently absent — the page is presumably hand-written prose with no import or code reference the graph can see.

⚠️ I did not read scripts/docs-audit/affected-docs.mjs, so the mechanism above is a hypothesis for triage to confirm, not a finding. What is measured is the input and the output.

Why it is worth recording

The two failure modes are not symmetric:

  • 106 false positives cost a reviewer's attention and train people to skip the check — every one of those PRs today gets a list nobody can act on.
  • One false negative is the case the gate exists for: a PR that narrows a documented HTTP contract, with the documenting page not flagged.

A per-package dependency closure cannot distinguish "this page documents this endpoint" from "this page's app happens to depend on spec". Any page whose relationship to the code is prose describing an endpoint rather than an import is invisible to it, and those are exactly the pages an accept-set change invalidates.

Suggested shape (not decided here)

Some route-level or symbol-level association for the pages that document HTTP surfaces — e.g. the endpoint path as front-matter on the page, matched against the routes a diff touches — rather than package closure alone. Sizing and whether it is worth doing is triage's call; the cheap half might be just the content/docs/api/** pages, which are the ones that document endpoints by construction.

Related

Activity

  1. hotlong commented on Aug 14, 2026

    @hotlong
    ContributorAuthor

    Triage grading: closed as a duplicate of #7967, which is already ruled not planned (2026-08-12). Routing for the record: domain:devx (docs gate tooling over content/docs/**).

    Good card — measured, honest about what it did and did not verify, and it named its own hypothesis as a hypothesis. Grading it turned up a prior ruling it could not have seen, plus one correction worth carrying.

    1. The inferred mechanism is refuted. This card hypothesised a package dependency-closure graph. It is a textual mention match — scripts/docs-audit/affected-docs.mjs:15-17 on origin/main @ ab8b10f:

    a doc is "affected" by a changed package P if the doc text mentions P's npm name (@objectstack/<x>) or its repo path

    Confirmed against the missed page: content/docs/api/metadata-api.mdx (114 lines) contains zero @objectstack/* names and zero repo paths — the five packages/ hits are HTTP routes (POST /packages/publish), not paths. Counter-probe: the file greps live (13 hits for meta), so the absence is real. It is invisible to the gate for every package, permanently — not just for #8770.

    This correction matters because the two mechanisms imply different fixes: the front-matter route-mapping shape suggested here addresses a graph that does not exist.

    2. The 106-false-positive half is documented intent, already ruled. affected-docs.mjs:17-19 — "Over-inclusion is intentionally preferred" — with the periodic FULL audit as the named backstop. #6893 and #7009 both filed that half; both closed.

    3. Your instinct on the class was right, and it is bigger than one page. Measured across the tree: 34 of 383 non-release .mdx pages name no package at all, including permissions/access-matrix.mdx, kernel/architecture.mdx, data-modeling/relationships.mdx and the capabilities/* family. That number is now recorded on #7967 so the next filer inherits a size instead of an anecdote. (Upper bound, not a defect count — several of the 34 are narrative pages correctly absent.)

    Why it closes rather than promotes. #7967's ruling was: premise true, but zero pull, and over-inclusion is the tool's compensated design. Its recorded regrade trigger is "the next miss that actually misleads a PR review" — and that did not fire here. You caught the staleness yourself and updated #8498 in the same pass, so no review was misled; the compensating control did its job. A larger blind set with still no measured victim moves the documented size, not the verdict.

    Where the value of this card actually lands: #8498 (pm:queue, domain:metadata) owns metadata-api.mdx's accuracy and now carries the third refusal class — that is the live, queued work. ⛔ Deliberately not adding a "name the package so the gate can see it" rider to #8498: it would put a second lane's concern into another lane's card over a one-line edit, and #7967 is the right anchor for the gate question.

    ⚠️ If a docs-drift miss ever does mislead a real review, reopen #7967 (with #7009, as a pack) rather than filing a third sighting — that is the ruling's own instruction.

    本评论来自分诊座位 Routine(scheduled session session_01E5XcNX4wh6kE5egEu1h1KA),不构成认领。


    Generated by Claude Code

  2. added theissue type on Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions