Skip to content

ci(generate): lint @relationship examples in skill Markdown - #98

Merged
dawsontoth merged 2 commits into
mainfrom
ci/lint-relationship-examples
Oct 1, 2026
Merged

dawsontoth merged 2 commits into
mainfrom
ci/lint-relationship-examples

Conversation

@kriszyp

@kriszyp kriszyp commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Adds a CI lint that fails npm run validate when a @relationship example in a skill names an attribute its schema doesn't declare, puts a relationship on a field type Harper won't resolve, or describes a cardinality its attribute types contradict. validate is what the Validate check runs on every PR, including the bot's auto/docs-sync PR. On the pre-fix text of @relationship examples: singular from mismatch + scalar from+to mislabeled many-to-many (documentation#582) it flags both bugs. It passes on current main.

Refs documentation#582

For the human reviewer

  1. Requirement: is a lint in this repo the right fix? As specified, yes. The skill markdown is what agents read, and it is an LLM rewrite of the docs (plus some hand-written rules), so checking the docs alone can't vouch for it. All four historical bugs came from the docs source, though. The docs repo is still not clean: at documentation@6e8a01cb this lint finds 6 real problems (listed under Verification). The CLI takes any Markdown path, so documentation CI could run it as well. That is a follow-up, not part of this PR.

  2. Where it runs: PR CI only, not generate.yaml's Validate step. The alternative is to also fail the regeneration job. That would stop the sync PR from being opened or updated: the reviewer never sees the bad diff, unrelated rule updates wait for the upstream docs fix (docs: regenerate rules from documentation@3749d0c #70 waited about 3 weeks), and nobody can patch the rolling branch by hand, because nothing was pushed to it. Changing this later is one line in generate.yaml. Caveat: main has no required status checks (required_status_checks.enforcement_level: off), so a red sync PR can still be merged. I suggest making Validate required.

  3. Two rules beyond the requested (a)/(b):

    Both are exact checks, not heuristics. The planning reviewer suggested deferring the field-type rule because it could not check the Harper source. I overruled that after tracing the code at harper 56511db95. To drop either rule, delete one block.

  4. How the cardinality heuristic (b) is tuned. False positives cost more than misses, so it skips anything ambiguous:

    Known misses: a claim in an earlier paragraph, a fence that mixes cardinalities, and a wrong direction (many-to-one vs one-to-many) on a scalar from+to join. The right direction there depends on whether the join values are unique, which the schema doesn't show. Many-to-many is defined structurally, as Harper does it (search.ts:338): prose that calls a non-unique scalar join many-to-many fails.

  5. Escape hatch. Put <!-- lint-relationships: ignore --> on the line before a fence to exempt a deliberate counter-example. It is an HTML comment in the linted file, not an allowlist in this repo. I have not checked whether the docs build and regeneration keep HTML comments. If they don't, a false positive on a sync PR can only be cleared by hand-editing the rule on auto/docs-sync (edits there survive until the source changes) or by changing the lint.

  6. Hand-written parser vs graphql's parse(). The tokenizer understands only the part of GraphQL schema syntax these examples use. graphql is not a dependency, and parse() rejects the elided (...) examples the docs use, where this parser still checks the fields it can see. Round 1 found and fixed one edge case (unbalanced bodies). One is left, declined as a nit: an indented code block that shows a fenced example as literal text is still linted.

  7. Attribution in the task text. It cites skills#69 as one of the three bugs. skills#69 (the harper-mcp PR) contains no @relationship text. The earlier shipped bug is feat: phase 3 — migrate 7 rules to mode:generate from docs sources #44's @relation(from: "resellerId"), and the table below attributes by commit.

Which rule catches which historical bug

Historical bug Where it shipped Rule that flags it
resellerIds: [Long] with @relationship(from: "resellerId") docs querying.md (documentation#582 bug 1); skills querying-rest-apis.md from #44 until #70 merged; skills#70 review heads, defining-relationships.md relationship-attribute
@relation(...) instead of @relationship skills querying-rest-apis.md, #44 until #70 merged relationship-directive
scalar from+to join described as many-to-many docs schema.md (documentation#582 bug 2); skills#70 review heads relationship-cardinality
product: Product @relationship(from: productSku, to: sku) (to: on a single-valued field) docs schema.md; skills defining-relationships.md, #58 until #70 merged relationship-field-type

Output on the pre-fix docs (documentation@dbb3c182, reference/rest/querying.md and reference/database/schema.md):

querying.md:228: [relationship-attribute] @relationship(from: "resellerId") on Product.resellers: type Product declares no attribute "resellerId" (declared: id, name, resellerIds, resellers)
schema.md:407: [relationship-field-type] @relationship(from: "productSku", to: "sku") on OrderItem.product: field type must be [Product] — Harper only resolves a relationship with `to:` on a list field
schema.md:407: [relationship-cardinality] the text before this example says many-to-many ("This is useful for many-to-many relationships that join on non-primary-key attributes."), but @relationship(from: "productSku", to: "sku") on OrderItem.product is not many-to-many (Product.sku is not a list)

✗ lint-relationships: 3 problems. If a skill rule is generated, the same example is usually in its docs source — fix it there and regenerate.

Changes

  • scripts/generation/lib/relationship-lint.mjs: the rules; the header states the Harper semantics they encode. A single-pass GraphQL tokenizer and type parser run over every fenced block, including fences nested in list items. Four rules then run per directive and per fence, and anything the example doesn't show outright is treated as unknown.
  • scripts/generation/lint-relationships.mjs: the CLI. By default it lints each registered skill's rules/*.md and SKILL.md. AGENTS.md is skipped because it is assembled from the rules and validate-generated.mjs already proves it matches. It also accepts explicit file paths, prints file:line: [rule] message, and exits 1 on any problem, including an unreadable file.
  • scripts/generation/lib/sources.mjs: splitFencedBlocks is factored out of stripFencedBlocks so both share one CommonMark fence scanner, with a nested option for list-item fences. stripFencedBlocks output is unchanged: byte-identical on 120 real Markdown files (the skill rules and the docs reference/*/*.md), and the existing scanner tests pass.
  • package.json: adds lint:relationships and appends it to validate.
  • .github/CONTRIBUTING.MD: documents the lint in the file tree, the sync flow (step 11) and the development workflow.

Verification

End-to-end route: the CLI run through npm run validate, which is the command the Validate workflow runs, on current main plus the historical corpora.

Authored by Claude Opus 5.5 (dispatch dev-agent).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Hpew1juGf6rPZJZNjezi2F

Complexity: medium

Origin — the dispatch brief this PR was written from

ci(generate): lint @relationship examples in skill Markdown

LIVE CONVERSATION about #98.

You are answering a person, in a thread, one turn at a time. Every turn:

  1. Read the whole thread in this dispatch file's # Log — it is the conversation so far, and
    each of your previous turns is in it. Read the PR/issue and the code as needed.
  2. Answer the LAST message. Append your answer to # Log as your turn. Prose, not a report:
    they are talking to you, and a status template is not an answer.
  3. Set status: needs-input and stop. The thread stays open; their next message resumes it.

Each turn arrives as ASK (answer it, change nothing) or PERFORM (do it, then say what you did) —
the person chose which when they sent it, and the run's own prompt tells you which one this is.
Never infer it from the wording: an unrequested commit in the middle of a discussion and a polite
description of work that was supposed to happen are the two failures this exists to prevent.

Never mark a PR ready and never merge from this conversation.

Dispatch: task chat-pr-skills-98-kriszyp · queued by unknown · ran by codex/gpt-6-astra/low · worker kzyp-xps-1

Review-Coverage: authored=claude; ran=codex,gemini; adjudicated=domain; declined=cursor-grok,cursor-composer,cursor-kimi,cursor-muse; rounds=2; full=1 @ 593295e

Human-Review-Need: 3 (decisions: gate-location, many-to-many-definition, cardinality-granularity, ignore-marker-reach, hand-rolled-sdl-parser) @ 593295e

kriszyp and others added 2 commits September 23, 2026 09:15
Adds scripts/generation/lint-relationships.mjs, run by `npm run validate`
(and so by the Validate check on every PR, including the auto/docs-sync
PR). It parses GraphQL fences in each skill's rules and SKILL.md and fails on:

- relationship-attribute: from:/to: names an attribute the example's type
  does not declare (Harper reads it verbatim and resolves nothing)
- relationship-field-type: a to: relationship, or a from: on a list
  attribute, on a single-valued field (Harper does not resolve it)
- relationship-cardinality: an inline comment or the preceding paragraph
  states a cardinality that the referenced attribute's list shape contradicts
- relationship-directive: @relation, which Harper ignores without a warning

Anything an example does not show outright is treated as unknown and skipped.
splitFencedBlocks is factored out of stripFencedBlocks so the lint shares
the CommonMark fence scanner; stripFencedBlocks output is unchanged.

Dispatch-Task: skills-relationship-example-lint
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hpew1juGf6rPZJZNjezi2F
A directive argument list that never closes, or a fence that ends inside a
type body, left the type marked complete, so a relationship referencing a
field declared after the break was reported missing. Both now make
absence unknown, like any other unparseable body.

Also hoists the fence-open regexes to module constants and drops comments
that restated their functions.

Dispatch-Task: skills-relationship-example-lint
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hpew1juGf6rPZJZNjezi2F

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new relationship linter (lint-relationships.mjs and relationship-lint.mjs) to validate @relationship examples in Markdown files, ensuring that referenced attributes exist, field types are resolvable, and cardinality prose matches the schema definitions. It also updates the contributing guidelines, integrates the linter into the npm run validate pipeline in package.json, refactors the Markdown parser in sources.mjs to support nested blocks, and adds comprehensive tests. There are no review comments to address, so no additional feedback is provided.

@kriszyp
kriszyp marked this pull request as ready for review October 1, 2026 14:32
@kriszyp
kriszyp requested a review from dawsontoth October 1, 2026 14:32
@kriszyp
kriszyp requested a review from a team as a code owner October 1, 2026 14:32
@dawsontoth
dawsontoth merged commit 1031c2c into main Oct 1, 2026
6 checks passed
@dawsontoth
dawsontoth deleted the ci/lint-relationship-examples branch October 1, 2026 14:40
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 1.14.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants