Repository navigation
feat(agents): accept and implement ADR-0031, per-repository instruction routing - #662
Conversation
…elivery Instruction routing reaches only this repository. Every runtime discovers instructions from the Git root of its working directory downward, and each organization repository is its own Git root, so the routing obligation in this repository's AGENTS.md never arrives anywhere else. A survey of the fifteen repositories found one that names the routing manifest, twelve that link to policy in prose only, and two with no agent instructions at all. Of 225 recorded sessions, 188 never opened the manifest. Twelve repositories vendor the code-review skill at pinned revisions that nothing reconciles against canonical. Their bodies are byte-identical once installer metadata is removed, but they sit on two different pins and neither is current. No workflow in any repository performs that check. ADR-0031 proposes a generated, delimited routing preamble in every repository's AGENTS.md, produced by a single generator owned here and delivered by a reusable workflow that also verifies vendored skill pins against a declared approved revision. It adds no policy: it delivers and verifies rules the canonical baseline already states. Status is PROPOSED; only a maintainer moves a record to ACCEPTED. Refs: #660
ss-o
left a comment
There was a problem hiding this comment.
Fallback review under ADR-0026: Copilot request not registered on ac1b403
z-shell/.github is class 4 in lib/repository-classes.yml, so ADR-0026 point 3 permits this fallback. A review_requested event was recorded on the timeline with a null reviewer and no run followed within the session, across two waits totalling roughly three minutes; requested_reviewers stayed empty. Reviewed under .github/skills/code-review/SKILL.md against code-review-generic.instructions.md.
Scope
Documentation only: one new ADR, one manifest surface, one regenerated index row. No executable code, no workflow, no shell. The dialect and lifecycle sections of the review criteria do not apply.
Deterministic checks
Re-run against a clean git archive export of HEAD plus this change, so untracked local tooling could not influence the result:
python3 scripts/validate-agent-policy.py passed
python3 scripts/decision-records.py --check passed
python3 -m unittest scripts/test_validate_agent_policy.py 92 tests, OK
python3 -m unittest scripts/test_decision_records.py 17 tests, OK
All 11 PR checks pass, Validate Agent Instructions included.
Claim verification
Every quantitative claim in the ADR was re-derived from the working tree rather than taken from the body:
- 15 repositories: confirmed.
- 1 repository naming the routing manifest: confirmed,
z-shell/.githubalone. - 188 of 225 sessions not reading the manifest: confirmed from
audit-agent-routing.py --json(claude-code 25 of 43, codex 12 of 182; 37 read, 188 did not). The 0-of-182 codex hook-context figure is confirmed. - All relative links in the ADR resolve to files that exist.
One correction is needed, below.
Findings
-
Severity: IMPORTANT
-
Rule / Category: docs/accuracy
-
Location:
decisions/0031-per-repository-instruction-routing-delivery.md, Context section and the survey table -
Problem: The ADR says twelve repositories vendor the code-review skill. There are thirteen copies on disk; twelve are downstream and the thirteenth is this repository's canonical file, which is the source rather than a vendored copy. The prose is defensible but the distinction is never stated, so a reader counting files gets a different number than the ADR gives and cannot tell which is wrong.
-
Fix: State it explicitly once, for example "thirteen copies exist: this repository's canonical file and twelve vendored downstream copies." The twelve-copy claims elsewhere then read unambiguously.
-
Severity: IMPORTANT
-
Rule / Category: docs/specification-completeness
-
Location:
decisions/0031-per-repository-instruction-routing-delivery.md:93and:110 -
Problem: Point 1 requires the preamble to enumerate "that repository's own declared surfaces", and point 4 has the workflow fail "when a declared surface is missing". Only
z-shell/.githubhas an.github/instruction-surfaces.json; no other repository has one. The ADR never says where a downstream repository's surfaces are declared: in a new per-repository manifest, in a section of the canonical manifest, or by convention from the repository's tree. An implementer cannot build the generator from this text without choosing for themselves, and the three options have materially different ownership consequences. -
Fix: Name the source of truth in point 1. Declaring downstream surfaces inside the canonical manifest is most consistent with the rest of the decision, since it keeps one inventory and avoids fifteen new files, but the choice should be explicit either way.
-
Severity: SUGGESTION
-
Rule / Category: docs/specification-completeness
-
Location:
decisions/0031-per-repository-instruction-routing-delivery.md:127 -
Problem: Point 6 says this repository "records the revision downstream repositories are expected to pin" without naming the file that holds it.
lib/repository-classes.ymlis the established precedent for exactly this kind of machine-readable per-repository fact. -
Fix: Name the intended location, or state that implementation chooses it.
-
Severity: SUGGESTION
-
Rule / Category: process/coordination
-
Location: PR body, "Notes for review"
-
Problem: The dependency on #543 is noted in prose only. If #543 lands first, the caller references this decision introduces must already follow it.
-
Fix: Consider linking these as related issues so the ordering is visible on the board rather than only here.
Non-blocking observations
- No em dash, no
zdharmareference, no literal backslash-n in the ADR, the issue body, or the PR body. No agent or bot credit in the commit trailer. - Status is
PROPOSEDwithDeciders: TBD, correct perrunbooks/adr.md; acceptance is the maintainer's to make onmain. - Unrelated but worth an issue:
scripts/bootstrap.sh --checkreports "instruction composite missing or stale" for nearly every checkout in the meta-workspace, andvalidate-agent-policy.pyfails inside a ws worktree on the untracked.claude/rules/ws-workspace.mdthat the workspace tooling generates, which the validator classifies as an undeclared runtime-specific instruction carrier. Both are pre-existing and outside this diff, but the second means an agent running the validator from a worktree sees a spurious failure.
Verdict
The evidence is sound and independently reproducible, and the decision follows from it. The two IMPORTANT findings are specification gaps in a document whose purpose is to be implemented from; both are a sentence each to close. No CRITICAL findings.
Name the source of truth for downstream surfaces. Point 1 required the preamble to enumerate a repository's declared surfaces, but only this repository has a manifest and the record did not say where the others would be declared. They are declared here, keyed by repository, which keeps one inventory as ADR-0014 point 5 intends. The schema and validator cost of that choice is recorded in Consequences. Name the file that records the approved skill revision, following the precedent lib/repository-classes.yml sets for per-repository facts that tooling consumes. Distinguish the thirteen copies of the code-review skill on disk from the twelve vendored downstream copies, so the counts in the record can be reproduced without guessing which file is the source. Refs: #660
|
Review findings addressed in 8ae70f8. All 11 checks pass on the new head.
Re-verified against a clean Status remains |
Add scripts/org-routing.py, the single generator and verifier that decision 0031 calls for. The downstream section of the instruction-surface manifest declares each consuming repository's routable surfaces (Copilot adapter, scoped instructions, agents, prompts, local skills) and the organization skills it vendors. apply writes a generated routing block into a repository's AGENTS.md between org-routing markers; check fails when the block, the declared surfaces, or a vendored skill disagree with the inventory. lib/approved-skills.json records the approved revision, file list, and content digest of each vendored organization skill. The digest excludes the installer metadata block and normalises frontmatter key order and indentation, so only a real content change or a stale pin is reported. verify-approved proves each digest against the recorded commit. The reusable Org Routing workflow runs check for a caller at the inventory of the pinned workflow commit. The agent policy validator validates the downstream section through the generator's own schema, so the two cannot disagree. Refs: #660
Record the maintainer's acceptance and the delivered implementation: manifest section, approved-skill record, generator, reusable workflow, and validator reuse. Route the instruction-update checklist and the org review's skill provenance and currency checks to the deterministic generator, and state how the approved revision advances. Refs: #660
The review of the routing generator found drift that passed check: - The workflow fell back to the caller's own commit when the called workflow identity was missing, which would let a caller supply the tooling that checks it. It now fails closed, as trunk.yml does, and decides the self-test path from the workflow source rather than the repository name, so a fork runs its self-test. - Instruction files that a discovery glob found but no surface pattern accepted were dropped, and nested or runtime-specific carriers (CLAUDE.md, .claude/, .cursor/, nested AGENTS.md) were never seen. Both now fail, and scoped instructions are discovered recursively. - The installer metadata block was excluded from the digest without restriction. A vendored skill now admits only the installer's keys, with no nesting and a consistent github-ref. - Symbolic links inside a vendored skill were skipped. Every entry is now listed, links marked, so any link fails the file comparison. - A block moved down AGENTS.md or wrapped in a fence passed. The block is now required at the top, after an optional title. verify-approved also requires the approved commit to be reachable, and paths with control characters are escaped in output. The concurrency group is suffixed so it cannot collide with other called workflows. The ADR records the implementation-time inventory beside the drafted survey. Refs: #660
ss-o
left a comment
There was a problem hiding this comment.
Fallback review under ADR-0026: Copilot request not registered on 33cd66e
The Copilot review request was sent after checks passed on 33cd66e9. It never registered: the timeline has no review_requested event for Copilot, only the earlier TSC team request, so the review is blocked on quota. z-shell/.github is class 4 in lib/repository-classes.yml, so this fallback review applies.
This review was run under .github/skills/code-review/SKILL.md against code-review-generic.instructions.md, on origin/main...33cd66e9. It was an independent adversarial pass by a separate agent in a fresh context.
Findings on 33cd66e: no CRITICAL, five IMPORTANT, all fixed in 79ee9d4
org-routing.yml: tooling source fell back to the caller. Whenjob.workflow_*was missing, the tooling source fell back to the caller's own commit, so a caller could supply theorg-routing.pythat checks it. It now fails closed, matchingtrunk.yml. The self-test path is chosen by comparing the workflow source with the repository, so forks run the self-test.org-routing.py: unroutable instruction files passed silently.- Glob hits that no surface pattern accepted (for example
.github/skills/My_Skill/SKILL.md) were dropped without an error. - Runtime carriers were never discovered: nested
AGENTS.md,CLAUDE.md,GEMINI.md,.claude/,.cursor/,.cursorrules,.github/chatmodes/. - All of these now fail check, and
.github/instructions/is scanned recursively.
- Glob hits that no surface pattern accepted (for example
org-routing.py: metadata was excluded from the digest without limit. Any key or nesting could hide there. A vendored skill now admits only the installer's keys (METADATA_KEYS, previously unused), with no nesting and no duplicates, and requiresgithub-refto equalgithub-pinned.org-routing.py: symbolic links inside a vendored skill were skipped. Every entry is now listed, with links marked by@, so a link always fails the approved-file comparison.org-routing.py: block position was not enforced. A block moved downAGENTS.mdor wrapped in a code fence passed, contrary to ADR point 1.splicenow removes the block and reinserts it at the top, after an optional title, so either placement fails check.
Suggestions, also taken
verify-approvedrequires the approved commit to be an ancestor ofHEAD.- Paths with control characters are printed with
repr, so they cannot inject workflow commands. - The concurrency group is suffixed
-org-routing, and the caller example warns against a caller-levelconcurrency:. - The ADR's Implementation section records the implementation-time inventory (fifteen repositories including
zi-setup, elevencode-reviewcopies) beside the drafted survey.
Verification on 79ee9d4
test_org_routing.py: 40 tests OK, locally and in CI on Python 3.10, where the self-test also ranvalidateandverify-approvedwith full history.- Mutation checks: disabling each new guard (the nine listed above) makes the suite fail.
- The reviewer's probe repositories now fail on every reported bypass.
- The downstream apply/check round trip across all fifteen repositories adds lines only. Six are current, eight fail only on stale pins, and
srcalso fails on driftedzi-installcontent, which is genuine. validate-agent-policy.py,decision-records.py --check, and the zsh standard validator pass, along with all four unit suites, on a clean export.trunk checkon the changed files reports no issues.- All PR checks pass.
No open findings remain. Residual gap: the reusable workflow's caller path (self == false) cannot run until a downstream repository calls it. That first run is part of the adoption follow-ups.
Code quality flagged the adjacent string literals in the routing block list as possible missing commas. Build each multi-part line as a named value joined with an explicit +. The rendered block is byte-identical across all fifteen downstream inventories. Refs: #660
What this does
Accepts ADR-0031 and delivers its organization-side implementation. It closes the two delivery gaps in ADR-0014: routing that never reaches other repositories, and vendored skill pins that nothing reconciles.
decisions/0031-...md: statusACCEPTED,Deciders: ss-o, plus an Implementation section.scripts/org-routing.pyis the single generator and verifier:apply: writes the generated block betweenorg-routingmarkers in one checkout'sAGENTS.md, or createsAGENTS.mdwhen it is absent.check: fails when a checkout disagrees with the inventory.validate: checks the inventory.verify-approved: confirms each approved skill digest against its recorded commit..github/instruction-surfaces.jsongains adownstreamsection covering all fifteen consuming repositories.AGENTS.mdis an alias, not a surface.lib/approved-skills.jsonrecords the approved revision, file list and content digest forcode-review(e1f8f6c1) andzi-install(1bfbf33a).metadatablock..github/workflows/org-routing.ymlis a reusableworkflow_callworkflow.verify-approvedinstead.scripts/validate-agent-policy.pyvalidatesdownstreamthrough the generator's own schema check, so the two cannot disagree. The three new files join the enforcement inventory.runbooks/instruction-update.mdandrunbooks/org-review.mdnow route surface changes, skill provenance and currency checks, and advancing an approved revision to the generator.Closes #660
Why
Every supported runtime discovers instructions from the Git root of its working directory downward, and each organization repository is its own Git root. The routing obligation that sends a runtime to
.github/instruction-surfaces.jsonlived only in this repository'sAGENTS.md, so it never arrived anywhere else.code-reviewat pins nothing reconciles.ADR-0014 point 7 requires that mandatory policy not depend on an optional mechanism. The ADR's context has the full evidence.
Impact review
The seven answers required by
runbooks/instruction-update.mdare in #660. In short:downstreamsection.Verification
All of the following ran against a clean export of the tracked tree, so local untracked workspace carriers could not affect the result. Where noted, they also ran in the checkout with full history.
I ran the checker against checkouts of all fifteen downstream repositories:
proseWrap: always) all leave it unchanged.apply:wiki,z-a-default-ice,z-a-eval,zd,zsh-fancy-completions,zunit.code-reviewcommit (23fb0c86orce74af22). That is the pin-currency failure decision point 5 asks for.srcvendorszi-installfromce74af22, which is stale in both pin and content.Mutation checks confirmed that the tests catch a disabled digest comparison and a disabled undeclared-surface check. They also exposed a missing test for surface ordering, which is now added.
Notes for review
org-routing.ymlmust pin a full SHA or an immutable release, not@main.