Skip to content

feat(agents): accept and implement ADR-0031, per-repository instruction routing - #662

Merged
ss-o merged 6 commits into
mainfrom
feature-660-agent-routing-delivery
Sep 26, 2026
Merged

ss-o merged 6 commits into
mainfrom
feature-660-agent-routing-delivery

Conversation

@ss-o

@ss-o ss-o commented Sep 26, 2026 •

Copy link
Copy Markdown
Member

What this does

Accepts ADR-0031 and delivers its organization-side implementation. It closes the two delivery gaps in ADR-0014: routing that never reaches other repositories, and vendored skill pins that nothing reconciles.

  • decisions/0031-...md: status ACCEPTED, Deciders: ss-o, plus an Implementation section.
  • scripts/org-routing.py is the single generator and verifier:
    • apply: writes the generated block between org-routing markers in one checkout's AGENTS.md, or creates AGENTS.md when it is absent.
    • check: fails when a checkout disagrees with the inventory.
    • validate: checks the inventory.
    • verify-approved: confirms each approved skill digest against its recorded commit.
  • .github/instruction-surfaces.json gains a downstream section covering all fifteen consuming repositories.
    • Each entry declares the repository's routable surfaces (Copilot adapter, scoped instructions, agents, prompts, local skills), with tasks and file patterns, and the organization skills it vendors.
    • An adapter that is a symbolic link to AGENTS.md is an alias, not a surface.
  • lib/approved-skills.json records the approved revision, file list and content digest for code-review (e1f8f6c1) and zi-install (1bfbf33a).
    • The digest excludes the installer's metadata block.
    • Frontmatter key order and indentation are normalised, so neither counts as drift.
  • .github/workflows/org-routing.yml is a reusable workflow_call workflow.
    • A caller is checked against the inventory at the called workflow's own commit, so the caller's pin selects the inventory it is held to.
    • It only reports; it never writes to the caller.
    • In this repository it runs the generator tests and verify-approved instead.
  • scripts/validate-agent-policy.py validates downstream through the generator's own schema check, so the two cannot disagree. The three new files join the enforcement inventory.
  • runbooks/instruction-update.md and runbooks/org-review.md now route surface changes, skill provenance and currency checks, and advancing an approved revision to the generator.

Closes #660

Why

Every supported runtime discovers instructions from the Git root of its working directory downward, and each organization repository is its own Git root. The routing obligation that sends a runtime to .github/instruction-surfaces.json lived only in this repository's AGENTS.md, so it never arrived anywhere else.

  • Routing: of the fifteen repositories, twelve link to policy in prose only and two have no agent instructions at all. Session telemetry agrees: 188 of 225 sessions never opened the manifest.
  • Skill pins: twelve repositories vendor code-review at pins nothing reconciles.

ADR-0014 point 7 requires that mandatory policy not depend on an optional mechanism. The ADR's context has the full evidence.

Impact review

The seven answers required by runbooks/instruction-update.md are in #660. In short:

  • This is enforcement and delivery, not new policy.
  • The canonical owner is unchanged.
  • This repository's manifest gains the three new enforcement surfaces and the downstream section.
  • The private control workspace needs no new route, because its composite already imports this baseline.

Verification

All of the following ran against a clean export of the tracked tree, so local untracked workspace carriers could not affect the result. Where noted, they also ran in the checkout with full history.

python3 scripts/validate-agent-policy.py                 # passed
python3 scripts/decision-records.py --check              # passed
python3 scripts/validate-zsh-standard-policy.py          # passed
python3 -m unittest scripts/test_validate_agent_policy.py        # OK (3 new tests)
python3 -m unittest scripts/test_org_routing.py                  # 32 tests OK
python3 -m unittest scripts/test_decision_records.py             # OK
python3 -m unittest scripts/test_validate_zsh_standard_policy.py # OK
python3 scripts/org-routing.py verify-approved           # approved skill revisions verified
trunk check (12 changed files)                            # no issues

I ran the checker against checkouts of all fifteen downstream repositories:

  • The generated block is idempotent. Default prettier, markdownlint, and wiki's prettier settings (proseWrap: always) all leave it unchanged.
  • Six repositories pass after apply: wiki, z-a-default-ice, z-a-eval, zd, zsh-fancy-completions, zunit.
  • Nine fail only because they pin a content-identical older code-review commit (23fb0c86 or ce74af22). That is the pin-currency failure decision point 5 asks for.
  • src vendors zi-install from ce74af22, which is stale in both pin and content.

Mutation checks confirmed that the tests catch a disabled digest comparison and a disabled undeclared-surface check. They also exposed a missing test for surface ordering, which is now added.

Notes for review

  • Downstream adoption is out of scope here. Each repository adds the block, calls the workflow, and re-pins its vendored skills in its own separately authorized change. That work is tracked in follow-up issues.
  • ci: version reusable workflows with immutable releases #543 governs pinning for organization reusable workflows. Callers of org-routing.yml must pin a full SHA or an immutable release, not @main.
  • Workspace-delivery prerequisites stay tracked in ss-o/ws#27 and ss-o/ws#33.
  • ADR-0014 point 9 is unchanged: runtime discovery stays a manual check.

…elivery

Instruction routing reaches only this repository. Every runtime discovers
instructions from the Git root of its working directory downward, and each
organization repository is its own Git root, so the routing obligation in this
repository's AGENTS.md never arrives anywhere else. A survey of the fifteen
repositories found one that names the routing manifest, twelve that link to
policy in prose only, and two with no agent instructions at all. Of 225
recorded sessions, 188 never opened the manifest.

Twelve repositories vendor the code-review skill at pinned revisions that
nothing reconciles against canonical. Their bodies are byte-identical once
installer metadata is removed, but they sit on two different pins and neither
is current. No workflow in any repository performs that check.

ADR-0031 proposes a generated, delimited routing preamble in every repository's
AGENTS.md, produced by a single generator owned here and delivered by a
reusable workflow that also verifies vendored skill pins against a declared
approved revision. It adds no policy: it delivers and verifies rules the
canonical baseline already states.

Status is PROPOSED; only a maintainer moves a record to ACCEPTED.

Refs: #660
@ss-o
ss-o requested a review from a team as a code owner September 26, 2026 01:07
@ss-o ss-o added type:maintenance Non-feature maintenance, cleanup, or org work. area:meta Organization-wide policy, templates, or meta-repo work. labels Sep 26, 2026

@ss-o ss-o left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fallback review under ADR-0026: Copilot request not registered on ac1b403

z-shell/.github is class 4 in lib/repository-classes.yml, so ADR-0026 point 3 permits this fallback. A review_requested event was recorded on the timeline with a null reviewer and no run followed within the session, across two waits totalling roughly three minutes; requested_reviewers stayed empty. Reviewed under .github/skills/code-review/SKILL.md against code-review-generic.instructions.md.

Scope

Documentation only: one new ADR, one manifest surface, one regenerated index row. No executable code, no workflow, no shell. The dialect and lifecycle sections of the review criteria do not apply.

Deterministic checks

Re-run against a clean git archive export of HEAD plus this change, so untracked local tooling could not influence the result:

python3 scripts/validate-agent-policy.py                     passed
python3 scripts/decision-records.py --check                  passed
python3 -m unittest scripts/test_validate_agent_policy.py    92 tests, OK
python3 -m unittest scripts/test_decision_records.py         17 tests, OK

All 11 PR checks pass, Validate Agent Instructions included.

Claim verification

Every quantitative claim in the ADR was re-derived from the working tree rather than taken from the body:

  • 15 repositories: confirmed.
  • 1 repository naming the routing manifest: confirmed, z-shell/.github alone.
  • 188 of 225 sessions not reading the manifest: confirmed from audit-agent-routing.py --json (claude-code 25 of 43, codex 12 of 182; 37 read, 188 did not). The 0-of-182 codex hook-context figure is confirmed.
  • All relative links in the ADR resolve to files that exist.

One correction is needed, below.

Findings

  • Severity: IMPORTANT

  • Rule / Category: docs/accuracy

  • Location: decisions/0031-per-repository-instruction-routing-delivery.md, Context section and the survey table

  • Problem: The ADR says twelve repositories vendor the code-review skill. There are thirteen copies on disk; twelve are downstream and the thirteenth is this repository's canonical file, which is the source rather than a vendored copy. The prose is defensible but the distinction is never stated, so a reader counting files gets a different number than the ADR gives and cannot tell which is wrong.

  • Fix: State it explicitly once, for example "thirteen copies exist: this repository's canonical file and twelve vendored downstream copies." The twelve-copy claims elsewhere then read unambiguously.

  • Severity: IMPORTANT

  • Rule / Category: docs/specification-completeness

  • Location: decisions/0031-per-repository-instruction-routing-delivery.md:93 and :110

  • Problem: Point 1 requires the preamble to enumerate "that repository's own declared surfaces", and point 4 has the workflow fail "when a declared surface is missing". Only z-shell/.github has an .github/instruction-surfaces.json; no other repository has one. The ADR never says where a downstream repository's surfaces are declared: in a new per-repository manifest, in a section of the canonical manifest, or by convention from the repository's tree. An implementer cannot build the generator from this text without choosing for themselves, and the three options have materially different ownership consequences.

  • Fix: Name the source of truth in point 1. Declaring downstream surfaces inside the canonical manifest is most consistent with the rest of the decision, since it keeps one inventory and avoids fifteen new files, but the choice should be explicit either way.

  • Severity: SUGGESTION

  • Rule / Category: docs/specification-completeness

  • Location: decisions/0031-per-repository-instruction-routing-delivery.md:127

  • Problem: Point 6 says this repository "records the revision downstream repositories are expected to pin" without naming the file that holds it. lib/repository-classes.yml is the established precedent for exactly this kind of machine-readable per-repository fact.

  • Fix: Name the intended location, or state that implementation chooses it.

  • Severity: SUGGESTION

  • Rule / Category: process/coordination

  • Location: PR body, "Notes for review"

  • Problem: The dependency on #543 is noted in prose only. If #543 lands first, the caller references this decision introduces must already follow it.

  • Fix: Consider linking these as related issues so the ordering is visible on the board rather than only here.

Non-blocking observations

  • No em dash, no zdharma reference, no literal backslash-n in the ADR, the issue body, or the PR body. No agent or bot credit in the commit trailer.
  • Status is PROPOSED with Deciders: TBD, correct per runbooks/adr.md; acceptance is the maintainer's to make on main.
  • Unrelated but worth an issue: scripts/bootstrap.sh --check reports "instruction composite missing or stale" for nearly every checkout in the meta-workspace, and validate-agent-policy.py fails inside a ws worktree on the untracked .claude/rules/ws-workspace.md that the workspace tooling generates, which the validator classifies as an undeclared runtime-specific instruction carrier. Both are pre-existing and outside this diff, but the second means an agent running the validator from a worktree sees a spurious failure.

Verdict

The evidence is sound and independently reproducible, and the decision follows from it. The two IMPORTANT findings are specification gaps in a document whose purpose is to be implemented from; both are a sentence each to close. No CRITICAL findings.

Name the source of truth for downstream surfaces. Point 1 required the
preamble to enumerate a repository's declared surfaces, but only this
repository has a manifest and the record did not say where the others would be
declared. They are declared here, keyed by repository, which keeps one
inventory as ADR-0014 point 5 intends. The schema and validator cost of that
choice is recorded in Consequences.

Name the file that records the approved skill revision, following the
precedent lib/repository-classes.yml sets for per-repository facts that
tooling consumes.

Distinguish the thirteen copies of the code-review skill on disk from the
twelve vendored downstream copies, so the counts in the record can be
reproduced without guessing which file is the source.

Refs: #660
@ss-o

ss-o commented Sep 26, 2026

Copy link
Copy Markdown
Member Author

Review findings addressed in 8ae70f8. All 11 checks pass on the new head.

  • IMPORTANT, downstream surface declaration (point 1): named the source of truth. Downstream surfaces are declared in this repository's .github/instruction-surfaces.json, keyed by repository; no other repository gains a manifest. That keeps one inventory, which is what makes the manifest usable for detecting duplicate and contradictory routes under ADR-0014 point 5. The schema and validator cost is now recorded in Consequences rather than left implicit.
  • IMPORTANT, copy count: the Context section now distinguishes the thirteen copies on disk from the twelve vendored downstream copies, so the counts can be reproduced without guessing which file is the source.
  • SUGGESTION, approved revision location (point 6): named lib/repository-classes.yml or a sibling file under lib/, following the precedent that file sets for per-repository facts consumed by tooling.
  • SUGGESTION, ci: version reusable workflows with immutable releases #543 coordination: left as prose. Linking issues is a board change and is the maintainer's call, not something to do unasked inside a documentation pull request.

Re-verified against a clean git archive export of the new head: validate-agent-policy.py passed, decision-records.py --check passed, 92 + 17 unit tests OK.

Status remains PROPOSED with Deciders: TBD. Acceptance is yours to make on main, per runbooks/adr.md.

Add scripts/org-routing.py, the single generator and verifier that decision
0031 calls for. The downstream section of the instruction-surface manifest
declares each consuming repository's routable surfaces (Copilot adapter,
scoped instructions, agents, prompts, local skills) and the organization
skills it vendors. apply writes a generated routing block into a
repository's AGENTS.md between org-routing markers; check fails when the
block, the declared surfaces, or a vendored skill disagree with the
inventory.

lib/approved-skills.json records the approved revision, file list, and
content digest of each vendored organization skill. The digest excludes the
installer metadata block and normalises frontmatter key order and
indentation, so only a real content change or a stale pin is reported.
verify-approved proves each digest against the recorded commit.

The reusable Org Routing workflow runs check for a caller at the inventory of
the pinned workflow commit. The agent policy validator validates the
downstream section through the generator's own schema, so the two cannot
disagree.

Refs: #660
Record the maintainer's acceptance and the delivered implementation:
manifest section, approved-skill record, generator, reusable workflow, and
validator reuse. Route the instruction-update checklist and the org review's
skill provenance and currency checks to the deterministic generator, and
state how the approved revision advances.

Refs: #660
@ss-o ss-o changed the title docs(decisions): draft ADR-0031, per-repository instruction routing delivery feat(agents): accept and implement ADR-0031, per-repository instruction routing Sep 26, 2026
Comment thread scripts/org-routing.py Fixed
Comment thread scripts/org-routing.py Fixed
The review of the routing generator found drift that passed check:

- The workflow fell back to the caller's own commit when the called
  workflow identity was missing, which would let a caller supply the
  tooling that checks it. It now fails closed, as trunk.yml does, and
  decides the self-test path from the workflow source rather than the
  repository name, so a fork runs its self-test.
- Instruction files that a discovery glob found but no surface pattern
  accepted were dropped, and nested or runtime-specific carriers
  (CLAUDE.md, .claude/, .cursor/, nested AGENTS.md) were never seen.
  Both now fail, and scoped instructions are discovered recursively.
- The installer metadata block was excluded from the digest without
  restriction. A vendored skill now admits only the installer's keys,
  with no nesting and a consistent github-ref.
- Symbolic links inside a vendored skill were skipped. Every entry is
  now listed, links marked, so any link fails the file comparison.
- A block moved down AGENTS.md or wrapped in a fence passed. The block
  is now required at the top, after an optional title.

verify-approved also requires the approved commit to be reachable, and
paths with control characters are escaped in output. The concurrency
group is suffixed so it cannot collide with other called workflows. The
ADR records the implementation-time inventory beside the drafted survey.

Refs: #660

@ss-o ss-o left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fallback review under ADR-0026: Copilot request not registered on 33cd66e

The Copilot review request was sent after checks passed on 33cd66e9. It never registered: the timeline has no review_requested event for Copilot, only the earlier TSC team request, so the review is blocked on quota. z-shell/.github is class 4 in lib/repository-classes.yml, so this fallback review applies.

This review was run under .github/skills/code-review/SKILL.md against code-review-generic.instructions.md, on origin/main...33cd66e9. It was an independent adversarial pass by a separate agent in a fresh context.

Findings on 33cd66e: no CRITICAL, five IMPORTANT, all fixed in 79ee9d4

  1. org-routing.yml: tooling source fell back to the caller. When job.workflow_* was missing, the tooling source fell back to the caller's own commit, so a caller could supply the org-routing.py that checks it. It now fails closed, matching trunk.yml. The self-test path is chosen by comparing the workflow source with the repository, so forks run the self-test.
  2. org-routing.py: unroutable instruction files passed silently.
    • Glob hits that no surface pattern accepted (for example .github/skills/My_Skill/SKILL.md) were dropped without an error.
    • Runtime carriers were never discovered: nested AGENTS.md, CLAUDE.md, GEMINI.md, .claude/, .cursor/, .cursorrules, .github/chatmodes/.
    • All of these now fail check, and .github/instructions/ is scanned recursively.
  3. org-routing.py: metadata was excluded from the digest without limit. Any key or nesting could hide there. A vendored skill now admits only the installer's keys (METADATA_KEYS, previously unused), with no nesting and no duplicates, and requires github-ref to equal github-pinned.
  4. org-routing.py: symbolic links inside a vendored skill were skipped. Every entry is now listed, with links marked by @, so a link always fails the approved-file comparison.
  5. org-routing.py: block position was not enforced. A block moved down AGENTS.md or wrapped in a code fence passed, contrary to ADR point 1. splice now removes the block and reinserts it at the top, after an optional title, so either placement fails check.

Suggestions, also taken

  • verify-approved requires the approved commit to be an ancestor of HEAD.
  • Paths with control characters are printed with repr, so they cannot inject workflow commands.
  • The concurrency group is suffixed -org-routing, and the caller example warns against a caller-level concurrency:.
  • The ADR's Implementation section records the implementation-time inventory (fifteen repositories including zi-setup, eleven code-review copies) beside the drafted survey.

Verification on 79ee9d4

  • test_org_routing.py: 40 tests OK, locally and in CI on Python 3.10, where the self-test also ran validate and verify-approved with full history.
  • Mutation checks: disabling each new guard (the nine listed above) makes the suite fail.
  • The reviewer's probe repositories now fail on every reported bypass.
  • The downstream apply/check round trip across all fifteen repositories adds lines only. Six are current, eight fail only on stale pins, and src also fails on drifted zi-install content, which is genuine.
  • validate-agent-policy.py, decision-records.py --check, and the zsh standard validator pass, along with all four unit suites, on a clean export.
  • trunk check on the changed files reports no issues.
  • All PR checks pass.

No open findings remain. Residual gap: the reusable workflow's caller path (self == false) cannot run until a downstream repository calls it. That first run is part of the adoption follow-ups.

Code quality flagged the adjacent string literals in the routing block
list as possible missing commas. Build each multi-part line as a named
value joined with an explicit +. The rendered block is byte-identical
across all fifteen downstream inventories.

Refs: #660
@ss-o
ss-o merged commit 58b42bd into main Sep 26, 2026
12 checks passed
@ss-o
ss-o deleted the feature-660-agent-routing-delivery branch September 26, 2026 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:meta Organization-wide policy, templates, or meta-repo work. type:maintenance Non-feature maintenance, cleanup, or org work.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Instruction routing does not reach any repository except this one, and vendored skill pins are never verified

1 participant