feat: cross-file reference validation for BMB source files and generated output

[arcaven]

Describe your idea

Add cross-file reference validation to BMad Builder in two layers:

1. CI validation of BMB's source files — catch broken references in BMB's 165 source files before they ship to users
2. Post-generation validation of BMB output — when BMB creates an agent, workflow, or module, validate that all file references in the generated output resolve to real files

This builds on the cross-file reference validator proposed for BMAD-METHOD (bmad-code-org/BMAD-METHOD#1494), extending it with an --installed mode that validates _bmad/ directories directly.

Why is this needed?

We ran the BMAD-METHOD validator against BMB — here's what it found

Metric	BMAD-METHOD	BMB
Source files scanned	217	159
References checked	483	442
Refs per file	2.2	2.8
Broken references found	5	154
Files with issues	4	69

The 154 broken references break down as:

Category	Count	Nature
Cross-module refs (core/, bmm/, cis/)	95	Not bugs — resolve at install time. Validator needs --skip-external.
Wrong relative path depth	20	Genuine bugs — ../../data/ should be ../data/ in module builder steps. Same class as BMAD-METHOD #1495/#1496.
Self-module path mapping	15	Validator mapping needs adjustment for BMB's source layout.
Template/doc placeholders	11	Not bugs — step-[N]-[name].md etc. in templates.
Example agent references	5	Illustrative, not resolvable.
Misnamed step reference	1	step-01-validate.md referenced but actual file is step-01-load-target.md.
Missing KB CSV files	3	Referenced in agent YAML but don't exist in source.
Absolute path leaks	3	/Users/ paths in documentation examples.

The 20 wrong-relative-path bugs affect every step file in the module builder's steps-b/, steps-c/, steps-e/, and steps-v/ directories — ../../data/ resolves one level too high; the correct path is ../data/.

In BMAD-METHOD's closed issue history (422 issues, 236 bugs), broken file references and path handling account for 59 bugs (25%) and 129+ comments. The same reference patterns and bug classes appear in BMB.

Generated output is not validated for reference correctness

BMB generates agent YAML, workflow markdown, and module configs containing file references (exec: paths, conversational_knowledge KB paths, nextStepFile chains, cross-module refs). BMB's Zod schema validation checks YAML structure but not whether referenced files exist. Users discover broken references at runtime when an LLM tries to load a missing file.

How should it work?

Roadmap

flowchart LR
    subgraph "Step 0 — Prerequisite"
        Z["Reopen & merge<br/>BMAD-METHOD #1493 + PR #1494"]
    end

    subgraph "Step 1 — BMAD-METHOD CI"
        A[validate-file-refs.js] -->|source mode| B["src/ files<br/>~480 refs"]
        B --> C{"Broken?"}
        C -->|Yes| D["⚠️ Warning in build log"]
        C -->|No| E["✓ OK"]
    end

    subgraph "Step 2 — PR Surfacing"
        D --> F["::warning:: annotations<br/>on Files Changed tab"]
        D --> G["PR review comment<br/>(like CodeRabbit)"]
    end

    subgraph "Step 3 — BMB CI"
        A -->|source mode| H["BMB src/ files<br/>~442 refs"]
        H --> I{"Broken?"}
        I -->|Yes| F
        I -->|Yes| G
    end

    subgraph "Step 4 — BMB Output"
        A -->|installed mode| J["User's _bmad/<br/>generated content"]
        J --> K{"Broken?"}
        K -->|Yes| L["🔍 Immediate feedback<br/>to user at creation time"]
    end

    Z --> A

Loading

Each step is a separate PR. Steps 2–4 build on Step 1 but don't depend on each other.

Sprint outline

Step	Work items	Est.
0. Prerequisite	Reopen BMAD-METHOD #1493 (closed due to misunderstanding — clarifying comment posted). Merge validator PR #1494 and fix PRs #1497, #1498, #1499.	—
1. BMAD-METHOD CI	Already complete. Validator runs in validate CI job, warning-only (exit 0).	Done
2. PR surfacing	Add ::warning:: annotations (1 pt), $GITHUB_STEP_SUMMARY (1 pt), PR review comment via workflow_run (3 pts).	5 pts
3. BMB CI integration	Add --installed mode to validator (1 pt). Add --skip-external for cross-module refs (2 pts). Add validate:refs CI step to BMB (2 pts). Surface findings in BMB PRs (2 pts).	7 pts
4. BMB output validation	Post-generation validation step in agent builder workflow (2 pts), workflow builder (2 pts), module builder (2 pts).	6 pts
5. Strict enforcement	Promote to --strict (exit 1) in both repos once existing broken refs are fixed.	1 pt
	Total	19 pts

Unified tool approach

Both CI and output validation use the same validator with a mode flag:

• --source (default): maps {project-root}/_bmad/X → src/X for CI validation
• --installed: checks _bmad/X directly for installed/generated content
• --skip-external: skip cross-module refs that resolve at install time

The reference extraction patterns (regex for {project-root}/_bmad/, relative paths, exec, nextStepFile, etc.) are shared. Only the path resolution differs.

What this does NOT do

• Does not validate runtime variable interpolation ({{mustache}}, {installed_path}, etc.)
• Does not modify any generated files — read-only validation
• Does not change existing CI checks or test behavior
• Does not require new dependencies (uses node:fs, node:path, and yaml)

PR

We're contributing this work. The BMAD-METHOD validator (PR bmad-code-org/BMAD-METHOD#1494) is ready for review. Once merged, we'll submit PRs for steps 2–4 following the one-fix-per-PR convention.

Additional context

• BMAD-METHOD feature request: feat: cross-file reference validator for BMAD source files BMAD-METHOD#1493 (closed due to misunderstanding — clarifying comment posted, awaiting response)
• BMAD-METHOD validator PR: feat: cross-file reference validator for BMAD source files BMAD-METHOD#1494 (open, passing CI)
• Broken ref fix PRs found by the validator: fix: correct relative path to prd-purpose.md in step-11-polish BMAD-METHOD#1497, #1498, #1499
• BMAD-METHOD historical analysis: 422 closed issues classified, 289 unique broken refs tracked across 26 release tags (alpha.0 → Beta.2)