chore(tools): align Augment config with skill-validator#2016
Conversation
…rce of truth Replace duplicated workflow-era rules in .augment/code_review_guidelines.yaml with a single reference to tools/skill-validator.md. Append the skill spec cheatsheet to the validator with a link to the Agent Skills specification. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
🤖 Augment PR SummarySummary: Aligns Augment code review guidance with the skill validator as the canonical ruleset. Changes:
Notes: Centralizes validation requirements to reduce drift between configuration and the validator documentation. 🤖 Was this summary useful? React with 👍 or 👎 |
![augmentcode[bot]](https://avatars.githubusercontent.com/in/1027498?s=60&v=4){kind=small}
📝 WalkthroughWalkthroughA "Skill Spec Cheatsheet" reference guide is added to the Skill Validator documentation, providing structure, fields, guidance, and examples for users. The content is inserted in both the primary guidance section and the report template section. This is a static documentation addition with no algorithmic or control flow changes. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Tip CodeRabbit can use oxc to improve the quality of JavaScript and TypeScript code reviews.Add a configuration file to your project to customize how CodeRabbit runs oxc. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tools/skill-validator.md`:
- Around line 326-398: The validator file currently includes a long "Skill Spec
Cheatsheet" reference that bloats the validation prompt; extract the detailed
prose (e.g., sections like "Extra frontmatter fields (Claude Code only)",
"String substitutions", "Dynamic context injection", "Subagent execution",
"Practical tips", and long examples) into a separate linked reference document
and reduce tools/skill-validator.md to a concise, strict checklist containing
only required rules (frontmatter field constraints, path resolution, progressive
disclosure summary, invocation control matrix and context budget) plus a single
"See: [detailed reference]" link; update any code that builds the validator
prompt to load only the trimmed SKILL.md content (or the concise checklist
symbol names above) so token usage and rule application remain tight.
- Around line 355-357: The documentation currently claims "Same SKILL.md format
works on Claude Code, Claude API, Claude.ai, Agent SDK" but also lists Claude
Code-only frontmatter fields without stating cross-platform behavior; update the
SKILL.md text to clarify either that "same format" only guarantees base fields
(name, description, body) portability or explicitly document how unknown or
platform-specific frontmatter fields are handled (e.g., ignored, cause
validation errors, or produce warnings) across Claude Code, Claude API,
Claude.ai, and Agent SDK; reference the frontmatter section and the example
Claude Code-only fields and add a short sentence specifying which approach was
chosen and the runtime/validation behavior on non-Claude Code surfaces.
- Around line 329-330: Add a short "Last verified against" note next to the
unversioned "Agent Skills specification" link: update the markdown around the
link text "Agent Skills specification" to include either the spec version or a
verification date (e.g., "Last verified against: YYYY-MM-DD (spec vX)") so
readers know which spec revision the validator was checked against and avoid
silent drift.
- Around line 375-376: Update the cheatsheet entry describing the ``!`command`
`` shell-execution syntax to clarify it only applies to Claude Code surfaces: in
the "Extra frontmatter fields (Claude Code only)" section (the lines describing
``!`command` ``), add a brief caveat such as "Claude Code surfaces only; verify
support before relying on this feature" and note that security constraints for
shell execution are the responsibility of the Claude Code runtime/deployment
rather than the skill descriptor; ensure this text sits next to the existing
``!`command` `` description so readers understand it's not part of BMAD-METHOD
or other runtimes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 98ffbf51-3447-4414-9208-aa49d75a76fb
⛔ Files ignored due to path filters (1)
.augment/code_review_guidelines.yamlis excluded by!.augment/**
📒 Files selected for processing (1)
tools/skill-validator.md
| For the full standard, see: [Agent Skills specification](https://agentskills.io/specification) | ||
|
|
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's check the file structure and read the specific file
git ls-files tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 93
🏁 Script executed:
# Read the file around the mentioned lines
wc -l tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 97
🏁 Script executed:
# Get context around lines 329-330
sed -n '320,340p' tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 1005
🏁 Script executed:
# Search for other references to the Agent Skills specification or versioning
rg -i "agent.*skill|specification|version" tools/skill-validator.md -A 2 -B 2Repository: bmad-code-org/BMAD-METHOD
Length of output: 802
🏁 Script executed:
# Check if there are any other spec links or version pinning in related files
rg -i "agentskills\.io|spec.*version|version.*spec" --type md -A 2 -B 2Repository: bmad-code-org/BMAD-METHOD
Length of output: 8852
🏁 Script executed:
# Search for "Agent Skills" mentions throughout the codebase
rg -i "agent.*skills|skills.*specification" --type md --type yaml --type json -B 2 -A 2Repository: bmad-code-org/BMAD-METHOD
Length of output: 21470
🏁 Script executed:
# Check if there are any version references related to this specification
rg -i "skill.*version|specification.*version|agentskills" --type mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 1240
🏁 Script executed:
# Look at the full context of the skill-validator.md file to see if versions are mentioned anywhere
cat -n tools/skill-validator.md | head -50Repository: bmad-code-org/BMAD-METHOD
Length of output: 2526
🏁 Script executed:
# Check if there's any documentation about how this validator relates to spec versions
rg -i "validate.*spec|spec.*validate|validator" tools/skill-validator.md -B 3 -A 3Repository: bmad-code-org/BMAD-METHOD
Length of output: 193
🏁 Script executed:
# Check if skill-validator.md itself mentions spec versioning anywhere
cat tools/skill-validator.md | grep -i "version\|dated\|release" | head -20Repository: bmad-code-org/BMAD-METHOD
Length of output: 240
🏁 Script executed:
# Search for any explicit spec version tracking in documentation
rg -i "agent.*skills.*version|specification.*version" --type md -B 2 -A 2Repository: bmad-code-org/BMAD-METHOD
Length of output: 900
🏁 Script executed:
# Check the distillate-format-reference for more context on the spec age/versioning
sed -n '1,100p' src/core/skills/bmad-distillator/resources/distillate-format-reference.md | grep -i "spec\|version" -B 2 -A 2Repository: bmad-code-org/BMAD-METHOD
Length of output: 874
🏁 Script executed:
# Check CHANGELOG for any mentions of spec version tracking or updates
rg -i "agent.*skill.*spec|spec.*version" CHANGELOG.md -B 2 -A 2Repository: bmad-code-org/BMAD-METHOD
Length of output: 676
🏁 Script executed:
# Check if there are any comments or documentation about the Agent Skills spec version itself
rg -i "agentskills" tools/skill-validator.md -B 5 -A 10Repository: bmad-code-org/BMAD-METHOD
Length of output: 904
🏁 Script executed:
# Look for any examples or documentation about how to reference external specs with versions
rg -i "external.*spec|spec.*external|reference.*external" tools/ -A 3 -B 3 | head -40Repository: bmad-code-org/BMAD-METHOD
Length of output: 2135
🏁 Script executed:
# Check if the validator mentions anything about validating against a specific spec version
sed -n '1,150p' tools/skill-validator.md | cat -nRepository: bmad-code-org/BMAD-METHOD
Length of output: 9898
Consider adding a spec version reference or last-verified date for clarity.
The link to Agent Skills specification is unversioned. While the spec itself is young and stable, adding a comment like "Last verified against: [date or spec version]" would align with the codebase's emphasis on version precision and prevent silent spec drift from causing validation logic mismatches in the future.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tools/skill-validator.md` around lines 329 - 330, Add a short "Last verified
against" note next to the unversioned "Agent Skills specification" link: update
the markdown around the link text "Agent Skills specification" to include either
the spec version or a verification date (e.g., "Last verified against:
YYYY-MM-DD (spec vX)") so readers know which spec revision the validator was
checked against and avoid silent drift.
tools/skill-validator.md
Outdated
| - Same SKILL.md format works on Claude Code, Claude API, Claude.ai, Agent SDK | ||
| - Runtime differs per surface: Code has full network access, API has none, claude.ai varies | ||
| - Skills don't sync across surfaces — deploy separately to each |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Read the specific lines from the file
wc -l tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 97
🏁 Script executed:
# Read lines around 355-367 to see the context
sed -n '350,370p' tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 1178
🏁 Script executed:
# Read a wider context to understand the full section
sed -n '345,375p' tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 1787
🏁 Script executed:
# Search for any guidance about handling platform-specific fields or unknown fields
rg -A 3 -B 3 "unknown.*field|unsupported.*field|ignore.*field|error.*field" tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 51
🏁 Script executed:
# Check if there's documentation about how other platforms handle Claude Code-only fields
rg -i "ignore|graceful|unknown|unsupported|deprecated" tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 51
🏁 Script executed:
# Look for any clarification about the base format vs extra fields
rg -B 5 -A 5 "frontmatter\|format" tools/skill-validator.md | head -50Repository: bmad-code-org/BMAD-METHOD
Length of output: 51
🏁 Script executed:
# Check if there are other platform-specific sections
rg "Claude API|claude.ai|Agent SDK" tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 238
Clarify cross-platform field behavior and align "same format" language with documented platform constraints.
Line 355 claims "Same SKILL.md format works on Claude Code, Claude API, Claude.ai, Agent SDK," but lines 359–367 introduce Claude Code-only frontmatter fields with no guidance on how other platforms handle them. The section header is clear these fields are platform-specific, but the portability claim is misleading without explicitly stating whether unknown fields are ignored, cause validation errors, or have other behavior. Add either: (1) clarification that "same format" refers only to base fields (name, description, body); or (2) documented behavior for cross-platform field handling to prevent users from deploying invalid skills to other surfaces.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tools/skill-validator.md` around lines 355 - 357, The documentation currently
claims "Same SKILL.md format works on Claude Code, Claude API, Claude.ai, Agent
SDK" but also lists Claude Code-only frontmatter fields without stating
cross-platform behavior; update the SKILL.md text to clarify either that "same
format" only guarantees base fields (name, description, body) portability or
explicitly document how unknown or platform-specific frontmatter fields are
handled (e.g., ignored, cause validation errors, or produce warnings) across
Claude Code, Claude API, Claude.ai, and Agent SDK; reference the frontmatter
section and the example Claude Code-only fields and add a short sentence
specifying which approach was chosen and the runtime/validation behavior on
non-Claude Code surfaces.
tools/skill-validator.md
Outdated
| - `` !`command` `` syntax runs shell commands at load time, output replaces placeholder | ||
| - Runs before Claude sees content — pure preprocessing |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, locate the file and check its existence
find . -name "skill-validator.md" -type fRepository: bmad-code-org/BMAD-METHOD
Length of output: 95
🏁 Script executed:
# Read the specific lines mentioned in the review
if [ -f "./tools/skill-validator.md" ]; then
wc -l ./tools/skill-validator.md
sed -n '370,385p' ./tools/skill-validator.md
fiRepository: bmad-code-org/BMAD-METHOD
Length of output: 839
🏁 Script executed:
# Get broader context around the command-substitution section
if [ -f "./tools/skill-validator.md" ]; then
sed -n '360,400p' ./tools/skill-validator.md
fiRepository: bmad-code-org/BMAD-METHOD
Length of output: 1905
🏁 Script executed:
# Search for implementation of the !`command` feature
rg '!`' tools/ --type md -B 2 -A 2 | head -50Repository: bmad-code-org/BMAD-METHOD
Length of output: 368
🏁 Script executed:
# Search for code that handles command substitution
rg -t js -t ts 'command\|substitut' tools/ --max-count 20 | head -40Repository: bmad-code-org/BMAD-METHOD
Length of output: 51
🏁 Script executed:
# Look for where this feature is actually implemented/parsed
fd . tools/ -name "*.js" -o -name "*.ts" | head -20Repository: bmad-code-org/BMAD-METHOD
Length of output: 299
🏁 Script executed:
# Search more broadly for how placeholders/injection is handled
rg 'load time|preprocessing|shell' tools/ -t mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 261
🏁 Script executed:
# Search for the skill-validator implementation itself
fd skill-validator -t js -t ts -t pyRepository: bmad-code-org/BMAD-METHOD
Length of output: 238
🏁 Script executed:
# Look for code that parses or validates skills
rg 'skill|SKILL' tools/ -t js -t ts --max-count 15 | grep -i 'parse\|load\|read\|valid'Repository: bmad-code-org/BMAD-METHOD
Length of output: 1377
🏁 Script executed:
# Check if there's any code handling dynamic context or command substitution
rg 'Dynamic context|command.*substitut|!`' --type js --type ts tools/Repository: bmad-code-org/BMAD-METHOD
Length of output: 216
🏁 Script executed:
# Search across entire repo for implementation
rg 'Dynamic context|command.*substitut' --type js --type tsRepository: bmad-code-org/BMAD-METHOD
Length of output: 51
🏁 Script executed:
# Look at git history of this specific line to understand when it was added
cd tools && git log --oneline skill-validator.md | head -10Repository: bmad-code-org/BMAD-METHOD
Length of output: 159
🏁 Script executed:
# Look at the overall structure and purpose of skill-validator.md
head -50 tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 2176
🏁 Script executed:
# Search for any references to command execution or shell in the entire codebase
rg 'exec|spawn|shell' --type js | head -20Repository: bmad-code-org/BMAD-METHOD
Length of output: 1432
🏁 Script executed:
# Look for where skills are actually loaded/processed
rg 'skill' tools/cli/installers/lib/ide/shared/*.js | grep -i 'load\|read\|process'Repository: bmad-code-org/BMAD-METHOD
Length of output: 1039
🏁 Script executed:
# Check if there's documentation or README about what skill-validator is
fd README tools/ -o fd '*.md' tools/ | head -10Repository: bmad-code-org/BMAD-METHOD
Length of output: 192
🏁 Script executed:
# Broader search: look for any command execution in skill loading
rg '`' tools/cli/installers/ --max-count 10 | head -20Repository: bmad-code-org/BMAD-METHOD
Length of output: 2700
🏁 Script executed:
# Check if the skill-validator.md is meant as an LLM prompt for understanding skills
grep -i 'inference\|llm\|prompt\|how to use' tools/skill-validator.md | head -5Repository: bmad-code-org/BMAD-METHOD
Length of output: 414
🏁 Script executed:
# Search for any actual implementation or example of !`command` syntax being used
rg '!`' --type mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 181
🏁 Script executed:
# Look for skill examples or templates that might use this syntax
find . -name 'SKILL.md' -o -name '*skill*example*' -o -name '*sample*skill*' | head -10Repository: bmad-code-org/BMAD-METHOD
Length of output: 753
🏁 Script executed:
# Check if skill loading code does any command substitution or shell execution
rg 'command\|shell\|exec\|spawn' tools/cli/installers/lib/ide/shared/ -A 3 -B 1Repository: bmad-code-org/BMAD-METHOD
Length of output: 51
🏁 Script executed:
# Verify the actual features supported by checking skill loading and variable expansion
cat tools/cli/installers/lib/ide/shared/skill-manifest.js | head -100Repository: bmad-code-org/BMAD-METHOD
Length of output: 4071
🏁 Script executed:
# Check if !` command syntax might be handled at CLI/installer level during skill installation
rg -i 'backtick|!`|command.substitut' tools/cli/ -A 2 -B 2 | head -50Repository: bmad-code-org/BMAD-METHOD
Length of output: 732
🏁 Script executed:
# Look for any template processing or variable expansion in the IDE/installer
rg 'interpolat|expand|substitut' tools/cli/installers/ -A 2 -B 1 | head -40Repository: bmad-code-org/BMAD-METHOD
Length of output: 2859
🏁 Script executed:
# Search for any placeholder or dynamic content processing
rg '\{\{|\$\{' tools/cli/installers/lib/ide/ --type js | head -20Repository: bmad-code-org/BMAD-METHOD
Length of output: 2771
🏁 Script executed:
# Check if the feature is mentioned in any other docs or comments
rg '!`|dynamic.*context|load.*time' . --type mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 1657
🏁 Script executed:
# Look at more context from skill-validator.md around the command substitution section
sed -n '350,390p' tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 2004
🏁 Script executed:
# Search for broader context about Claude Code skill execution
rg 'Claude Code|Claude API|claude.ai' tools/skill-validator.md -B 2 -A 2Repository: bmad-code-org/BMAD-METHOD
Length of output: 737
🏁 Script executed:
# Check if there's any documentation about what skill-validator.md is documenting
head -20 tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 813
🏁 Script executed:
# Look for any mentions of this being aspirational or context-dependent
rg 'Claude Code only|runtime differs|depends|varies' tools/skill-validator.md -B 1 -A 1Repository: bmad-code-org/BMAD-METHOD
Length of output: 677
🏁 Script executed:
# Check the git history to understand when this feature was documented
cd tools && git log --oneline -p skill-validator.md | grep -A 10 -B 10 '!`command`' | head -40Repository: bmad-code-org/BMAD-METHOD
Length of output: 1055
🏁 Script executed:
# Verify the document's actual scope and purpose from its preamble
head -40 tools/skill-validator.md | tail -20Repository: bmad-code-org/BMAD-METHOD
Length of output: 1012
🏁 Script executed:
# Check if this is documenting BMAD-METHOD behavior or external Agent Skills spec
rg 'Agent Skills|specification|agentskills.io|open standard' tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 329
🏁 Script executed:
# Look for any security or safety guidance elsewhere in the document
rg -i 'security|secret|credential|safe|dangerous|risk|warning|caution' tools/skill-validator.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 184
Clarify that !command`` security considerations apply only to Claude Code deployments.
Lines 375-376 document a Claude Code-specific feature (per the "Extra frontmatter fields (Claude Code only)" context), not a BMAD-METHOD implementation. The cheatsheet is a quick reference for the Agent Skills open standard, which already notes that "Runtime differs per surface: Code has full network access, API has none, claude.ai varies."
Adding a brief caveat stating "Claude Code surfaces only; verify support before relying on this feature" would be appropriate. However, this is a documentation inconsistency in a reference section, not a critical security footgun in BMAD-METHOD code. If expanding the guidance, note that security constraints for shell execution are the responsibility of the Claude Code runtime and deployment context, not the skill descriptor itself.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tools/skill-validator.md` around lines 375 - 376, Update the cheatsheet entry
describing the ``!`command` `` shell-execution syntax to clarify it only applies
to Claude Code surfaces: in the "Extra frontmatter fields (Claude Code only)"
section (the lines describing ``!`command` ``), add a brief caveat such as
"Claude Code surfaces only; verify support before relying on this feature" and
note that security constraints for shell execution are the responsibility of the
Claude Code runtime/deployment rather than the skill descriptor; ensure this
text sits next to the existing ``!`command` `` description so readers understand
it's not part of BMAD-METHOD or other runtimes.
Keep cheatsheet focused on the Agent Skills open standard only. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 participant
Summary
.augment/code_review_guidelines.yamlwith a single reference totools/skill-validator.mdas the single source of truthTest plan
🤖 Generated with Claude Code