Skip to content

fix(create-story): restore validate-workflow task and harden checklist execution for conditional contexts #1652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pure-maple wants to merge 6 commits into from
Closed

fix(create-story): restore validate-workflow task and harden checklist execution for conditional contexts #1652

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1b57ae9
fix(create-story): restore validate-workflow executor and conditional…
Feb 14, 2026
9e4e376
fix(create-story): address validator determinism and bot feedback
pure-maple Feb 14, 2026
d098517
fix(create-story): clarify required vars and metadata prompt flow
pure-maple Feb 14, 2026
99c7883
fix(create-story): address coderabbit full-review findings
pure-maple Feb 14, 2026
bcb2ec5
fix(validate-workflow): clarify critical vs non-critical remediation …
pure-maple Feb 14, 2026
0db123d
fix(validate): remove pass halt and add parse/report failure safeguards
pure-maple Feb 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions src/bmm/workflows/4-implementation/create-story/checklist.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ This is a COMPETITION to create the **ULTIMATE story context** that makes LLM de

- The `{project-root}/_bmad/core/tasks/validate-workflow.xml` framework will automatically:
- Load this checklist file
- Load the newly created story file (`{story_file_path}`)
- Load the story file (`{story_file}` when provided, otherwise `{default_output_file}`)
- Load workflow variables from `{installed_path}/workflow.yaml`
- Execute the validation process

Expand All @@ -51,7 +51,7 @@ This is a COMPETITION to create the **ULTIMATE story context** that makes LLM de
- **Story file**: The story file to review and improve
- **Workflow variables**: From workflow.yaml (implementation_artifacts, epics_file, etc.)
- **Source documents**: Epics, architecture, etc. (discovered or provided)
- **Validation framework**: `validate-workflow.xml` (handles checklist execution)
- **Validation framework**: `validate-workflow.xml` (handles checklist execution and report generation)

---

Expand All @@ -62,10 +62,20 @@ You will systematically re-do the entire story creation process, but with a crit
### **Step 1: Load and Understand the Target**

1. **Load the workflow configuration**: `{installed_path}/workflow.yaml` for variable inclusion
2. **Load the story file**: `{story_file_path}` (provided by user or discovered)
2. **Load the story file**: `{story_file}` first, fallback to `{default_output_file}` (or explicit `{document}` input)
3. **Load validation framework**: `{project-root}/_bmad/core/tasks/validate-workflow.xml`
4. **Extract metadata**: epic_num, story_num, story_key, story_title from story file
5. **Resolve all workflow variables**: implementation_artifacts, epics_file, architecture_file, etc.
4. **Resolve variables deterministically**:
- Load config_source file if present
- Parse workflow.yaml key/value pairs
- For any value matching `{config_source}:key`, resolve from the loaded config source
- Resolve system path variables (for example `{project-root}`, `{installed_path}`) in every path value
- Resolve system-generated values (for example `{date}`) using current execution context
- Required for this checklist flow: `{epics_file}`, `{architecture_file}`, `{implementation_artifacts}`, `{project-root}`, `{installed_path}`, and at least one story locator (`{story_file}` or `{default_output_file}`)
- Optional/fallback-capable values: validation `{checklist}` input and validation `{report}` input
- Validation task input contract: `workflow` is required; `checklist`, `document`, and `report` are optional with deterministic fallback
- Note: create-story invoke-task passes `document={default_output_file}` explicitly, which overrides fallback discovery
- If any required value remains unresolved, stop and request explicit user input before continuing
5. **Extract metadata**: epic_num, story_num, story_key, story_title from story file
6. **Understand current status**: What story implementation guidance is currently provided?

**Note:** If running in fresh context, user should provide the story file path being reviewed. If running from create-story workflow, the validation framework will automatically discover the checklist and story file.
Expand Down
19 changes: 7 additions & 12 deletions src/bmm/workflows/4-implementation/create-story/instructions.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -280,21 +280,16 @@
<template-output file="{default_output_file}">testing_requirements</template-output>

<!-- Previous story intelligence -->
<check
if="previous story learnings available">
<template-output file="{default_output_file}">previous_story_intelligence</template-output>
</check>
<action>If previous story learnings are unavailable (for example this is story 1 in epic), set previous_story_intelligence to the canonical format: N/A: {one-line reason}</action>
<template-output file="{default_output_file}">previous_story_intelligence</template-output>

<!-- Git intelligence -->
<check
if="git analysis completed">
<template-output file="{default_output_file}">git_intelligence_summary</template-output>
</check>
<action>If git analysis is unavailable or not relevant, set git_intelligence_summary to the canonical format: N/A: {one-line reason}</action>
<template-output file="{default_output_file}">git_intelligence_summary</template-output>

<!-- Latest technical specifics -->
<check if="web research completed">
<template-output file="{default_output_file}">latest_tech_information</template-output>
</check>
<action>If web research was not completed or not needed, set latest_tech_information to the canonical format: N/A: {one-line reason}</action>
<template-output file="{default_output_file}">latest_tech_information</template-output>

<!-- Project context reference -->
<template-output
Expand All @@ -311,8 +306,8 @@
</step>

<step n="6" goal="Update sprint status and finalize">
<invoke-task>Validate against checklist at {installed_path}/checklist.md using _bmad/core/tasks/validate-workflow.xml</invoke-task>
<action>Save story document unconditionally</action>
<invoke-task>Run {project-root}/_bmad/core/tasks/validate-workflow.xml with workflow={installed_path}/workflow.yaml checklist={installed_path}/checklist.md document={default_output_file}</invoke-task>

<!-- Update sprint status -->
<check if="sprint status file exists">
Expand Down
179 changes: 179 additions & 0 deletions src/core/tasks/validate-workflow.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
<task id="_bmad/core/tasks/validate-workflow.xml"
name="Validate Workflow Output"
description="Run a workflow checklist against a target document and produce an evidence-backed validation report">

<objective>Validate a generated document against checklist requirements with deterministic variable resolution and produce an actionable pass/fail report</objective>

<inputs>
<input name="workflow" required="true" desc="Workflow yaml path used to resolve variables and checklist location" />
<input name="checklist" required="false" desc="Checklist file path. Defaults to workflow.yaml validation field, then checklist.md beside workflow" />
<input name="document" required="false" desc="Document to validate. If omitted, resolve from workflow variables only" />
<input name="report" required="false" desc="Output report file path. Defaults to document folder validation-report-{timestamp_utc}.md" />
</inputs>

<llm critical="true">
<i>MANDATORY: Execute ALL steps in order. Do not skip any checklist item.</i>
<i>Always read COMPLETE files; do not sample with offsets.</i>
<i>If a file cannot be loaded in one read, read it in deterministic sequential chunks until full coverage is achieved and recorded.</i>
<i>Every non-N/A judgment must include concrete evidence from the document.</i>
<i>If a required path cannot be resolved, stop and ask for explicit user input.</i>
<i>Be strict and objective: no assumptions without evidence.</i>
<i>N/A is allowed only when an explicit conditional requirement is not applicable; never use N/A due to missing evidence.</i>
</llm>

<flow>
<step n="1" title="Load Workflow Context">
<action>Load workflow yaml from input {workflow}</action>
<action>Resolve variables in this order:
1) load config_source file if present
2) resolve all {config_source}:key references
3) resolve system path variables ({project-root}, {installed_path})
4) resolve system-generated values (date)
</action>
<action>Determine checklist path in priority order:
- explicit input {checklist}
- workflow.yaml field "validation"
- sibling file checklist.md in workflow directory
</action>
<action>Determine document path in priority order:
- explicit input {document}
- resolved variable {story_file} if present
- resolved variable {default_output_file} if present
</action>
<action if="still unresolved">Ask user: "Which document should I validate?" and WAIT</action>
<action>Normalize resolved workflow/checklist/document paths to absolute paths before loading files</action>
</step>

<step n="2" title="Load Checklist and Target Document" critical="true">
<action>Load full checklist content (use chunked sequential reads only when needed for large files, and record covered ranges)</action>
<action>Load full target document content (use chunked sequential reads only when needed for large files, and record covered ranges)</action>
<action>Extract story metadata when available (epic_num, story_num, story_id, story_key, title) from filename, heading, or frontmatter</action>
<action>Parse checklist into ordered sections and atomic validation items; assign each item a stable id (section_index.item_index)</action>
<action>Compute parsed_item_count and expected_item_count (from raw checklist requirement markers such as - [ ], - [x], and requirement list lines)</action>
<action if="expected_item_count &gt; 0 AND parsed_item_count &lt; expected_item_count * 0.9">HALT with error: "Checklist parse divergence too large (parsed={parsed_item_count}, expected~={expected_item_count}); fix checklist formatting before validation"</action>
<action>Determine critical checks from explicit signals only:
- item-level markers: [CRITICAL], critical:true, MUST FIX
- section-level markers: headings containing CRITICAL (case-insensitive), MUST FIX, MUST-FIX, or the exact phrase "Must Fix Before Proceeding"
- XML section attributes: critical="true" or critical="yes"
- do not infer criticality from generic keywords alone
</action>
<action>Detect conditional expressions in checklist items (for example: if/when/unless + variable references)</action>
<action if="no checklist items parsed">HALT with error: "Checklist is empty or unparsable"</action>
<action if="conditional items reference metadata and metadata is missing">Ask user to provide all missing referenced metadata fields (epic_num, story_num, story_id, story_key, title/story_title) and WAIT before evaluating checklist items</action>
</step>

<step n="3" title="Validate Every Checklist Item" critical="true">
<mandate>For every checklist item, evaluate one of: PASS, PARTIAL, FAIL, N/A</mandate>
<action>Initialize counters to zero before evaluation:
- pass_count, partial_count, fail_count, na_count
- critical_fail_count, critical_partial_count
- applicable_count, total_item_count, processed_item_count
- total_section_count, processed_section_count
</action>
<action>For each item:
- restate requirement in one short sentence
- if item contains explicit condition (for example "If story_num > 1") and condition is false, mark N/A with the exact reason
- locate explicit evidence in document (include line references when possible)
- consider implied coverage only when explicit text is absent
- assign verdict and rationale
- if PARTIAL/FAIL, describe impact and a concrete fix
- update all counters immediately after each verdict
</action>
<action>Process sections in deterministic order and increment processed_section_count after each section completes</action>
<action if="processed_section_count != total_section_count">HALT with error: "Validation incomplete: one or more checklist sections were not processed"</action>
<action if="processed_item_count != total_item_count">HALT with error: "Validation incomplete: one or more checklist items were not processed"</action>
<action>Compute applicable_count = pass_count + partial_count + fail_count</action>
<action>Compute pass_percent using applicable_count (if 0, set pass_percent=0)</action>
<action>Set gate_warning:
- "No applicable checklist items were evaluated; manual confirmation required" if applicable_count == 0
- "N/A" otherwise
</action>
<action>Set gate decision deterministically:
- NEEDS_REVIEW if applicable_count == 0
- FAIL if critical_fail_count &gt; 0
- FAIL if critical_partial_count &gt; 0
- FAIL if fail_count &gt; 0
- PASS otherwise
</action>
<critical>DO NOT SKIP ANY ITEM OR SECTION</critical>
</step>

<step n="4" title="Generate Validation Report">
<action>Generate timestamp values:
- timestamp_utc for filenames in YYYYMMDD-HHmmss (UTC)
- generated_at_utc for report display in ISO-8601 UTC
</action>
<action>Set report path:
- use explicit input {report} when provided
- else save to target document folder as validation-report-{timestamp_utc}.md
</action>
<action>Write report with the format below</action>
<action if="report write/save fails">HALT with error: "Validation report could not be saved"</action>

<report-format>
# Validation Report

- Document: {document}
- Checklist: {checklist}
- Workflow: {workflow}
- Date: {generated_at_utc}

## Summary
- Overall pass rate: {pass_count}/{applicable_count} ({pass_percent}%)
- Critical failures: {critical_fail_count}
- Critical partials: {critical_partial_count}
- Gate decision: {PASS|FAIL|NEEDS_REVIEW}
- Gate warning: {gate_warning}

## Section Results
### {Section Name}
- PASS: {count}
- PARTIAL: {count}
- FAIL: {count}
- N/A: {count}

For each checklist item:
- [MARK] {item}
- Evidence: {quote or line reference}
- Analysis: {why mark is correct}
- Action (if PARTIAL/FAIL): {specific remediation}

## Must Fix Before Proceeding
- {all critical FAIL and critical PARTIAL items}

## Should Improve
- {all non-critical FAIL and non-critical PARTIAL items}

## Final Recommendation
1. {highest-priority fix}
2. {second-priority fix}
3. {third-priority fix}
</report-format>
</step>

<step n="5" title="Return Decision and Halt">
<action>Present concise summary with counts and gate decision</action>
<action>Provide report path</action>
<action if="critical failures exist">State clearly that workflow should not proceed until fixes are applied</action>
<action if="gate decision == PASS">Return control immediately and continue calling workflow execution</action>
<action if="gate decision == FAIL OR gate decision == NEEDS_REVIEW">HALT and wait for user direction</action>
</step>
</flow>

<halt-conditions>
<condition>HALT if workflow file cannot be loaded</condition>
<condition>HALT if checklist file cannot be loaded</condition>
<condition>HALT if target document cannot be determined after user prompt</condition>
<condition>HALT if any checklist section is skipped</condition>
<condition>HALT if validation report cannot be saved</condition>
</halt-conditions>

<critical-rules>
<rule>Never skip checklist items</rule>
<rule>Every PASS/PARTIAL/FAIL must have evidence</rule>
<rule>Use deterministic variable resolution before asking the user</rule>
<rule>Always save a validation report file</rule>
<rule>N/A is valid only for explicit conditional non-applicability</rule>
<rule>Criticality must come from explicit checklist markers or critical sections</rule>
</critical-rules>
</task>