spec rendering does not aggregate scope vBRIEFs from lifecycle folders (v0.20 model)

[MScottAdams]

Summary

In the v0.20 vBRIEF-centric model, the project "spec" is defined as the union of vbrief/specification.vbrief.json (intent/architecture narratives) plus the scope vBRIEFs in lifecycle folders (vbrief/proposed/, pending/, active/, completed/, cancelled/). But task spec:render (scripts/spec_render.py) reads only the single file passed as argv[1] -- it never walks the lifecycle folders. The rendered SPECIFICATION.md is therefore incomplete for any project using the post-cutover model: it shows project intent but not the actual scopes / work units that constitute the implementation plan.

The data model from the v0.20 cutover is in place (lifecycle folders, planRef, references[], PROJECT-DEFINITION.vbrief.json items registry). The rendering layer is not. This is a half-shipped piece of #338 / Phase 2.

Reproduction

1. Greenfield or migrated project on v0.20+. vbrief/specification.vbrief.json exists with status approved. One or more scope vBRIEFs in vbrief/pending/ and/or vbrief/active/.
2. Run task spec:render.
3. Observe: SPECIFICATION.md contains only the contents of specification.vbrief.json. None of the scope vBRIEFs in the lifecycle folders are mentioned. A reader of SPECIFICATION.md cannot see what work has been scoped, accepted, or is in progress.

task roadmap:render walks pending/ and produces ROADMAP.md correctly. task project:render walks lifecycle folders to refresh PROJECT-DEFINITION items registry. No renderer in deft today aggregates lifecycle folders into SPECIFICATION.md.

Why this is its own issue (vs. #434)

#434 is a renderer bug specific to speckit narrative shape -- spec_render drops 9 of 10 narratives. That fix is local to one script and one model.

This issue is broader and applies to every vBRIEF-centric project, not just speckit. Even a perfectly-shaped Model-1-style spec with full Overview + items would still produce an incomplete SPECIFICATION.md once scopes are split out into lifecycle folders, because nothing pulls them back in.

Fixing #434 alone leaves this gap. Fixing this alone leaves #434.

Proposed fix (sketch)

Two reasonable paths -- pick one, or do both:

Option 1: Aggregator inside task spec:render

Extend scripts/spec_render.py to optionally walk vbrief/pending/ and vbrief/active/ (and optionally completed/ with status pins) and render each scope vBRIEF as a section under "Implementation Plan / Scopes" in the output. Sketch:

# {project title}

## {narrative keys, in order}
...

## Implementation Plan

### Pending
- [scope-slug] -- title -- summary narrative
  - items / acceptance criteria

### Active
- [scope-slug] -- title -- ...

### Completed (last N)
- [scope-slug] -- title (completed YYYY-MM-DD)

Trigger via a flag (e.g. --include-scopes default on for v0.20+) so legacy Model-1 projects still get the existing behavior.

Option 2: Hub document, multiple thin renderers

Keep task spec:render thin (just specification.vbrief.json). Keep task roadmap:render for pending/. Add task plan:render (per-scope or aggregated). Add a top-level "spec hub" markdown that links to all of them with status counts. Multiple small files, not one fat one.

Either way: the v0.20 cutover is incomplete without one of these. Today users on v0.20 see thinner SPECIFICATION.md exports than they did pre-cutover for the same amount of project content.

Severity

Architectural gap. Affects all v0.20+ projects, not just speckit. The data exists and is correct; only the rendering layer is missing. Fix is medium-sized but well-scoped.

Related

• Phase 2: vBRIEF Architecture Cutover -- master frozen #338 -- Phase 2 vBRIEF cutover tracker (this is a half-shipped piece of that effort).
• Auto-render PRD.md and SPECIFICATION.md when specification.vbrief.json changes #398 / PR fix(scripts,skills): auto-render staleness warning + pre-pr auto-render step (#398) #400 -- staleness/auto-render hooks; assumes a renderer exists that produces complete output.
• speckit Phase 3 -> Phase 4 advances without enforcing ! task spec:render #432 -- speckit Phase 3 -> 4 transition does not enforce task spec:render.
• Greenfield projects can complete speckit with no PRD/SPECIFICATION export and no prompt #433 -- greenfield projects can complete speckit with no PRD/SPEC export and no prompt.
• spec_render.py is item-centric and silently drops all speckit narratives #434 -- spec_render.py drops speckit narratives (renderer bug, narrower scope than this).
• docs(readme): Document Generation table missing task prd:render row #420 -- README discoverability gap for task prd:render.

Discovered

During RC2 walkthrough; investigated why SPECIFICATION.md was empty after speckit and traced the issue back to the broader pattern that no renderer aggregates lifecycle folders.

[MScottAdams]

Companion: #436 (speckit Phase 4 collides with plan.vbrief.json definition). Fixing #436 by emitting scope vBRIEFs into pending/ would make this aggregator the natural renderer for speckit projects.

[MScottAdams]

Tracked by #437 (v0.20 vBRIEF-centric rendering is half-shipped).

[MScottAdams]

Risk-5 decision (2026-04-20): aggregator ordering

Per the #437 deep-think walkthrough. Two surfaces need ordering correctness in the aggregator: speckit's ordinal IP-N scopes (#436 Opt 1) and non-ordinal issue-backed scopes (today's YYYY-MM-DD-<issue-number>-<slug> convention from migrate_vbrief.py:726-733). Split fix along ordinal-vs-non-ordinal axis.

Adopted approach

1. Speckit filename convention: 3-digit zero-padded ordinal.

Filename: YYYY-MM-DD-ip<NNN>-<slug>.vbrief.json

Examples:

• 2026-04-20-ip001-foundation-setup.vbrief.json
• 2026-04-20-ip015-observability-hardening.vbrief.json
• 2026-04-20-ip042-rollout-automation.vbrief.json (future-proofed)

Padding is correct because the speckit IP index is ordinal (IP-1 really is phase 1, IP-2 is phase 2, ...). 999 phases covers every realistic project. Formalizes the Source-D filename in the #436 Opt 1 scope. Amend comment on #436 follows.

2. Issue-backed and task-id filenames stay unpadded.

Filename: YYYY-MM-DD-<issue-number>-<slug>.vbrief.json (as today)

Examples from vbrief/active/:

• 2026-04-16-404-run-project-lifecycle-subdirs.vbrief.json
• 2026-04-17-417-migrate-overview-narrative.vbrief.json

No padding because the issue number is a ticket identifier, not an ordinal. Issue #404 is not "the 404th scope." Padding would desynchronize from GitHub's #NNN reference notation (which is also unpadded) and from plan.references[].id.

3. Aggregator sorts on semantic keys, not filename keys.

Port _build_edge_map + _topo_sort_items from scripts/roadmap_render.py:80-144 into scripts/spec_render.py. Sort order:

1. Primary: dependency depth -- built from cross-scope metadata.dependencies. Same depth-based topo sort roadmap_render already does within a single vBRIEF, applied across scopes.
2. Secondary: creation date -- parsed from the YYYY-MM-DD- filename prefix.
3. Tertiary: filename lexicographic -- stable tiebreaker. For speckit's 3-digit-padded IPs, this yields correct numeric order. For unpadded issue numbers on the same date, this yields lexicographic order (which is semantically acceptable because issue numbers are non-ordinal -- order within a date is irrelevant).

This single sort pass handles both ordinal and non-ordinal scopes correctly without knowing which is which.

4. No schema field for ordering.

Explicitly rejected: metadata.sequence, metadata.index, or a top-level order key. Three reasons:

• Duplicates the filename's numeric content, creating a second source of truth that can drift.
• Speckit-specific; other strategies don't emit ordered sequences. Pollutes the schema for non-speckit users.
• Requires spec_validate.py and vbrief_validate.py changes and possibly monotonicity checks. Not worth the cost for a problem the filename already solves.

5. No filename migration for existing scopes.

Existing vbrief/active/2026-04-16-404-*.vbrief.json files stay untouched. Ordering fix lives in the aggregator, not the emit side. Zero rename cost.

Latent pre-existing quirk (documented but not fixed here)

Lexicographic sort on unpadded issue numbers is already semi-broken for projects spanning issues #4, #40, #400 on the same date. This repo happens to have clustered issue numbers (400s) so lexicographic sort matches numeric sort by coincidence. Not fixing here because:

• Non-ordinal: numeric order within a date is not semantically meaningful for tickets.
• The aggregator's dependency-depth primary sort dominates filename sort in any case where ordering actually matters.
• Projects that want explicit ordering use narratives.Phase + narratives.Tier grouping, already supported by roadmap_render.py:201-243.

New concern discovered: missing ingester

The #435 aggregator's effectiveness depends on scopes existing in lifecycle folders. The #437 walkthrough confirmed that post-GA, GitHub issue ingestion has no mechanized path (vbrief/vbrief.md:87-89 documents it as a ! MUST but ships no tool). Filed as a 6th v0.20.0 GA blocker.

Test coverage (lands in the #435 PR)

Two new fixtures in tests/cli/test_spec.py covering aggregator ordering:

• test_aggregator_orders_padded_numeric_scopes -- creates ip001 through ip012 filenames in non-creation-order; asserts rendered order is 1..12.
• test_aggregator_respects_dependency_depth -- later-indexed scope declares metadata.dependencies: ["<earlier-scope-slug>"]; asserts the dependency chain dominates filename sort.

Follows the existing inline-dict + tmp_path convention in tests/cli/test_spec.py and tests/cli/test_roadmap_render.py.

GA status

Risk-5 resolution is a component of #435's v0.20.0 GA scope. Lands in the same PR.

[MScottAdams]

Test-coverage decisions (2026-04-20)

Per the Risk-2 walkthrough on #437:

• Assertions, not snapshots -- use the existing tests/cli/ idiom (substring / position / count asserts).
• Inline fixture dicts + tmp_path lifecycle folders -- extend tests/cli/test_spec.py with a _write_scope helper and build scratch vbrief/{proposed,pending,active,completed}/ under tmp_path. No on-disk fixture files.
• Tests ship with this PR. Adds fixture D (lifecycle aggregation) plus the Risk-5 ordering tests test_aggregator_orders_padded_numeric_scopes and test_aggregator_respects_dependency_depth.

Full rationale: #437 (comment)

[MScottAdams]

Risk-6 resolver behavior (2026-04-20)

Per the #437 walkthrough and the #436 Risk-6 decision establishing plan.metadata.dependencies as a string array of filename stems.

Resolver contract

The aggregator's cross-scope topo-sort (Risk-5 in the earlier comment on this issue) reads each scope's plan.metadata.dependencies and resolves each stem to a file across all five lifecycle folders.

def resolve_dependency(stem: str, vbrief_root: Path) -> Path | None:
    """Find scope vBRIEF <stem>.vbrief.json across lifecycle folders."""
    for folder in ("proposed", "pending", "active", "completed", "cancelled"):
        candidate = vbrief_root / folder / f"{stem}.vbrief.json"
        if candidate.is_file():
            return candidate
    return None

Unresolved-dependency behavior (v0.20.0)

Warning, not error. If a stem doesn't resolve:

1. Log a warning line to stderr: "warning: scope <this-scope> declares dependency on <stem> which does not exist in any lifecycle folder".
2. Treat the missing dependency as a zero-depth node in the topo sort (matches _topo_sort_items:117-124's existing cycle-break behavior -- depth 0 when a dep can't be resolved).
3. Continue rendering. The aggregator produces output; the orphan edge is simply dropped.

Rationale: transient unresolved deps are legitimate during bulk creation (e.g., speckit Phase 4 writing 15 files), forward references in proposed/, in-flight migrator runs, and scope-deletion cleanup windows. Hard-erroring in v0.20.0 would turn recoverable drift into trust-destroying failures.

Post-GA tightening options are tracked as a single umbrella follow-up issue.

Consistency with roadmap_render.py

Matches the existing pattern:

• roadmap_render.py:117-124 already handles unresolvable in-scope dependency IDs by returning depth 0 without error.
• Malformed vBRIEF files are skipped silently in _load_vbriefs:46-48.
• The spec rendering does not aggregate scope vBRIEFs from lifecycle folders (v0.20 model) #435 aggregator carries forward the same "warn-then-continue" philosophy for cross-scope refs.

Lands in this PR

This resolver behavior is part of the #435 implementation alongside the topo-sort port and the ordering decisions already filed.