Repository navigation
validate: MODIFIED/REMOVED/RENAMED-from headers that don't exist in base spec aren't caught until archive (proposal: opt-in cross-change MODIFIED) #1112
Description
Activity
- added 5 commits that reference this issue
on Jun 29, 2026 Case 2 here (Change B MODIFYing a requirement Change A introduced but hasn't archived) is the same shape I ran into from the other side, working on cross-change overlap reporting for #1669 — so a note in case it's useful for scoping the fix.
The two implementations that disagree are
validator.ts:607(if (!current) continue;— a MODIFIED whose target is missing validates clean) andspecs-apply.ts:454(MODIFIED failed ... - not found). Anything else that needs to answer "does this delta apply?" — overlap reporting, a--dry-run, thediffcommand in #427 — becomes a third implementation of the same question unless the check lands as one function bothvalidateandbuildUpdatedSpeccall.I tried to answer it a second time and got it wrong twice, verified against real archive runs:
specs-apply.tshas several branches that treat a repeat as already-synced rather than as a collision, andRENAMED's version of that tests the counterpart header's presence, so a check reasoning about one header at a time gets renames wrong in both directions. I've written that up in more detail on #1113, which looks like the right home for the fix.Not proposing anything competing here — #1113 is open for this and @ixxie has offered to finish it.
@ryandemelo you may wanna have a look at #1683 and the associated PR; I came to the design trying to solve CI issues but it should make #1669 a lot simpler, at least in the long term.
Of course since my proposal is an opt-in mode with migrations, I guess it would be a bit complicated during the intermediate dual support time. But Im convinced explicit state makes everything easier.
Thanks @ixxie — read it properly, and replied over on #1683 since the argument is with II and IV rather than with anything on this issue. Short version for anyone following this thread: I think IV is the decision that bears on #1112, and it states the shared-implementation property for
sync --checkand the fold, which is a different pair from thevalidate/buildUpdatedSpecsplit this issue is about. Same principle, neither one closes the other.We hit both of your cases on a 73-change cross-repo project and ended up building an external deny-gate for them. Since #1918 was closed into this issue with the remaining question stated as "making validate fail on a lone typo without blocking valid stacked changes", here is how we discriminate the two — in case the shape is useful, not as a request.
The discriminator we use: resolve the absent target against the declared predecessor's ADDED requirements.
Our changes carry a
blocked_by:list (a project-local field — OpenSpec has no equivalent, which is the obvious limit of this report). The gate, run onarchive:- Parse the
## MODIFIEDheaders out of the change's deltas. - For each, look for an exact match in
openspec/specs/<cap>/spec.md. - On a miss, look for the same header under
## ADDED Requirementsin each declared predecessor's delta. - Found there → resolved, not a finding. That is your Case 2, correctly queued, and firing on it would report correctly-ordered work as broken.
- Not found anywhere → deny, naming the header and the spec file. That is your Case 1.
So the signal that separates them is not anything about the header — it is whether the author declared the dependency. A typo declares nothing.
Two things we got wrong first, both worth more than the mechanism:
🔴 Our first version took 27.2 seconds. A
PreToolUsehook that exceeds the harness timeout fails silently, so for a deny gate that means the archive proceeds unguarded — the exact false all-clear it existed to remove. If you do put this invalidate, the in-memory merge is already paid for by #1710, but the predecessor walk is not. Ours now measures 0.8–3.9s over 168 archived changes, and we plan against the worst case, never the best.🔴 Matching must be anchored. An archived
2026-08-11-<id>must not satisfy ablocked_byof a different id that happens to be a suffix of it.A second gate, because the first answers the wrong question for a reader. The deny gate judges only the change being archived. We also compute, across the whole in-flight set, which other changes declare a requirement this delta also declares under
MODIFIED/REMOVED/RENAMED— the three body-reproducing sections. That one is an ask, never a deny: two in-flight changes touching one requirement is legitimate and, with most of ours sharing a capability, routine. The resolution is ordering, not exclusion, and a gate that blocks the normal path is one people route around.Both reuse the same heading extractor and the same scope expression byte-for-byte, because two mechanisms parsing the same text differently is worse than one — both keep running and only one is right.
Why we think INFO alone was not enough for us, stated as experience rather than as a disagreement with the call made in #1918: a MODIFIED requirement reproduces its whole body, so when two changes modify one requirement and the second archives against the pre-rebase body, archive reports success while reverting everything the first added. We measured one such pair where the scenario set of the second was a strict superset of the current spec's — it added one scenario and dropped none — so the 1.8.0 scenario-drop guard correctly did not fire. The loss would have been confined to prose inside scenarios both deltas kept, which nothing compares. That case is invisible to any check keyed on scenario names.
Happy to answer questions; we are not asking for the
blocked_byfield itself, only offering the discrimination rule in case it saves someone the two wrong turns above.- Parse the
@jean-sturacci this is the most useful thing on this issue. Rather than agree with it I went and checked the part that bears on my own work against current
main, then measured it here, so the below is confirmation and one correction to my own first attempt at measuring it.The guard is name-keyed, exactly as you say.
Two call sites, the archive-time one at
specs-apply.ts:542and the validate-time one atvalidator.ts:765. Both calldiffScenarioNames, whose first two lines are:const currentNames = parseScenarioBlocks(current.raw).map((scenario) => scenario.name); const incomingNames = parseScenarioBlocks(incoming.raw).map((scenario) => scenario.name);
Names only, and nothing anywhere in
src/core/compares one scenario body against another. The apply path then doesnameToBlock.set(key, mod)atspecs-apply.ts:555, replacing the whole block. The only thing between the guard and that line is anormalizeBlockRawcomparison at552, and it exists to decide whether a write happened, not to warn that one did. So your case lands precisely where you said: scenario set a superset, guard silent, requirement statement and every kept scenario's body replaced wholesale.How often the shape occurs in this repository.
85 archived changes, 122 delta spec files, 93 MODIFIED blocks, 52 distinct (capability, requirement) pairs modified.
- 14 of those 52 requirements were modified by more than one archived change, with a long tail: one claimed by 10 changes, one by 9, one by 7.
- 41 consecutive claim pairs on one requirement, 29 of them within 14 days of each other.
- 15 archive commits where more than one change modified the same requirement with differing block text, so one won and the rest were discarded with nothing reported.
- Of those 15, 7 also disagreed on the scenario name set, which the guard can see. 12 disagreed on the body of a scenario that all of them kept, which the guard cannot.
One named case, since an aggregate is easy to wave away. Four changes archived together on 2025-10-14 all modified
cli-init :: Slash Command Configuration, and all four keep the scenario "Generating slash commands for OpenCode". Two of them disagree on the path inside it,.opencode/command/against.opencode/commands/. One of those is what got written and nothing said a choice had been made.What I did not find, which matters as much.
I also wrote the stronger test, because "the bodies differ" is not the same claim as "content was lost". For each archived MODIFIED block: reconstruct the main spec as it stood at the parent of that change's archive commit, walk the full git history of that requirement's statement text, and flag any delta whose statement matches a value that appeared earlier in that history than the base did. That is a revert to a stale base, as opposed to an author shortening something on purpose.
Zero hits across 84 blocks.
My first pass was the easy measurement, "did kept prose get shorter", and it reported 14 cases. Every one I opened was either a batch artifact, where I had compared three deltas from a single archive commit against the same parent, or a deliberate rewrite. That is the number I would have posted had I stopped there, and it would have been wrong.
I read the zero as supporting your framing rather than weakening it. Whether a body difference is intent or an accident is not decidable from the artifacts. The tool cannot tell, and it never asks. That is the same conclusion I reached from the other direction measuring scenario omissions: the omission itself carries no signal about whether the author meant it, so the marker has to be authored.
On
blocked_by: the field exists as a proposal.True of shipped OpenSpec, not of the queue. The open
add-change-stacking-awarenesschange proposesdependsOnas change metadata, withdependsOnas the source of truth for archive ordering, alongsideprovides,requires,touchesandparent. Your discriminator plugs into that field directly, and your report is the strongest argument for it so far because it is a measured account of what goes wrong without it. Worth saying so there if you have the appetite.Your anchored-matching warning applies to it immediately:
dependsOnwill hold change ids, archived ids carry a date prefix, and a suffix match would let2026-08-11-foosatisfy a dependency on a different id ending infoo.Three checks, not one.
Worth separating, because this issue now carries all three and they have different answers.
- MODIFIED target absent from the base. This issue. Decidable from one change plus the main specs.
- Two in-flight changes both claim one requirement. feat(validate): report requirements two active changes both claim #1698, design in docs(openspec): propose reporting requirements two active changes both claim #1963. Advisory, reported without ranking, which is the same ask-never-deny conclusion you reached independently. That was the part of the design I was least sure of, so I will take the corroboration.
- Target present, but the body the delta reproduces is stale. Yours. Neither 1 nor 2 catches it. 2 reports the precondition at validate time, which is the earliest anything can, but it warns and does not compare the two bodies, so an author who ignores the warning still gets last-writer-wins.
I am not proposing 3 here. It needs a body comparison that does not exist anywhere in the codebase, and after the 27.2 second story I would rather see the cost measured before anyone designs the check than after.
On the timeout, taking the general lesson.
The asymmetry is the part worth keeping: a report that is slow is just slow, a gate that is slow is a false all-clear. It changed how I wrote up a static scan I have been working on elsewhere in the tool. The cheap prefilter is now a fixed decision with the measured worst case beside it, rather than an implementation detail someone later has to rediscover the reason for.
Capability-path form of the same split. Main has
scheduling/dispatch. The change puts## MODIFIED RequirementsforAssign a technicianunderspecs/scheduling/dispatchh/spec.md(the directory name is mistyped).openspec validate add-region-filter --strictexits 0. The only finding is this INFO line:ℹ [INFO] scheduling/dispatchh/spec.md: Archive would refuse this delta: scheduling/dispatchh: target spec does not exist; only ADDED requirements are allowed for new specs. MODIFIED and RENAMED operations require an existing spec.openspec archive add-region-filter --yesexits 1:scheduling/dispatchh: target spec does not exist; only ADDED requirements are allowed for new specs. MODIFIED and RENAMED operations require an existing spec. Aborted. No files were changed.Expected:
validate --strictshould fail with an error when aMODIFIEDorREMOVEDdelta targets a capability or requirement that is not in main, consistent with archive. This is the lone typo left open when #1918 was closed into this issue: the capability directory does not exist, rather than a requirement header missing inside an existing spec.Environment: OpenSpec 1.14.1 (
npx -y @fission-ai/openspec@1.14.1 --no-color), Node v24.21.0, macOS (Darwin 25.6.0, arm64). Same INFO line and same exit codes on 1.12.0. I ran the script twice from scratch on each version. Both runs on a version matched.Impact: Two branches editing the same capability in parallel is common.
validate --strictis the check that runs on the branch, and it stays green for a mistyped capability path. Archive is the first command that refuses the delta.Repro
#!/usr/bin/env bash # Bug: validate --strict accepts a MODIFIED delta whose capability does not # exist. archive refuses the same delta. set -euo pipefail OPENSPEC=(npx -y @fission-ai/openspec@1.14.1 --no-color) ROOT="$(mktemp -d)" cd "$ROOT" git init -q echo "+ ${OPENSPEC[*]} init --tools none --no-animation ." "${OPENSPEC[@]}" init --tools none --no-animation . >/dev/null echo "(exit 0)" mkdir -p openspec/changes/add-dispatch/specs/scheduling/dispatch cat > openspec/changes/add-dispatch/.openspec.yaml <<'EOF' schema: spec-driven created: 2026-10-06 EOF cat > openspec/changes/add-dispatch/specs/scheduling/dispatch/spec.md <<'EOF' ## Purpose Assigns one technician to a job that is ready to be dispatched. ## ADDED Requirements ### Requirement: Assign a technician The system SHALL assign a technician to a ready job. #### Scenario: Single technician - **WHEN** a job is ready for dispatch - **THEN** the system assigns one technician EOF echo "=== setup: archive the base capability ===" echo "+ ${OPENSPEC[*]} validate add-dispatch --strict" "${OPENSPEC[@]}" validate add-dispatch --strict echo "(exit 0)" echo "+ ${OPENSPEC[*]} archive add-dispatch --yes" "${OPENSPEC[@]}" archive add-dispatch --yes echo "(exit 0)" echo "=== main specs ===" find openspec/specs -type f -print | sort echo "=== change add-region-filter targets scheduling/dispatchh (typo) ===" mkdir -p openspec/changes/add-region-filter/specs/scheduling/dispatchh cat > openspec/changes/add-region-filter/.openspec.yaml <<'EOF' schema: spec-driven created: 2026-10-06 EOF cat > openspec/changes/add-region-filter/specs/scheduling/dispatchh/spec.md <<'EOF' ## MODIFIED Requirements ### Requirement: Assign a technician The system SHALL assign a technician to a ready job only within the job's service region. #### Scenario: Single technician - **WHEN** a job is ready for dispatch - **THEN** the system assigns one technician EOF echo "=== validate --strict ===" echo "+ ${OPENSPEC[*]} validate add-region-filter --strict" set +e "${OPENSPEC[@]}" validate add-region-filter --strict validate_status=$? set -e echo "(exit ${validate_status})" echo "=== archive ===" echo "+ ${OPENSPEC[*]} archive add-region-filter --yes" set +e "${OPENSPEC[@]}" archive add-region-filter --yes archive_status=$? set -e echo "(exit ${archive_status})" echo "=== proof ===" if [[ "$validate_status" -ne 0 ]]; then echo "BUG NOT REPRODUCED: validate --strict exited ${validate_status}" exit 1 fi if [[ "$archive_status" -eq 0 ]]; then echo "BUG NOT REPRODUCED: archive exited 0" exit 1 fi if [[ -e openspec/specs/scheduling/dispatchh/spec.md ]]; then echo "BUG NOT REPRODUCED: archive created the typo capability" exit 1 fi echo "BUG REPRODUCED: validate --strict exited 0 and archive exited ${archive_status}." echo "Main spec left in place:" find openspec/specs -type f -print | sort
Actual output (1.14.1, trimmed)
archive add-dispatch --yesexits 0. Main specs areopenspec/specs/.gitkeepandopenspec/specs/scheduling/dispatch/spec.md. Then:=== validate --strict === + npx -y @fission-ai/openspec@1.14.1 --no-color validate add-region-filter --strict Change 'add-region-filter' is valid ℹ [INFO] scheduling/dispatchh/spec.md: Archive would refuse this delta: scheduling/dispatchh: target spec does not exist; only ADDED requirements are allowed for new specs. MODIFIED and RENAMED operations require an existing spec. (exit 0) === archive === + npx -y @fission-ai/openspec@1.14.1 --no-color archive add-region-filter --yes Task status: No tasks Specs to update: scheduling/dispatchh: create scheduling/dispatchh: target spec does not exist; only ADDED requirements are allowed for new specs. MODIFIED and RENAMED operations require an existing spec. Aborted. No files were changed. (exit 1) === proof === BUG REPRODUCED: validate --strict exited 0 and archive exited 1. Main spec left in place: openspec/specs/.gitkeep openspec/specs/scheduling/dispatch/spec.md
Symptom
If a change delta references a
### Requirement: …header in## MODIFIED Requirements,## REMOVED Requirements, or## RENAMED Requirements(FROM side) whose header does not appear in the base spec,openspec validate <change>returnsChange is valid— even with--strict. The bug only surfaces later, atopenspec archivetime, as:By then the implementing PR has typically shipped (days or weeks earlier). The session that authored the spec is gone. Recovering requires forensics + cross-change unblock work.
Real-world reproductions
I hit this twice within 6 days on the same project:
Case 1 — authoring misclassification. An agent wrote
## MODIFIED Requirementsfollowed by a brand-new requirement title that doesn't exist in the base spec. Should have been ADDED. The agent's mental model was "I'm modifying behavior of an existing feature" (true — the implementation extended an existing component); OpenSpec's semantics is "MODIFIED replaces a Requirement whose### Requirement:header exactly matches one in the base spec" (whitespace-insensitive). The mismatch sat undetected from 2026-05-15 to 2026-05-21.Case 2 — cross-change MODIFIED, sister change not yet archived. Change B wanted to MODIFY a requirement that Change A introduced. Change A's implementing PR had already merged and shipped to production, but Change A itself was never archived — so the requirement lived in
openspec/changes/<A>/specs/<cap>/spec.md, not in canonicalopenspec/specs/<cap>/spec.md. Change B's agent wrote## MODIFIED Requirementsagainst that. Archive aborted weeks later.Both cases share the same root cause: agents conflate "MODIFIED = extending existing behavior" with OpenSpec's actual semantics "MODIFIED = replace canonical Requirement by exact header match".
Minimal reproduction
Proposal
Move the existing archive-time check into
Validator.validateChangeDeltaSpecsso it fires at write time too — with a deliberate opt-in for the legitimate cross-change case.Default behavior (no flag)
openspec validate <change>errors when MODIFIED/REMOVED/RENAMED-from references a header that doesn't exist in canonicalopenspec/specs/<cap>/spec.md. Matches archive-time strictness.Opt-in:
--accept-cross-change-baseWhen set, the check also looks in sister-pending changes at
openspec/changes/<other>/specs/<cap>/spec.md. If the header appears there, validate passes. The user has consciously opted into cross-change MODIFIED authoring.openspec archiveremains strict regardless of this flag. Archive only accepts canonical-base references. The user opting into--accept-cross-change-baseat write time has committed to either (a) archiving the sister change first, or (b) folding this change into it, before this change can itself archive.Why this split
Encoding the existing-but-unenforced doc rule:
openspec instructions specsalready says "Locate the existing requirement inopenspec/specs/<capability>/spec.md". The rule's just not checked. Move the check earlier. The flag accommodates the one legitimate cross-change pattern (sister change is in flight; you're extending it) without weakening archive-time semantics where the canonical record matters.Implementation
PR follows shortly — keeps the change small (one new helper method on
Validator, one new CLI flag, no breaking API changes; legacynew Validator(true)constructor preserved via overload). 9 new tests added; full existing suite (1511 tests) still green.Happy to discuss flag name (
--accept-cross-change-baseis descriptive but verbose). Other contenders:--allow-pending-base,--allow-cross-change,--accept-sister-deltas. I'd take any direction here.