Name the text, do not count the lines (#195 Tier 1: parser and applier) - #196
Merged
Conversation
…applier)
Measured, rdpapp 2026-08-05: four model calls, all HTTP 200, a healthy
gateway, and 0 of 2 items delivered.
T1: hunk ends 0 source and 7 result line(s) short of what its header declares
T2: the last hunk supplies 1 fewer source line
Neither model misunderstood its item. Both miscounted lines. A unified diff
asks for arithmetic the model cannot check — `@@ -401,7 +401,12 @@` claims what
the hunk consumes and produces, computed blind from a file read once in a
prompt — and one miscount loses the whole item. It scales with file size, so a
704 KB source file is the worst case.
An edit block has no line numbers, so that failure class cannot be
represented. What remains is a different and more dangerous risk: applying the
*wrong* text confidently. Most of this is therefore about refusing.
- **Whole-line anchoring, not substring.** `foo` occurs inside `foobar`;
substring matching would replace part of a longer identifier and produce
code that still compiles, means something else, and shows a diff that looks
deliberate. A match must begin and end on a line boundary. The cost is that
a model dropping indentation is refused, which is the intended trade: the
alternative is inferring which similar line was meant, and inference is what
this format exists to remove.
- **Ambiguity is refused.** Text occurring twice is not a location. Taking the
first match is how an edit lands in the wrong function. The count is
reported so the next attempt can widen its context instead of guessing.
- **Not found is refused**, and is distinct from create. Treating a mistaken
path as "create it" would leave a file nobody asked for in a tree the
reviewer then reads as intentional.
- **All or nothing.** A half-applied set runs the checks against a tree no
model intended and shows a reviewer a diff nobody wrote. Every edit is
validated against pending content first, so two edits to one file are
checked in the sequence they will be applied.
- **Confined to the worktree.** Paths are resolved and checked to be under the
root handed in, symlinks included, so a model naming `../../.ssh/id_rsa` is
refused. That is the file-write half of #187, arriving here for a different
reason; the shell half is unaffected and #187 stays open.
Requires nothing of the endpoint: no tool calling, no function calling, no
provider feature. It works against any model and any gateway, which is the
floor an open-source consumer gets regardless of what they point it at.
Parser and applier only. Wiring into the executor's implementer step is
deliberately not in this commit — three sibling branches are editing
`executor.py` for #152, #155 and #187, and this would conflict with all three.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
thedancingdeveloper
added a commit
that referenced
this pull request
Aug 5, 2026
…Tier 1) (#208) * feat: ask the implementer for edit blocks, and compute the diff here (#195 Tier 1) The parser and applier landed in #196. This is what makes them the executor's actual path. Measured on rdpapp, 2026-08-05: four model calls, all HTTP 200, a healthy gateway, 0 of 2 items delivered. T1: hunk ends 0 source and 7 result line(s) short of what its header declares T2: the last hunk supplies 1 fewer source line Neither model misunderstood its item. Both miscounted lines in a header they had to compute blind. `IMPLEMENT_PROMPT` now asks for edit blocks -- name the text, do not count the lines -- and there is nothing left to miscount. **The diff is rendered here, from content the harness has read**, before anything else runs. Every gate downstream is untouched: the patch validator, the apply ladder, the checks, the reviewer and the commit all still see a unified diff, and none of them learns a second way for changes to arrive. The model stops doing arithmetic; nothing else changes. Three decisions worth naming: - **A unified diff is still accepted.** A reply with no edit blocks falls through to `extract_diff` exactly as before, so a model that ignores the instruction, an `--implementer` with its own habits, and every durable attempt recorded before this change all still work. Refusing them would turn a format preference into an outage. - **`to_diff` writes nothing.** If it touched the tree the apply ladder would then apply an already-applied patch, and the item would fail for a reason the model had nothing to do with. Asserted by a test. - **A rejected edit says something usable.** "the SEARCH text occurs 2 times and is therefore not a location" can be put back in front of the model and fixed on the next attempt, where a bad hunk header cannot. Well-formed edits that change nothing are reported as no change rather than as a broken patch, because that is what they are. New events: `edits_parsed`, `edits_rejected`, and `no_diff` carrying "edit blocks that change nothing". What this does NOT fix, and should not be read as fixing: the implementer still gets one shot against a context the planner chose in advance. If it needs a file it was not given it is still stuck -- it now fails honestly rather than silently miscounting. Reading on demand and correcting its own failed edit are Tier 2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: reopen D10 on evidence Stage E1 could not have had E1 chose the model-authored unified diff on ten cases where it and search/replace scored identically, and only whole-file put a change in the wrong place. That was correct on the evidence it had. rdpapp, 2026-08-05: four model calls, all HTTP 200, healthy gateway, 0 of 2 items delivered, both to hunk headers disagreeing with their bodies. Neither model misunderstood its item. E1's cases are small and synthetic; the arithmetic a hunk header demands gets harder as a file grows, and rdpapp's is 704 KB. Reopened by the owner on that evidence. The experiment's safety boundary moves with the decision rather than being deleted: whole-file is still rejected and still implemented only in the test module, because its wrong-location result is exactly what that boundary keeps out of core. A unified diff is still read, so no durable attempt recorded earlier is stranded. Rule 7 in use: correct the document when reality disagrees, and name the disagreement rather than quietly changing the answer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: sprooty <sprooty@sprooty.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #195. Does not close it — this is Tier 1's parser and applier only; the executor wiring and Tier 2 remain.
Why
rdpapp, 2026-08-05: four model calls, all HTTP 200, healthy gateway, 0 of 2 items delivered.
Neither model misunderstood its item. Both miscounted lines. A unified diff demands arithmetic the model cannot check, computed blind from a file read once in a prompt — and one miscount loses the item.
An edit block has no line numbers, so that failure class cannot be represented.
The interesting decision: whole-line anchoring
My first implementation used plain substring matching. A test I wrote to prove indentation must match failed, and it exposed a worse problem than the one it was checking:
foooccurs insidefoobar, so an edit naming a short identifier would replace part of a longer one and produce code that still compiles, means something different, and shows a diff that looks deliberate.Matching is now anchored to whole lines. The cost is real and accepted: a model that drops the leading indentation gets refused rather than helped. That is the right trade — the alternative is inferring which of several similar lines was meant, and inference is precisely what this format exists to remove.
Refusal rules
Overlap with #187
Paths are confined to the item's worktree, so
../../.ssh/id_rsais refused by construction. That is the file-write half of #187, arriving here for a different reason. #187 stays open: it is about the agent's shell, and a check command still runs arbitrary commands.What this does not do
The model still gets one shot against a planner-chosen context. If it needs a file it was not given, it is still stuck — it now fails honestly ("that text is not there") instead of silently miscounting. On-demand reading and self-correction are Tier 2.
Requires nothing of the endpoint: no tool calling, no provider feature. It is the floor an open-source consumer gets regardless of the gateway they point at.
Not wired in yet, deliberately
Three sibling branches are editing
executor.pyfor #152, #155 and #187. Wiring the implementer step would conflict with all three, so it lands after they merge.21 tests.
ruff,mypyand the full suite pass.🤖 Generated with Claude Code