Skip to content

chore(claude): trim resolve-issue from six narrated steps to seven rules - #254

Open
joshukraine wants to merge 3 commits into
masterfrom
chore/gh-252-trim-resolve-issue
Open

chore(claude): trim resolve-issue from six narrated steps to seven rules#254
joshukraine wants to merge 3 commits into
masterfrom
chore/gh-252-trim-resolve-issue

Conversation

@joshukraine

@joshukraine joshukraine commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Phase 5 of #252. 649 → 386 words (40%). Does not close the issue.

resolve-issue was the clearest instance in the toolset of the pattern Anthropic singles out as the common failure mode with current models — "I want you to do this, but I want you to do it in this way, this way, this way. You must do like one, then two, then three, then four." Six numbered steps with per-step progress reporting, where most step bodies described what any competent agent does unprompted.

Cut

Removed Why
"Track your progress through the 6 steps below. Report completion of each step before moving to the next." Process theater. Spent output tokens narrating the procedure instead of doing it, and made every issue feel like a status meeting.
Step 1 — run gh issue view, extract title/labels/comments Reading the issue you were handed is not an instruction that needs giving. Folded into the opening line.
Step 2 — search PRs, search codebase, review similar issues Derivable. The one semi-novel bit, gh pr list --search "fix issue N", rarely hit anything: it assumes "fix issue" appears in a PR title, which Conventional Commits makes unlikely.
Step 3 — break into tasks, identify files, consider edge cases Derivable. The CHECKPOINT beneath these bullets was kept.
Step 5 — git log --oneline to confirm commits exist, "verify the issue is resolved" Confirming your own commits exist is busywork.
Trailing "Important" recap Restated verbatim what Steps 4 and 6 had already said. The rule and its rationale now appear exactly once, inline.

Kept

Every survivor is something an agent would otherwise get wrong or have no way to know:

Kept Why it isn't derivable
Small-vs-complex checkpoint policy A judgment call about when to stop; the wrong default is expensive in either direction
Branch reuse, incl. autopilot worktree re-entry Breaking it breaks /autopilot-batch's fan-out
Branch prefix table Arbitrary vocabulary, and the fallback for repos with no convention of their own
Never reference the issue number in a commit Deliberate convention with a real rationale
3–8 working commits per issue A cadence preference, not general practice
Acceptance-criteria checkbox recipe Non-obvious gh mechanic
Stop before the PR → /simplify/create-pr The workflow boundary the whole pipeline rests on

One addition

The prefix table now defers to the project's own branch convention where one exists. That's decision D3 (narrowest scope) applied to a skill that runs across many repos — ComixDistro's <type>/gh-<issue#>-<desc> should win over a generic table, not compete with it.

Note on the observation window

This lands mid-Phase-2, so the global trim and this skill trim are both in flight. Accepted deliberately: the failure modes are distinguishable (a misfiring skill is localized and obvious within one invocation), and testing this against a real issue beats reasoning about it in the abstract. Revert is git revert on one commit if it misbehaves.

Quarantined in docs/attic-2026-08.md with per-removal reasoning; the attic title now covers skills as well as the global file.


Second commit — the experiment already paid off, against itself

This skill ran live on comix_distro #512 before merge, and found a regression in its own trim. Final word count 560 (up from 386); still a 14% cut from the original 649, and the file now says less while covering more.

What broke

The trim cut Step 1's "extract title, description, labels, and comments" as GENERIC, leaving only gh issue view Nwhich does not print the comment thread.

On #512 the three comments carried roughly a third of the actual scope, plus one that superseded an acceptance criterion still written in the body. Following the trimmed skill literally would have shipped incomplete work while ticking every AC — a silent, confident miss. The live session caught it only because it reached for the --json form out of habit, which is not something to rely on across sessions or model versions.

Verified before fixing

Command Behavior
gh issue view N body only. Header does carry a comments: N count — contrary to the initial report — but in a metadata block that is easy to skim past.
gh issue view N --comments comments only; omits title and body, so not a drop-in replacement

The fetch step now runs both calls unconditionally, with the rationale inline: the body is the oldest artifact on an issue; comments carry scope added later, sequencing, and decisions that supersede it.

Also in this commit

The acceptance-criteria write-back moves from --body "$var" to a file round-trip with --body-file. Issue bodies routinely contain backticks, and a file keeps them out of shell parsing entirely — the same reasoning that survived the Q1 commit-protocol test.

What it taught the method — D7 on #252

A skill that hands over a command incomplete for the job it describes is a command bug, not a behavioral correction, and is out of ablation scope. The instruction introduces the gap rather than failing to prevent one, and no amount of model capability closes it — the model is doing exactly what it was told.

Test: if the model followed this line perfectly, would it still get the wrong result? If yes, fix it; don't trim it.

Honest accounting: the original was already self-contradictory — an incomplete command paired with an instruction to extract something that command does not return. The trim did not create the flaw, but it removed the one word that prompted going around it. Logged as the stumble log's first entry, and the attic's Step 1 entry is annotated as partially reverted rather than quietly amended.

Phase 5 of #252. 649 → 386 words (40%). The skill was the clearest instance of
the "one, then two, then three, then four" shape that current models do not
need: six numbered steps with per-step progress reporting, where most step
bodies described what any competent agent already does.

Cut the scaffolding and the derivable bodies — "track your progress through the
6 steps below", reading the issue you were handed, the codebase search, the
task breakdown, confirming your own commits exist, and a trailing "Important"
block that restated verbatim what two earlier steps had already said.

Kept every rule that is not derivable: the small-vs-complex checkpoint policy,
branch reuse including autopilot worktree re-entry, the branch prefix table,
never referencing the issue number in a commit, the 3-8 working-commit cadence,
the acceptance-criteria checkbox recipe, and the stop-before-PR handoff to
/simplify and /create-pr.

One addition rather than a trim: the prefix table now defers to the project's
own branch convention where one exists, which is decision D3 (narrowest scope)
applied to a skill that runs across many repos.

Cuts are quarantined in the attic with the reasoning per removal, and the attic
title now covers skills as well as the global file.
@joshukraine joshukraine added the chore Maintenance, refactoring, dependencies label Aug 1, 2026
First live run of the trimmed skill (comix_distro #512) surfaced a regression.
The trim cut Step 1's "extract title, description, labels, and comments" as
GENERIC, leaving only `gh issue view N` — which prints the body but not the
comment thread. On #512 the three comments held roughly a third of the real
scope plus one that superseded an acceptance criterion still written in the
body, so following the skill literally would have shipped incomplete work while
ticking every AC. That session read them anyway out of habit, which is not
something to rely on across sessions or model versions.

Verified against the live CLI before fixing:

- `gh issue view N` returns the body only. Its header does carry a `comments: N`
  count, but in a metadata block that is easy to skim past.
- `gh issue view N --comments` returns the comments only, omitting title and
  body — so it is not a drop-in replacement for the first call.

The fetch step now runs both unconditionally, with the rationale inline: the
body is the oldest artifact on an issue, and comments carry scope added later,
sequencing, and decisions that supersede it.

Also switches the acceptance-criteria write-back from `--body "$var"` to a file
round-trip with `--body-file`. Issue bodies routinely contain backticks, and a
file keeps them out of shell parsing entirely — the same reasoning that
survived the Q1 commit-protocol test.

Logged as the stumble log's first entry, and the attic entry for Step 1 is
annotated as partially reverted rather than quietly amended. The distinction it
sharpened is recorded as D7 on the tracking issue: a skill handing over an
incomplete command is a command bug, not a behavioral correction, and is out of
ablation scope — the instruction introduces the gap rather than failing to
prevent one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Maintenance, refactoring, dependencies

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant