Review agent should validate implementation approach against issue requirements

[fullsend-ai-retro]

What happened

The review bot approved PR #1412 with no findings, praising the dedup logic, fail-open design, and TOCTOU mitigation. However, it did not check whether the implementation matched the preferred approach stated in the linked issue #1408. The human subsequently rejected the PR because it used the wrong approach entirely.

What could go better

The review agent should cross-reference the implementation against the linked issue's requirements and stated preferences, not just evaluate code quality in isolation. An implementation can be well-crafted but still wrong if it solves the problem using a rejected or non-preferred approach. Confidence: high — the review approved code that the issue author explicitly did not prefer, and a human had to intervene to redirect.

Proposed change

Update the review agent definition or the pr-review / code-review skill in fullsend-ai/fullsend to include a review dimension that checks: does the implementation approach match what the linked issue requested? Specifically, when the issue lists multiple approaches with preference labels, the review should flag if the PR implements a non-preferred approach without justification.

Validation criteria

In the next 5 PRs where the linked issue specifies a preferred approach, the review agent should flag any mismatch between the implemented approach and the preferred one. Zero false negatives on approach mismatch.

---

Generated by retro agent from #1412

[fullsend-ai-triage]

Triage Summary

Severity: Medium · Category: Feature enhancement

Problem

The code-review skill's intent-alignment dimension checks that a PR traces to a linked issue and that scope is appropriate, but does not specifically verify that the implementation approach matches what the issue requested. When an issue lists multiple approaches with stated preferences, the reviewer may approve a well-crafted implementation of a non-preferred approach.

This was observed on PR #1412, where the review agent approved code implementing an approach the issue author had explicitly not preferred.

Root Cause

The intent-alignment bullet in internal/scaffold/fullsend-repo/skills/code-review/SKILL.md (line 78) reads: "Does the implementation match what the linked issue describes?" — this is too coarse to catch approach-level mismatches. The reviewer evaluates code quality without parsing the issue for approach preferences or rejected alternatives.

Recommended Fix

Add a sub-bullet under intent alignment instructing the reviewer to:

1. Read the linked issue for stated approach preferences, rejected alternatives, or constraints on how the problem should be solved
2. Compare the PR's implementation approach against those preferences
3. Produce a finding if the PR uses a non-preferred or rejected approach without justification in the PR body

Proposed Test Case

Scenario: Approach mismatch detection
  Given an issue listing Approach A (preferred) and Approach B (not preferred)
  When a PR implements Approach B without justification
  Then the review produces an intent-alignment finding flagging the mismatch

Edge cases:
  - Issue with no stated preference → no finding expected
  - PR body justifies choosing non-preferred approach → informational finding
  - Issue mentions a single approach → no mismatch finding expected

---

Labels: Issue targets the review agent's code-review skill with a specific enhancement to the intent-alignment dimension, medium priority based on the single observed incident.

[fullsend-ai-prioritize]

RICE Priority Score: 2.4

Score breakdown

Dimension	Score	Reasoning
Reach	1.5	The review agent runs on all PRs in the repo, so this affects all contributors. However, the specific scenario—issues with explicitly stated approach preferences—is a subset of all reviews, bringing this to 1.5 rather than higher.
Impact	1	Medium impact. When this fails, a human reviewer must catch the mismatch and reject the PR, wasting a review cycle. A workaround exists (human review), but it undermines trust in the automated reviewer. One confirmed incident on PR #1412.
Confidence	0.8	High confidence. The issue is well-described with a specific incident, a clear root cause (intent-alignment bullet in SKILL.md is too coarse), a concrete fix location, and detailed test cases from the triage agent.
Effort	0.5	Simple change—adding sub-bullets to an existing review dimension in the code-review skill definition. The triage comment identifies the exact file and line. No architectural changes needed, just prompt refinement.

Formula: (1.5 x 1 x 0.8) / 0.5 = 2.4