Repository navigation
feat(review): multi-agent review consensus engine (verify + dedup + tag) - #143
Merged
Merged
Conversation
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
📝 WalkthroughWalkthroughAdds a multi-agent review consensus engine with SQLite-backed findings, changed-line verification, clustering and verdicts, Markdown/JSON output, CLI commands, and MCP tools for review submission and round management. ChangesReview consensus workflow
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Finder
participant CLI_or_MCP
participant ReviewStorage
participant Git
participant Consensus
Finder->>CLI_or_MCP: submit findings
CLI_or_MCP->>ReviewStorage: store findings by repo and round
CLI_or_MCP->>Git: compute base...head diff
Git-->>Consensus: changed lines
ReviewStorage-->>Consensus: stored findings
Consensus-->>CLI_or_MCP: ranked items and rendered Markdown
Possibly related PRs
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
This was referenced Jul 11, 2026
getappz
pushed a commit
that referenced
this pull request
Jul 11, 2026
Review follow-up on #143. changed_lines() treated any non +/-/@@ line while a file was open as a context line — so in a multi-file diff the 'diff --git' / 'index' headers between file sections inserted a phantom changed-line into the PRECEDING file, falsely verifying a finding that cited it (and, in step 3, inflating that agent's accuracy). Reset the file at each 'diff --git' boundary and ignore '\ No newline at end of file' markers. Regression tests added.
added 2 commits
July 11, 2026 14:24
Finders submit findings for a round; agentflare mechanically verifies each cited
file:line against the git diff (base...head), clusters nearby findings from
different agents, and tags each CONFIRMED (>=2 agents) / UNIQUE (1) / DISPUTED
(conflicting severity) / UNVERIFIED (line not in the diff). Stored in a
review_findings table in agentflare.db; re-submit replaces an agent's prior set.
Surface: MCP tools review_submit/review_consensus/review_list/review_clear on
the flare server, plus `agentflare review {submit|consensus|list|clear}`.
Deterministic core only — finders stay external; posting to the PR (via the
step-0 github-mcp-server) is a later step.
Closes #142
Review follow-up on #143. changed_lines() treated any non +/-/@@ line while a file was open as a context line — so in a multi-file diff the 'diff --git' / 'index' headers between file sections inserted a phantom changed-line into the PRECEDING file, falsely verifying a finding that cited it (and, in step 3, inflating that agent's accuracy). Reset the file at each 'diff --git' boundary and ignore '\ No newline at end of file' markers. Regression tests added.
getappz
force-pushed
the
feat/review-consensus
branch
from
July 11, 2026 09:15
18a5eab to
b690436
Compare
3 tasks done
4 tasks done
getappz
added a commit
that referenced
this pull request
Aug 18, 2026
#549) Headless CLI agents run in print mode and emit no incremental stdout — only one blob once the whole turn finishes — so idle detection can't tell a hung process from one legitimately still computing (e.g. a cargo build/test mid-turn). DEFAULT_IDLE_TIMEOUT_SECS defaulted to 300, so any real SDD turn taking longer than 5 minutes died every time, confirmed live via items #143/#150/#151 failing identically 8 times in a row. It now defaults to the same 21600s (6h) ceiling as --timeout, the only backstop that still means what its doc comment says. Agentflare-Agent: claude-code Agentflare-Branch: task/143-idle-timeout-inherit-hard-cap Agentflare-Item: 143 Co-authored-by: shiva <shiva@gosysinfo.tech>
4 tasks done
getappz
added a commit
that referenced
this pull request
Sep 12, 2026
…es (#671) DEFAULT_IDLE_TIMEOUT_SECS was set equal to DEFAULT_TIMEOUT_SECS (21600s = 6h), deliberately walked back to that value after items #143/#150/#151 (a shorter idle-timeout killed real long turns as false positives, since headless CLI agents in plain print mode emit no incremental stdout). That made the idle-timeout mechanism a no-op by default: it never fires before the hard cap does. Item #222 found this live -- a claude-code job producing zero output beyond its 3-line claim banner squatted a daemon worker slot for 79+ minutes (likely a hung interactive auth prompt), blocking 1400+ queued jobs. The underlying liveness-watchdog mechanism (agent_launch::run_captured, byte-level stdout/stderr activity tracking) already exists and is already wired through every agent turn via idle_timeout -- the only gap was the default value disabling it. 30 minutes is short enough to free a genuinely wedged worker slot while staying well above the false-positive threshold that caused the prior incident (a 300s/5min default). The real work-item dispatch path also already forces --output-format stream-json for Claude Code/cursor-agent specifically to get incremental output (avoiding the all-buffered-json trap that caused #143/#150/#151), so this isn't reintroducing that exact failure mode, just picking a meaningfully-enforced threshold instead of an effectively-disabled one. The kill-reason distinction ("went idle for Ns (no new output)" vs "exceeded hard cap") and the failure flowing through the normal dispatch_failure_ceiling/orphan_reconcile cap logic (WORK_FAILURE_MARKER, same as any other failure) both already existed -- no changes needed there. Agentflare-Agent: claude-code Agentflare-Branch: task/222-bug-dispatcher-hung-workers-with-no-outp Agentflare-Item: 222 Agentflare-Session: 80f0d808-b8e6-4b0b-b889-91811d8fe19a Co-authored-by: shiva <shiva@gosysinfo.tech>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Step 2 of the GitHub-across-agents roadmap: the deterministic consensus core
for multi-agent PR review. Finders (agents running
/code-review, ponytail,subagents) submit findings; agentflare verifies, dedups, and tags them into one
review.
Scope decision
agentflare agents launchis an interactive single-agent launcher, not aheadless "run N agents and collect JSON" primitive — so this PR owns only the
deterministic, reusable consensus layer. Finders stay external and just submit.
(Headless fan-out and PR-posting are separate later steps.)
How
review_findingstable in~/.agentflare/agentflare.db(reuses
src/db.rs). Re-submit by an agent replaces its prior set for theround, so a re-run never double-counts.
changed_lines()parses the unified diff (git diff base...head,default
master...HEAD) into the new-side line set per file; a finding whosefile:lineisn't a changed line is UNVERIFIED.consensus()clusters verified findings by file + nearbyline (±3), then tags each cluster CONFIRMED (≥2 distinct agents) /
UNIQUE (1) / DISPUTED (≥2 agents, conflicting
severity) /UNVERIFIED. Ranked most-confident first; markdown + JSON renderings.
Surface
review_submit,review_consensus,review_list,review_clear.agentflare review {submit|consensus|list|clear} [--pr] [--repo] [--base --head].Tests
10 unit tests (diff parsing incl. new-file hunks, unverified-off-diff, 2-agent
clustering → CONFIRMED, single → UNIQUE, same-agent-twice ≠ CONFIRMED,
conflicting-severity → DISPUTED, distant lines don't cluster, ranking,
submit-replaces, round-scoped load/clear). Full suite: 278 passed, 0 failed.
Driven end-to-end via the CLI on this branch's real diff: two finders citing
nearby committed lines → one CONFIRMED cluster (
db.rs:5-7), singles → UNIQUE,off-diff citation → UNVERIFIED.
Closes #142 · related #137 (posting later), #141 (db)
Summary by CodeRabbit