Harden triage tier against prompt regressions and timeouts - #22
Conversation
Two recent failures in the PR Review Agent workflow each aborted the hourly batch session-fatally: - Run #461: triage model returned conversational text asking clarifying questions instead of JSON, breaking the contract. - Run #470: triage process exited 1 on a single PR (likely timeout or subprocess issue with no diagnostic output). Changes: 1. Bump TRIAGE_TIMEOUT_SEC default 180s → 300s. The 180s cap was tight for large PRs and offers no headroom for the model to complete; 300s matches ACTION_TIMEOUT_SEC and reduces the chance of a kill being misclassified as a deterministic failure. 2. Tighten the triage prompt: explicitly forbid clarifying questions, plan proposals, and prose. Add a fallback rule that says even on incomplete context the model must still emit JSON (with escalate=true) rather than respond conversationally. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR hardens the tier-1 triage path in the PR review workflow to reduce session-fatal failures from model regressions (non-JSON output) and overly tight timeouts.
Changes:
- Increase the default triage timeout from 180s to 300s.
- Tighten the triage prompt to forbid clarifying questions/plans/prose and to require JSON output even when context is incomplete.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| scripts/engine.sh | Raises default TRIAGE_TIMEOUT_SEC to reduce triage subprocess timeouts. |
| prompts/triage.md | Strengthens prompt constraints to reduce non-JSON / conversational triage outputs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| with `"escalate": true` and a signal explaining what was missing — never | ||
| respond with prose. |
There was a problem hiding this comment.
The new fallback instruction says to emit JSON with "escalate": true and "a signal" explaining what's missing. Since the required schema below uses a signals array (and scripts parse .signals | join(...)), this wording risks the model outputting a signal field or omitting signals, which would break the contract. Suggest rephrasing to explicitly say to populate the signals array (and still include required risk and summary fields).
| with `"escalate": true` and a signal explaining what was missing — never | |
| respond with prose. | |
| with `"escalate": true`, populate the `signals` array with what was missing, | |
| and still include the required `risk` and `summary` fields — never respond | |
| with prose. |
Two recent failures in the PR Review Agent workflow each aborted the hourly batch session-fatally: - Run #461: triage model returned conversational text asking clarifying questions instead of JSON, breaking the contract. - Run #470: triage process exited 1 on a single PR (likely timeout or subprocess issue with no diagnostic output). Changes: 1. Bump TRIAGE_TIMEOUT_SEC default 180s → 300s. The 180s cap was tight for large PRs and offers no headroom for the model to complete; 300s matches ACTION_TIMEOUT_SEC and reduces the chance of a kill being misclassified as a deterministic failure. 2. Tighten the triage prompt: explicitly forbid clarifying questions, plan proposals, and prose. Add a fallback rule that says even on incomplete context the model must still emit JSON (with escalate=true) rather than respond conversationally. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Two recent failures in the PR Review Agent workflow each aborted the hourly batch session-fatally: - Run #461: triage model returned conversational text asking clarifying questions instead of JSON, breaking the contract. - Run #470: triage process exited 1 on a single PR (likely timeout or subprocess issue with no diagnostic output). Changes: 1. Bump TRIAGE_TIMEOUT_SEC default 180s → 300s. The 180s cap was tight for large PRs and offers no headroom for the model to complete; 300s matches ACTION_TIMEOUT_SEC and reduces the chance of a kill being misclassified as a deterministic failure. 2. Tighten the triage prompt: explicitly forbid clarifying questions, plan proposals, and prose. Add a fallback rule that says even on incomplete context the model must still emit JSON (with escalate=true) rather than respond conversationally. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Two recent failures in the PR Review Agent workflow each aborted the hourly batch session-fatally: - Run #461: triage model returned conversational text asking clarifying questions instead of JSON, breaking the contract. - Run #470: triage process exited 1 on a single PR (likely timeout or subprocess issue with no diagnostic output). Changes: 1. Bump TRIAGE_TIMEOUT_SEC default 180s → 300s. The 180s cap was tight for large PRs and offers no headroom for the model to complete; 300s matches ACTION_TIMEOUT_SEC and reduces the chance of a kill being misclassified as a deterministic failure. 2. Tighten the triage prompt: explicitly forbid clarifying questions, plan proposals, and prose. Add a fallback rule that says even on incomplete context the model must still emit JSON (with escalate=true) rather than respond conversationally. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Two recent failures in the PR Review Agent workflow each aborted the hourly batch session-fatally: - Run #461: triage model returned conversational text asking clarifying questions instead of JSON, breaking the contract. - Run #470: triage process exited 1 on a single PR (likely timeout or subprocess issue with no diagnostic output). Changes: 1. Bump TRIAGE_TIMEOUT_SEC default 180s → 300s. The 180s cap was tight for large PRs and offers no headroom for the model to complete; 300s matches ACTION_TIMEOUT_SEC and reduces the chance of a kill being misclassified as a deterministic failure. 2. Tighten the triage prompt: explicitly forbid clarifying questions, plan proposals, and prose. Add a fallback rule that says even on incomplete context the model must still emit JSON (with escalate=true) rather than respond conversationally. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Two recent failures in the PR Review Agent workflow each aborted the hourly batch session-fatally: - Run #461: triage model returned conversational text asking clarifying questions instead of JSON, breaking the contract. - Run #470: triage process exited 1 on a single PR (likely timeout or subprocess issue with no diagnostic output). Changes: 1. Bump TRIAGE_TIMEOUT_SEC default 180s → 300s. The 180s cap was tight for large PRs and offers no headroom for the model to complete; 300s matches ACTION_TIMEOUT_SEC and reduces the chance of a kill being misclassified as a deterministic failure. 2. Tighten the triage prompt: explicitly forbid clarifying questions, plan proposals, and prose. Add a fallback rule that says even on incomplete context the model must still emit JSON (with escalate=true) rather than respond conversationally. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Two recent failures in the PR Review Agent workflow each aborted the hourly batch session-fatally: - Run #461: triage model returned conversational text asking clarifying questions instead of JSON, breaking the contract. - Run #470: triage process exited 1 on a single PR (likely timeout or subprocess issue with no diagnostic output). Changes: 1. Bump TRIAGE_TIMEOUT_SEC default 180s → 300s. The 180s cap was tight for large PRs and offers no headroom for the model to complete; 300s matches ACTION_TIMEOUT_SEC and reduces the chance of a kill being misclassified as a deterministic failure. 2. Tighten the triage prompt: explicitly forbid clarifying questions, plan proposals, and prose. Add a fallback rule that says even on incomplete context the model must still emit JSON (with escalate=true) rather than respond conversationally. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Two recent failures in the PR Review Agent workflow each aborted the hourly batch session-fatally: - Run #461: triage model returned conversational text asking clarifying questions instead of JSON, breaking the contract. - Run #470: triage process exited 1 on a single PR (likely timeout or subprocess issue with no diagnostic output). Changes: 1. Bump TRIAGE_TIMEOUT_SEC default 180s → 300s. The 180s cap was tight for large PRs and offers no headroom for the model to complete; 300s matches ACTION_TIMEOUT_SEC and reduces the chance of a kill being misclassified as a deterministic failure. 2. Tighten the triage prompt: explicitly forbid clarifying questions, plan proposals, and prose. Add a fallback rule that says even on incomplete context the model must still emit JSON (with escalate=true) rather than respond conversationally. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Two recent failures in the PR Review Agent workflow each aborted the hourly batch session-fatally: - Run #461: triage model returned conversational text asking clarifying questions instead of JSON, breaking the contract. - Run #470: triage process exited 1 on a single PR (likely timeout or subprocess issue with no diagnostic output). Changes: 1. Bump TRIAGE_TIMEOUT_SEC default 180s → 300s. The 180s cap was tight for large PRs and offers no headroom for the model to complete; 300s matches ACTION_TIMEOUT_SEC and reduces the chance of a kill being misclassified as a deterministic failure. 2. Tighten the triage prompt: explicitly forbid clarifying questions, plan proposals, and prose. Add a fallback rule that says even on incomplete context the model must still emit JSON (with escalate=true) rather than respond conversationally. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Summary
Two recent failures in the PR Review Agent workflow each aborted the hourly batch session-fatally — see the investigation report for full analysis.
This PR addresses both root causes with minimal, low-risk changes.
Changes
TRIAGE_TIMEOUT_SECdefault 180s → 300s (scripts/engine.sh). The 180s cap was tight for large PRs and offered no headroom; 300s matchesACTION_TIMEOUT_SECand reduces the chance of a kill being misclassified as a deterministic failure (kills at 124 are retried; exit 1 is not).escalate=trueand a signal explaining what was missing) rather than respond conversationally.Test plan
TRIAGE_TIMEOUT_SEC=300shows in the workflow env block🤖 Generated with Claude Code