Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 87 additions & 13 deletions packages/cli/src/commands/review/compose-review.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -478,6 +478,17 @@ function blindPlan(): string {
return plan();
}

function findingsFile(content: string): string {
const f = join(dir, 'qwen-review-findings.md');
writeFileSync(f, content);
return f;
}

const TAGGED =
'- **File:** src/pay.ts:42\n' +
'- **Issue:** off-by-one in the retry cap\n' +
'- **Severity:** Critical — [unverified]\n';

const FOOTER = `_— ${MODEL} via Qwen Code /review (vunknown)_`;

function base(overrides: Partial<ComposeReviewInput>): ComposeReviewInput {
Expand Down Expand Up @@ -1553,6 +1564,28 @@ describe('composeReview — presubmit downgrades', () => {
);
});

it('keeps the presubmit downgrade reasons when a cap softens the Approve first', () => {
// The APPROVE→COMMENT cap runs before the presubmit arms, so a capped
// zero-finding Approve reached arm 1 as COMMENT — its `event === 'APPROVE'`
// gate failed, arm 2's REQUEST_CHANGES gate failed too, and the presubmit
// reasons vanished from the body and the verdict line while the identical
// uncapped run rendered both. The gate is derived from `baseEvent` — the
// row before every cap — exactly like its RC sibling.
const r = composeReview(
base({
contextUnavailable: true,
presubmit: { downgradeApprove: true, downgradeReasons: ['self-PR'] },
}),
);
expect(r.baseEvent).toBe('APPROVE');
expect(r.event).toBe('COMMENT');
expect(r.cappedBy).toContain('context-unavailable');
expect(r.downgraded).toBe(true);
expect(r.downgradedFrom).toBe('Approve');
expect(r.body).toContain('⚠️ Downgraded from Approve to Comment: self-PR.');
expect(verdictLine(r)).toContain('a presubmit check failed');
});

it('a downgraded Approve never certifies "no blockers" in the same body (the downgrade names failing CI two clauses earlier)', () => {
const r = composeReview(
base({
Expand Down Expand Up @@ -3548,6 +3581,32 @@ describe('the Step 4/5 gate — verify and reverse audit must have run (high eff
expect(verdictLine(r)).toContain('its blockers were never verified');
});

it('keeps the presubmit downgrade reasons when the findings-tag cap also holds', () => {
// The tag cap softens the event while setting NEITHER legacy flag, so the
// recovery arm's enumeration missed it: the presubmit reasons vanished
// from the body — the silent loss the arm's own comment forbids. Verdict
// keeps the tag sentence; the body's downgrade clause carries the reasons.
const r = composeReview({
criticalsInline: 1,
planPath: coveredPlan(['verify', 'reverse-audit']),
env: ENV,
findingsPath: findingsFile(TAGGED),
presubmit: {
downgradeRequestChanges: true,
downgradeReasons: ['self-PR'],
},
modelId: MODEL,
});
expect(r.event).toBe('COMMENT');
expect(r.cappedBy).toContain('findings-unverified-at-compose');
expect(r.body).toContain(
'Downgraded from Request changes to Comment: self-PR',
);
expect(verdictLine(r)).toContain(
'findings were still unverified when the loop ended',
);
});

it('verify on record with the reverse audit absent still blocks — softening gates on verify alone', () => {
const r = composeReview({
criticalsInline: 1,
Expand All @@ -3574,6 +3633,31 @@ describe('the Step 4/5 gate — verify and reverse audit must have run (high eff
expect(r.body).toContain('**[Critical]** whole-PR blocker X');
});

it('keeps the body Criticals when the FINDINGS-TAG cap softens the event', () => {
// The third softening path, and the one the enumerated condition missed:
// coverage is proven and the verifier ran, so `criticalsUnverified` is
// false and no presubmit downgrade fired — the cap comes from the
// findings file still carrying `— [unverified]` at compose time. The
// posted body was 239 characters of opener and disclosure with the
// blocker — its only copy — nowhere in it.
const r = composeReview({
planPath: coveredPlan(['verify', 'reverse-audit']),
env: ENV,
modelId: MODEL,
criticalsInline: 0,
suggestionsInline: 0,
bodyCriticals: ['whole-PR blocker X'],
findingsPath: findingsFile(TAGGED),
});
expect(r.baseEvent).toBe('REQUEST_CHANGES');
expect(r.event).toBe('COMMENT');
// Neither flag the old condition listed is set on this path.
expect(r.downgradedFrom).toBeNull();
expect(r.cappedBy).toContain('findings-unverified-at-compose');

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The test pins only one of the two flags its comment claims absent: it never asserts criticalsUnverified is false. All five assertions are satisfiable with criticalsUnverified === true (both cap names are pushed independently, compose-review.ts ~2040-2043), and the sibling test 'softens a Request changes whose blockers are non-deterministic' (~line 7058) pins exactly this premise under the same harness. — Failure scenario: a future drift flipping criticalsUnverified true in this fixture (digest keying in verificationGaps/recordStep45, the coveredPlan transcript shape) leaves the test green — and would also leave it green against the reverted old condition (that flag satisfies || criticalsUnverified) → the 239-character-body bug this PR fixes regresses with no red test.

Witness: probe — reverted condition + criticalsUnverified flipped true passes all five original assertions; adding the suggested assertion turns it red (expected [ 'unreviewed-dimension', …(2) ] to not include 'criticals-unverified'); with the PR's code it passes.

Suggested change
expect(r.cappedBy).toContain('findings-unverified-at-compose');
expect(r.cappedBy).toContain('findings-unverified-at-compose');
expect(r.cappedBy).not.toContain('criticals-unverified');
中文说明

该测试只钉住了其注释声称缺席的两个标志之一:它从未断言 criticalsUnverified 为 false。当 criticalsUnverified === true 时全部五条断言仍可满足(两个 cap 名称是独立推入的,见 compose-review.ts 约 2040-2043 行),而姊妹测试「softens a Request changes whose blockers are non-deterministic」(约 7058 行)在同一夹具下恰好钉住了这一前提。失败场景:未来某次漂移使该夹具中 criticalsUnverified 变为 true(verificationGaps/recordStep45 的摘要键控、coveredPlan 的 transcript 形态)时,测试仍然为绿 —— 而且对回退后的旧条件同样为绿(该标志满足 || criticalsUnverified)→ 本 PR 修复的 239 字符正文问题会在没有任何红色测试的情况下回归。

证据:探针 —— 回退条件并把 criticalsUnverified 翻为 true 后,原有五条断言全部通过;加上建议的断言后变红(expected [ 'unreviewed-dimension', …(2) ] to not include 'criticals-unverified');在 PR 代码下该断言通过。

— qwen3.8-max via Qwen Code /review (v0.21.13)

expect(r.cappedBy).not.toContain('criticals-unverified');
expect(r.body).toContain('**[Critical]** whole-PR blocker X');
});

it('a mixed review keeps its Request changes — the deterministic blocker is confirmed with or without a verifier', () => {
// One [build] Critical (pre-confirmed) beside one non-deterministic
// Critical with the verifier absent: softening the whole event would
Expand Down Expand Up @@ -6701,9 +6785,9 @@ describe("composeReview — the composed body fits GitHub's limit", () => {
// that qualify the verdict before it spent a single blocker.
//
// The cap here is the absent verifier, which still POSTS the blockers
// (clause 7 rides on `criticalsUnverified`); the findings-file tag route
// caps without that flag, so its body carries no blocker and cannot
// reach the cut at all.
// (clause 7 rides on the softened RC→COMMENT transition); the
// findings-file tag route softens the same way, so its body carries the
// blocker too and can reach the cut.
const r = composeReview({
planPath: coveredPlan(['reverse-audit']),
env: ENV,
Expand Down Expand Up @@ -6959,16 +7043,6 @@ describe('composeReview — the findings file tag check', () => {
// round's findings digest — so compose-review reads the cumulative
// findings file itself and caps on any surviving tag.

function findingsFile(content: string): string {
const f = join(dir, 'qwen-review-findings.md');
writeFileSync(f, content);
return f;
}

const TAGGED =
'- **File:** src/pay.ts:42\n' +
'- **Issue:** off-by-one in the retry cap\n' +
'- **Severity:** Critical — [unverified]\n';
const CLEAN =
'- **File:** src/pay.ts:42\n' +
'- **Issue:** off-by-one in the retry cap\n' +
Expand Down
30 changes: 22 additions & 8 deletions packages/cli/src/commands/review/compose-review.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2143,20 +2143,24 @@ function composeReviewBody(
event = 'COMMENT';
}

// Presubmit downgrades apply after the caps and only when the verdict
// they name is the one on the table.
// Presubmit downgrades apply after the caps and only when the verdict they
// name was the one on the table — `baseEvent` is the row before every cap,
// so a softening cap that ran first cannot erase the presubmit's reasons.
let downgraded = false;
let downgradedFrom: 'Approve' | 'Request changes' | null = null;
if (event === 'APPROVE' && downgradeApprove) {
if (
(event === 'APPROVE' || (baseEvent === 'APPROVE' && event === 'COMMENT')) &&
downgradeApprove
) {
event = 'COMMENT';
downgraded = true;
downgradedFrom = 'Approve';
} else if (
(event === 'REQUEST_CHANGES' ||
(baseEvent === 'REQUEST_CHANGES' && criticalsUnverified)) &&
(baseEvent === 'REQUEST_CHANGES' && event === 'COMMENT')) &&
downgradeRequestChanges
) {
// The unverified-blockers cap softened the event first, but the presubmit
// A softening cap moved the event first, but the presubmit
// still ruled: without this arm its reasons (self-PR, failing CI) would
Comment on lines +2163 to 2164

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The derived treatment this diff gives the RC presubmit arm is not carried to the sibling arm directly above it: if (event === 'APPROVE' && downgradeApprove) (~line 2150) still tests the post-cap event. The APPROVE→COMMENT cap (~line 2111) runs before the presubmit arms, so any cap that softens a zero-finding Approve first (contextUnavailable, an unreviewed dimension, a cannot-tell existing Critical, a surviving findings-file tag) fails arm 1's gate — and arm 2's baseEvent === 'REQUEST_CHANGES' gate fails too — leaving downgraded/downgradedFrom false/null, so the presubmit reasons (self-PR, failing CI) vanish from both the body and the verdict line. The behavior predates this diff and its reachability is unchanged (hence Suggestion); it is in scope as the fourth member of the predicate family this diff converts from enumeration to derivation, and because the comment this diff rewrites now asserts the invariant — "without this arm its reasons … would silently vanish" — that the chain's other arm still violates. — Failure scenario: a self-PR review whose run also trips any cap over a zero-finding verdict → the event is softened to COMMENT before the arms run → no "⚠️ Downgraded from Approve to Comment: self-PR" clause in the body and no "a presubmit check failed" sentence in the verdict line, while the identical uncapped run renders both.

Witness (probe in the real vitest harness):

capped:   event=COMMENT baseEvent=APPROVE downgraded=false downgradedFrom=null cappedBy=["context-unavailable"]
          verdictLine: Verdict: Comment — an Approve was NOT available: the PR's existing discussion could not be read
          body: no downgrade clause, no 'self-PR'
control:  downgraded=true downgradedFrom=Approve
          verdictLine: … — an Approve was NOT available: a presubmit check failed
          body: ⚠️ Downgraded from Approve to Comment: self-PR.

Applying the derived gate flips the capped arm to downgraded=true downgradedFrom=Approve with both surfaces restored; the full suite stays green.

Suggested fix (two locations — arm 1's condition, plus a regression test pairing a capped zero-finding run with downgradeApprove: true):

if (
  (event === 'APPROVE' ||
    (baseEvent === 'APPROVE' && event === 'COMMENT')) &&
  downgradeApprove
) {
  downgraded = true;
  downgradedFrom = 'Approve';
}
中文说明

本 diff 给 RC presubmit 分支的推导式处理没有传导到它正上方的姊妹分支:if (event === 'APPROVE' && downgradeApprove)(约 2150 行)仍在检测 cap 之后的 event。APPROVE→COMMENT 的 cap(约 2111 行)先于 presubmit 分支执行,因此任何先把零发现 Approve 软化的 cap(contextUnavailable、未审查维度、cannot-tell 的既有 Critical、findings 文件残留标签)都会使分支 1 的门槛失败——分支 2 的 baseEvent === 'REQUEST_CHANGES' 门槛同样失败——downgraded/downgradedFrom 保持 false/null,presubmit 的理由(自身 PR、CI 失败)从正文与裁决行两处同时消失。该行为在本 diff 之前已存在且可达性未变(故为 Suggestion);它属于本 diff 从枚举转为推导的谓词家族的第四个成员,且本 diff 重写的注释现在断言了「没有该分支其理由会静默消失」这一不变量——链条的另一分支仍然违反它。失败场景:一次自身 PR 的评审在零发现裁决上同时触发任一 cap → event 在分支执行前被软化为 COMMENT → 正文没有「⚠️ Downgraded from Approve to Comment: self-PR」子句、裁决行没有「a presubmit check failed」一句,而同一运行不加 cap 时两处都能渲染。

证据(真实 vitest 探针):cap 组 downgraded=false downgradedFrom=null cappedBy=["context-unavailable"],裁决行只剩「an Approve was NOT available: the PR's existing discussion could not be read」,正文无降级子句、无 'self-PR';同一 presubmit 的无 cap 对照组 downgraded=true downgradedFrom=Approve,两处均正常渲染。应用推导式门槛后 cap 组翻转为 downgraded=true,两处恢复;全套测试保持绿。

建议修复(两处——分支 1 的条件,外加一个把「带 cap 的零发现运行 + downgradeApprove: true」配对的回归测试):

if (
  (event === 'APPROVE' ||
    (baseEvent === 'APPROVE' && event === 'COMMENT')) &&
  downgradeApprove
) {
  downgraded = true;
  downgradedFrom = 'Approve';
}

— qwen3.8-max via Qwen Code /review (v0.21.13)

// silently vanish from the body whenever both held. The verdict line
// keeps the unverified sentence — the more fundamental defect — and the
Expand Down Expand Up @@ -3289,10 +3293,20 @@ function composeReviewBody(
clauses.push(...continuityBlock);

// 7. Body Criticals — on a COMMENT that stands where a REQUEST_CHANGES
// would have been: the presubmit carve-out, and the unverified-blockers
// cap. Either way the body copy is the ONLY copy of an unanchorable
// would have been. The body copy is the ONLY copy of an unanchorable
// blocker, and softening the event must never erase it.
if (downgradedFrom === 'Request changes' || criticalsUnverified) {
//
// DERIVED, not enumerated. The condition was a list of the two
// softening flags known when it was written — the presubmit carve-out
// and `criticalsUnverified` — and a third path shipped past it: the
// findings-file `— [unverified]` tag softens a Request changes at the
// event line above while setting NEITHER flag, so a run whose coverage
// was proven posted a 239-character body carrying the opener and the
// tag disclosure and no blocker at all. `baseEvent` is the row before
// every cap and downgrade, so this comparison asks the question the
// clause is actually about, and answers it for softening paths that do
// not exist yet.
if (baseEvent === 'REQUEST_CHANGES' && event === 'COMMENT') {
clauses.push(...bodyCriticalBlock);
Comment on lines +3309 to 3310

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] This widening falsifies the pre-existing comment at compose-review.test.ts:6734-6737, which still asserts the pre-fix behavior: "(clause 7 rides on criticalsUnverified); the findings-file tag route caps without that flag, so its body carries no blocker and cannot reach the cut at all." Post-fix, the tag route's body DOES carry the blocker (this PR's own new test asserts it) and CAN reach the trim cut — the same file now carries two contradictory descriptions of the tag route. Anchored here because the comment sits outside the diff hunks and this condition change is what invalidates it. — Failure scenario: a maintainer extending the truncation oracles reads the stale comment, concludes a tag-route body can never contain a blocker and never reach the trim cut, and reasons about keep/trim ranking — or "simplifies" an oracle — from pre-fix semantics.

Witness: probe — a tag-route body with a 70k-char blocker reaches the trim cut (r.bodyTrim.truncated === true) under the fix; reverting clause 7 to the pre-fix condition flips the probe (no blocker posted, no truncation).

Suggested fix: rewrite that comment paragraph to the post-fix semantics, e.g. (clause 7 rides on the softened RC→COMMENT transition); the findings-file tag route softens the same way since that fix, so its body carries the blocker too and can reach the cut.

中文说明

本次放宽使 compose-review.test.ts:6734-6737 处的既有注释失真:该注释仍断言修复前的行为 ——「(clause 7 rides on criticalsUnverified); the findings-file tag route caps without that flag, so its body carries no blocker and cannot reach the cut at all.」。修复后,tag 路径的正文确实会携带阻断项(本 PR 自己的新测试就是这样断言的),也确实能到达裁剪切口 —— 同一文件现在对该路径携带了两处相互矛盾的描述。锚定在此是因为该注释位于 diff 块之外,而使其失效的正是这处条件变更。失败场景:维护者扩展截断预言时读到这条陈旧注释,认定 tag 路径正文永远不会包含阻断项、也永远不会到达裁剪切口,从而基于修复前的语义推理 keep/trim 排序,甚至「简化」某个预言。

证据:探针 —— 修复后,携带 7 万字符阻断项的 tag 路径正文会到达裁剪切口(r.bodyTrim.truncated === true);将 clause 7 回退为修复前条件后探针翻转(阻断项未发出、无截断)。

建议修复:将该注释段落改写为修复后的语义,例如:(clause 7 rides on the softened RC→COMMENT transition); the findings-file tag route softens the same way since that fix, so its body carries the blocker too and can reach the cut.

— qwen3.8-max via Qwen Code /review (v0.21.13)

}
Comment on lines +3309 to 3311

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The presubmit recovery arm (lines 2153-2157) still enumerates only criticalsUnverified — the same enumeration disease this clause derives around. When findings-tag softening (findingsUnverifiedAtCompose, which sets neither legacy flag) and presubmit's downgradeRequestChanges both hold, the arm matches neither disjunct, so the presubmit reasons never reach the body or the verdict line — exactly the silent vanishing its own comment forbids. Pre-existing (identical at HEAD1); may reasonably be deferred to a follow-up. — Failure scenario: a run whose findings file still carries — [unverified] at compose time (verifier clean) plus downgradeRequestChanges: true (self-PR / failing CI) → downgraded/downgradedFrom stay null → the "Downgraded from Request changes to Comment: " sentence and its reasons are lost from both the posted body and the terminal verdict line.

Witness: probe — PROBE-A (tag path): downgraded:false, downgradedFrom:null, bodyHasPresubmitReason:false, while the criticalsUnverified control twin in the identical run posts the reasons; a one-line derived fix flips PROBE-A to downgraded:true with the reasons in the body, and all 345 tests stay green with it.

Suggested fix: (event === 'REQUEST_CHANGES' || (baseEvent === 'REQUEST_CHANGES' && event === 'COMMENT')) && downgradeRequestChanges, plus a sibling regression test mirroring the one at ~line 3531 with a tagged findings file instead of an absent verifier.

中文说明

presubmit 恢复分支(约 2153-2157 行)仍只枚举 criticalsUnverified —— 与本子句以推导替代的枚举同病。当 findings 标签软化(findingsUnverifiedAtCompose,两个旧标志均不设置)与 presubmit 的 downgradeRequestChanges 同时成立时,该分支两个析取项都不匹配,presubmit 的理由既进不了正文也进不了裁决行 —— 正是其自身注释所禁止的「静默消失」。属既有问题(HEAD~1 相同);可合理地推迟到后续 PR。失败场景:一次 compose 时 findings 文件仍带 — [unverified](验证器干净)且 presubmit 判定 downgradeRequestChanges: true(自身 PR / CI 失败)的运行 → downgraded/downgradedFrom 保持 null → 「Downgraded from Request changes to Comment: 」一句及其理由同时从发出的正文与终端裁决行中丢失。

证据:探针 —— PROBE-A(tag 路径):downgraded:false, downgradedFrom:null, bodyHasPresubmitReason:false,而同一运行中 criticalsUnverified 对照组能正常发出理由;一行推导式修复即可将 PROBE-A 翻转为 downgraded:true 且理由进入正文,且全部 345 个测试仍为绿。

建议修复:改为 (event === 'REQUEST_CHANGES' || (baseEvent === 'REQUEST_CHANGES' && event === 'COMMENT')) && downgradeRequestChanges,并仿照约 3531 行的测试补一个以带标签 findings 文件替代「验证器缺席」的姊妹回归测试。

— qwen3.8-max via Qwen Code /review (v0.21.13)


Expand Down
Loading