test(e2e): inherit suite timeout in flaky subagent cases (#8244) - #8246
Merged
Conversation
The two subagent-execution E2E cases capped their timeout at 60s, below the suite's configured real-model E2E budget (TB_TIMEOUT_MINUTES, default 5m) that every other case in the file inherits. A delegated run is multi-turn (main agent delegates, subagent reads and reports, main agent summarizes), so under the slower docker sandbox plus CI load and model rate-limiting it can exceed 60s and fail all retry attempts. The failing main-branch run failed only in sandbox:docker while the same shard passed in sandbox:none, implicating timing rather than model nondeterminism. Drop the per-case 60s overrides so these cases use the same configured timeout as the rest of the suite. No assertions change. Co-authored-by: Qwen Code Autofix <qwen-code-autofix@users.noreply.github.com>
Collaborator
Author
Autofix E2E Report — #8244Failing test: DiagnosisThis is a timing/environment flake in the real-model E2E suite, not a logic bug:
FixRemoved the two per-case The raw CI log was not retrievable (the logs endpoint requires admin rights), so the diagnosis is built from the public job/annotation data plus code and history inspection. The exact docker E2E environment (real model credentials + docker sandbox) is not available on this runner; that environment-specific run is left to the workflow's independent CI as the final gate. Verification
中文说明Autofix E2E 报告 — #8244失败用例： 诊断这是真实模型 E2E 套件中的时序/环境性偶发失败，而非逻辑缺陷：
修复移除了这两处用例级的 原始 CI 日志无法获取（日志接口需要管理员权限），因此诊断基于公开的 job/annotation 数据以及代码与历史检查构建。精确的 docker E2E 环境（真实模型凭证 + docker 沙箱）在本运行机上不可用；该环境相关的运行交由工作流的独立 CI 作为最终关卡。 验证
🧠 Handled by Qwen Code · model/模型 |
Collaborator
|
✅ Qwen Triage finished — CI landed green on ✅ Qwen Triage 已完成 —— |
github-actions
Bot
added
the
review/self-reported
Collaborator
|
Thanks for the PR! Template looks good ✓ Problem: observed, not theoretical. Linked issue #8244 is a main-branch Direction: aligned. This is a test-infrastructure flake fix. I verified the claim against the code: Size: not applicable — one test file, +2/−2, zero production logic lines. No core paths touched, so the Stage 0 two-tier gate does not apply. Approach: minimal and consistent. These were the only two per-case timeouts left in the file; dropping them makes both cases inherit the suite budget just like every other case. No assertions or test logic change, no scope creep, and I don't see a simpler path. Risk: no elevated risk signals. The Stage 1e high-risk-path check excludes Moving on to code review. 🔍 中文说明感谢贡献！ 模板完整 ✓ 问题： 已观测到，非理论性问题。关联的 issue #8244 是 main 分支 方向： 对齐。这是一个测试基础设施的 flake 修复。我已对照代码核实其说法： 规模： 不适用——单个测试文件，+2/−2，零生产逻辑行。未触及核心路径，Stage 0 两级门控不适用。 方案： 最小且一致。这是文件中仅剩的两个用例级超时；去掉后两个用例与文件中其他所有用例一样继承套件预算。未改动任何断言或测试逻辑，无范围蔓延，也没有更简路径。 风险： 无升级风险信号。Stage 1e 高风险路径检查会排除 进入代码审查 🔍 — Qwen Code · qwen3.8-max-preview Reviewed at |
Collaborator
Code reviewBefore reading the diff, my own take on the flake: a multi-turn real-model delegated flow capped at 60 s, failing only in the slower The diff is two lines plus a stale comment, and I confirmed against the code in
No correctness, security, or regression concerns; no AGENTS.md violations. A genuine hang still fails — at the suite's 5-minute budget rather than 60 s, which is the accepted tradeoff for every other real-model case here. TestingThis is an unattended CI run, so no local real-scenario capture applies; for a test-only change with no user-visible product behavior the product-level before/after is N/A. The evidence below is the PR's own CI signal, quoted by real check name and conclusion. The Linux suite ( Final CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行（取最新一次运行），省略 skipped，失败项排在最前。 On whether a sandboxed lane settles this: the change carries no product-behavior claim — the product surface is unchanged and the only behavior affected is the test's own timeout budget. 中文说明代码审查在读 diff 之前，我对这个 flake 的独立判断是：一个多轮真实模型委派流程被限制在 60 s，仅在更慢的 diff 只有两行加一处过时注释，我已对照
无正确性、安全或回归问题；无 AGENTS.md 违规。真正的卡死仍会失败——只是在套件的 5 分钟预算下而非 60 s，这是此处所有其他真实模型用例所接受的权衡。 测试这是无人值守的 CI 运行，因此不适用本地真实场景捕获；对于无用户可见产品行为的纯测试改动，产品级 before/after 为 N/A。下方证据是 PR 自身的 CI 信号，按真实检查名与结论引用。 Linux 套件（ 关于沙箱通道是否能定论：本改动不含产品行为声明——产品面未变，唯一受影响的行为是测试自身的超时预算。 — Qwen Code · qwen3.8-max-preview Reviewed at |
Collaborator
|
Confidence: 4/5 — a clean, minimal, well-justified flake fix; the only reservation is that the fix is only confirmable by the E2E Stepping back: this is the right fix done the right way. The problem is real and observed — #8244 is a main-branch E2E failure on exactly this case, twice, in the slower docker shard and through The diff is exactly the minimal set — two timeout args and a stale comment, nothing driven-by, no scope creep. If I had to maintain this in six months I would not give it a second thought. I am approving because it is genuinely correct and needed, not because I ran out of objections. The one thing I cannot yet attest to is that the flake is actually gone — that lives in the real-model docker E2E run, which PR CI does not carry (no model credentials / docker sandbox here) and which has not reported on this commit. That is inherent to a flake fix rather than a defect of this PR, and it is why approval is conditional on CI rather than immediate. Approval deferred until CI lands green on 中文说明置信度：4/5 —— 一个干净、最小、论证充分的 flake 修复；唯一的保留是：该修复只能由 E2E 退一步看：这是用正确方式做的正确修复。问题真实且已观测——#8244 是 main 分支 E2E 恰在此用例上的失败，两次，发生在更慢的 docker shard，且即便 diff 恰为最小集合——两个超时参数与一处过时注释，无顺手改动，无范围蔓延。若半年后维护它，我不会多想。我批准是因为它确实正确且必要，而非因为没有反对意见。 唯一尚不能背书的是 flake 是否真的消除——那取决于真实模型 docker E2E 运行，而 PR CI 不携带它（此处无模型凭证 / docker 沙箱），也未在此提交上报告。这是 flake 修复的固有属性，而非本 PR 的缺陷，也正是批准以 CI 为条件、而非立即生效的原因。 批准将推迟到 CI 在该提交上转绿之后。 — Qwen Code · qwen3.8-max-preview Reviewed at |
Contributor
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
qwen-code-ci-bot
approved these changes
Jul 31, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, looks ready to ship — CI landed green after the review. ✅
qwen-code-ci-bot
approved these changes
Jul 31, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
No issues found. LGTM! ✅
中文说明
未发现问题。LGTM！✅
— qwen3.8-max-preview via Qwen Code /review
wenshao
approved these changes
Jul 31, 2026
This was referenced Jul 31, 2026
Collaborator
|
Released in v0.21.3. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Removes the two per-case
60000ms timeout overrides on theSubagent ExecutionE2E cases in the SDK TypeScript integration suite, so those cases inherit the suite's configured real-model E2E timeout (TB_TIMEOUT_MINUTES, default 5 minutes) just like every other case in the file. No assertions or test logic change.Why it's needed
The delegated subagent flow is multi-turn real-model work — the main agent delegates, the subagent reads a file and reports back, and the main agent summarizes — roughly four model round-trips. Capping that at 60 seconds sits below the budget the suite deliberately configures for real-model E2E cases. The failing main-branch run for issue #8244 failed only in the slower
sandbox:dockershard while the same shard passed insandbox:none, and it failed despite the suite'sretry: 2, which points at a slow-under-load delegated run timing out prematurely rather than a model misbehaving. Letting these cases use the same configured timeout as the rest of the suite removes that premature cutoff without weakening any check; a genuine hang still fails at the 5-minute budget.Reviewer Test Plan
How to verify
Confirm the change is timeout-only and that the cases still register:
, 60000argument (and a now-stale comment) from the twoSubagent Executioncases; everyexpect(...)is untouched.cd integration-tests && npx vitest list sdk-typescript/subagents.test.ts— all 11 cases, includingshould delegate task to subagent when appropriateandshould complete simple task with subagent, should be listed.sandbox:dockershard that failed for Main CI failed: E2E Tests — sdk-typescript/subagents.test.ts > … > should delegate task to subagent when appropriate #8244 should no longer time out on these cases under load.Evidence (Before & After)
N/A — non-user-visible test-infrastructure change (test timeout argument only).
Tested on
Environment (optional)
Linux runner:
npm run build,npm run typecheck, ESLint on the touched file, andvitest listcollection. The real-model docker E2E execution of the touched case was not run locally (requires model credentials and a docker sandbox unavailable here) and is deferred to CI.Risk & Scope
Linked Issues
Fixes #8244
中文说明
本 PR 做了什么
移除 SDK TypeScript 集成套件中两个
Subagent ExecutionE2E 用例上用例级的60000毫秒超时覆盖，使它们继承套件为真实模型 E2E 配置的超时（TB_TIMEOUT_MINUTES，默认 5 分钟），与文件中其他所有用例一致。未改动任何断言或测试逻辑。为什么需要
委派的子 agent 流程是多轮真实模型调用——主 agent 委派、子 agent 读取文件并回报、主 agent 汇总——大约四次模型往返。把它限制在 60 秒，低于套件专门为真实模型 E2E 用例配置的预算。issue #8244 对应的 main 分支失败运行仅在更慢的
sandbox:dockershard 中失败，而同一 shard 在sandbox:none中通过，并且即便套件配置了retry: 2仍然失败——这指向负载下偏慢的委派运行过早超时，而非模型出错。让这些用例使用与套件其余用例相同的配置超时，消除了这一过早截断，同时不削弱任何检查；真正的卡死仍会在 5 分钟预算下失败。评审者测试计划
如何验证
确认改动仅涉及超时，且用例仍能正常注册：
Subagent Execution用例末尾的, 60000参数（以及一处现已过时的注释）；所有expect(...)均未改动。cd integration-tests && npx vitest list sdk-typescript/subagents.test.ts——应列出全部 11 个用例，包括should delegate task to subagent when appropriate与should complete simple task with subagent。sandbox:dockershard 在这些用例上不应再因负载而超时。证据（改动前后）
N/A——非用户可见的测试基础设施改动（仅测试超时参数）。
测试环境
环境（可选）
Linux 运行机：
npm run build、npm run typecheck、对改动文件的 ESLint，以及vitest list收集。改动用例的真实模型 docker E2E 执行未在本地运行（需要此处不可用的模型凭证与 docker 沙箱），交由 CI 验证。风险与范围
关联 Issue
Fixes #8244