Skip to content

ci(windows): diagnose flaky Runtime Host Local IPC startup failures #3237

Description

@M4n5ter

Summary

Windows GitHub-hosted runners intermittently fail Runtime Host startup checks on identical source trees. The failure moves between the dedicated Local IPC trust-boundary workflow and the installed CLI first-run smoke, while reruns of the same code pass.

This currently makes PR checks non-deterministic and the CLI smoke exposes only INTERNAL_STARTUP_FAILURE, deleting the isolated data root before the underlying Host diagnostic can be inspected.

Evidence

Observed on #3183:

  • Head bded591:
  • Empty retrigger commit 5da258d has the exact same Git tree (b6cad43ab27a6dee887b7285877a097a5e84b875) as bded591:
  • The failed CLI runner was windows-2025-vs2026 in westus; the immediately preceding successful Windows CLI run used the same image version in centralus.
  • The only base change between the two PR merge refs was fix(ui): restore usable scrolling for one-line code blocks #3170, limited to Desktop/UI scrolling and selection files; it did not touch Runtime Host control endpoints, ACL handling, or CLI startup.
  • The same contemporary main revision passed the dedicated Windows recovery workflow.

This rules out a deterministic #3183 code regression and points to a runner-sensitive Local IPC directory/ACL or startup race.

Expected outcome

  • A failed installed-CLI smoke prints or uploads the underlying Runtime Host diagnostic instead of only INTERNAL_STARTUP_FAILURE.
  • On Windows failure, capture the relevant storage/control paths, runner identity/region, directory owner and ACL state, and registration/service logs before the isolated root is deleted.
  • Determine why prepareStorageRootControlDirectory or the subsequent first-run recovery intermittently fails on windows-2025 hosted runners.
  • Stabilize the checks without weakening the Local IPC trust boundary or broadly retrying genuine product failures.

Reproduction status

Intermittent on GitHub-hosted Windows runners; not reproduced deterministically. Two identical PR trees produced disjoint failures in consecutive runs.

Activity

  1. self-assigned this
    on Aug 19, 2026
  2. Astro-Han commented on Aug 19, 2026

    @Astro-Han
    Contributor
    English

    #3235 should cover the windows_recovery half of this. The failure cited above (run 32222355735) spent 20s on the trust-boundary step where a healthy run takes ~7s — that matches the 10s ACL timeout firing on a slow PowerShell cold start, not a real ACL error. The budget is now 30s, and the fixture readiness deadline 45s.

    The installed-CLI half is untouched: INTERNAL_STARTUP_FAILURE still hides the underlying diagnostic.

    One more case of a swallowed diagnostic, same family: the finally block in scripts/windows-runtime-host-local-ipc-trust.ps1 (185–207) throws on a non-zero fixture exit code, replacing whatever exception was already propagating. That is why the original #3225 log showed the ACL message rather than the readiness timeout that fired first.

    中文

    #3235 应该已经覆盖了这里的 windows_recovery 那一半。上面引用的那次失败(run 32222355735)在 trust-boundary 步骤上花了 20s,而健康时约 7s——这与 10s ACL 超时打在 PowerShell 冷启动上吻合,不是真的 ACL 错误。该预算现在是 30s,fixture readiness deadline 是 45s。

    installed-CLI 那一半没有动:INTERNAL_STARTUP_FAILURE 仍然盖住了底层诊断。

    同一族里还有一处诊断被吞:scripts/windows-runtime-host-local-ipc-trust.ps1 的 finally 块(185–207 行)在 fixture 退出码非 0 时 throw,会替换掉当时正在传播的异常。这正是 #3225 最初那条日志显示 ACL 消息、而不是先触发的 readiness 超时的原因。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions