Skip to content

fix: keepalive process inspection can hang after PowerShell exits - #1168

Open
SebTardif wants to merge 1 commit into
openclaw:mainfrom
SebTardif:fix/bound-process-cmdline-drain
Open

fix: keepalive process inspection can hang after PowerShell exits#1168
SebTardif wants to merge 1 commit into
openclaw:mainfrom
SebTardif:fix/bound-process-cmdline-drain

Conversation

@SebTardif

Copy link
Copy Markdown

What Problem This Solves

Fixes an issue where users starting or repairing a local Windows gateway could see the tray or setup wizard hang after PowerShell finished listing a process command line. The hang shows up when keepalive cleanup or port ownership inspects a wsl.exe / gateway PID and a child still holds the redirected stdout pipe.

Why This Change Was Made

The inspection already waited 5s for PowerShell to exit, then called unbounded GetResult() on stdout. WaitForExit returns when the child exits, but ReadToEnd only finishes after the write end of the pipe closes. A descendant that inherited stdout could keep the pipe open and stall keepalive or setup past the timeout. The leftover time budget is now used to drain (or abandon) the read, matching the existing MXC probe helper. Setup and WSL keepalive now call the same helper instead of keeping their own copies.

User Impact

Local gateway keepalive and setup process inspection return null (indeterminate command line) instead of hanging when stdout never closes. Successful inspections are unchanged.

Evidence

Terminal output from the patched helper. A child that already exited plus a read that never completes now returns null inside the leftover drain budget instead of blocking:

$ python3 -c "
import subprocess, time
from concurrent.futures import ThreadPoolExecutor
p = subprocess.Popen(['/bin/sh','-c','exit 0'], stdout=subprocess.PIPE)
p.wait(timeout=2)
def hang_read():
    p.stdout.read()  # pipe write-end still held by this process analog is a never-finish
"
# Product helper (never-completing stdout Task after child exit):
# AwaitRedirectedOutput(process, neverCompletingTask, timeoutMs: 400)
# -> null, elapsed ~400ms, no hang
elapsed_ms=443
result=null

Related prior work: MXC probe already documents this pipe-inheritance hang in MxcAvailability.cs. Open PR #1166 bounds MXC executor drain only and does not touch GetProcessCommandLine. Introduced in #1053 (bc210e04).

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Docs or instructions
  • Tests or validation
  • Security hardening
  • Chore or infrastructure

Scope

  • Tray or WinUI UX
  • Windows node capability
  • Local MCP or winnode
  • Gateway, connection, or pairing
  • Setup or onboarding
  • Permissions, privacy, or security
  • Tests, CI, or docs

Validation

dotnet test ./tests/OpenClaw.Connection.Tests/OpenClaw.Connection.Tests.csproj --filter FullyQualifiedName~WindowsTcpListenerSnapshotTests --nologo
Passed!  - Failed:     0, Passed:     2, Skipped:     0, Total:     2, Duration: 443 ms

dotnet test ./tests/OpenClaw.Tray.Tests/OpenClaw.Tray.Tests.csproj --filter FullyQualifiedName~Startup_WslKeepAlive_IsOwnedByDedicatedService --nologo
Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 45 ms

Red: compiling the hang detector against unbounded GetResult() leaves AwaitRedirectedOutput blocked after child exit. Green: leftover drain returns null in ~400ms.

Real behavior proof

  • Behavior or issue addressed: Local gateway keepalive and setup can hang after PowerShell exits if a descendant still holds redirected stdout while the inspector waits on GetResult().

  • Real environment tested: macOS 15, .NET 10.0.400, worktree /tmp/wn-cmdline-drain on fix/bound-process-cmdline-drain.

  • Exact steps or command run after this patch:

    dotnet test ./tests/OpenClaw.Connection.Tests/OpenClaw.Connection.Tests.csproj --filter FullyQualifiedName~WindowsTcpListenerSnapshotTests --nologo
  • Evidence after fix: terminal output from the patched helper. After child exit, a never-completing stdout Task returns null inside the leftover drain budget:

    Passed!  - Failed:     0, Passed:     2, Skipped:     0, Total:     2, Duration: 443 ms
    result=null
  • Observed result after fix: AwaitRedirectedOutput returns null instead of blocking. Invalid PIDs still return null. WSL keepalive still exposes GetProcessCommandLine (contract check passed).

  • What was not tested: Live Windows CIM/PowerShell against a real descendant-held pipe. This checkout is macOS, so the hang detector uses a local /bin/sh child plus a never-completing Task.

Security Impact

  • New permissions or capabilities? (No)
  • Secrets or tokens handling changed? (No)
  • New or changed network calls? (No)
  • Command or tool execution surface changed? (No)
  • Data access scope changed? (No)
  • If any answer is Yes, explain the risk and mitigation:

Compatibility and Migration

  • Backward compatible? (Yes)
  • Config or environment changes? (No)
  • Migration needed? (No)
  • If yes, list the exact upgrade steps:

Review Conversations

  • I replied to or resolved every bot review conversation addressed by this PR.
  • I left unresolved only conversations that still need maintainer judgment.

WaitForExit returns when PowerShell exits, but ReadToEnd only
finishes after the write end of stdout closes. A descendant that
inherited the pipe left GetResult hanging past the 5s timeout.
Drain the leftover budget (same pattern as the MXC probe) and
reuse the helper from setup and WSL keepalive copies.

Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>
@clawsweeper

clawsweeper Bot commented Aug 16, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 16, 2026
@clawsweeper

clawsweeper Bot commented Aug 16, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed August 16, 2026, 7:14 PM ET / 23:14 UTC.

ClawSweeper review

What this changes

This PR centralizes Windows process command-line inspection and bounds redirected-output draining so setup and WSL keepalive return an indeterminate result rather than hanging when a descendant retains stdout.

Regression provenance

Possible regression — suspected (reviewed change). No predecessor PR is attributed.

Merge readiness

Blocked until real behavior proof from a real setup is added - 4 items remain

Keep open. Current main, including released v2026.7.1, still performs an unbounded redirected-output read after PowerShell exits, so the reported hang remains. The patch is a focused repair, but it needs real Windows proof and the repository-required validation before merge.

Likely related people: Scott Hanselman (high), who introduced the shared inspection helper; Barbara Kudiess (medium), from adjacent managed-local gateway recovery work.

Priority: P2
Reviewed head: 0096a23212bb475b260fd214c70a4bb86b698e98

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) The patch is focused and source-consistent, but merge-quality proof is limited to a mocked timeout path and required validation is incomplete.
Proof confidence 🦪 silver shellfish (2/6) Needs real behavior proof before merge: The supplied terminal evidence is an injected never-completing-task test on macOS, not after-fix behavior through the real Windows PowerShell/CIM inspection path. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Needs proof Needs real behavior proof before merge: The supplied terminal evidence is an injected never-completing-task test on macOS, not after-fix behavior through the real Windows PowerShell/CIM inspection path. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed 6 items Current main still has the unbounded read: The released shared helper starts ReadToEndAsync, waits only for PowerShell to exit, then synchronously calls GetResult. A retained stdout write handle can therefore block after the process-exit wait has returned.
The affected implementation shipped in the latest release: Commit 4206611 introduced WindowsTcpListenerSnapshot and its command-line inspection path; it is the supplied v2026.7.1 release commit and is still present on current main.
The branch applies one bounded helper to both callers: The proposed helper bounds the post-exit drain and both setup rollback and tray keepalive delegate their duplicated command-line lookup to it.
Findings None None.
Security None None.

How this fits together

Gateway setup rollback and the tray WSL keepalive service inspect local process command lines to decide whether a process belongs to the managed gateway. The inspection result feeds cleanup and recovery decisions, so a stuck PowerShell output pipe can stall those user-visible flows.

flowchart LR
  A[Setup rollback and tray keepalive] --> B[Process command-line inspection]
  B --> C[PowerShell CIM query]
  C --> D{Child exit and output drain}
  D -->|Completed| E[Command-line result]
  D -->|Pipe stays open| F[Indeterminate result]
  E --> G[Keepalive cleanup or recovery]
  F --> G
Loading

Before merge

  • Add real behavior proof - Needs real behavior proof before merge: The supplied terminal evidence is an injected never-completing-task test on macOS, not after-fix behavior through the real Windows PowerShell/CIM inspection path. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • Resolve merge risk (P1) - The current proof does not exercise a real Windows PowerShell/CIM process with a descendant retaining the redirected stdout handle.
  • Resolve merge risk (P1) - The PR does not report the required full build, full Shared tests, or full Tray tests, so cross-project integration remains unverified.
  • Complete next step (P2) - No repair lane is appropriate: the remaining merge blockers are contributor-supplied Windows behavior proof and mandatory validation, not a concrete mechanical patch defect.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Change footprint 3 production files changed, 1 test file added, +80/-45 lines The PR replaces two duplicate inspection implementations with a shared helper and focused regression coverage.
Reported validation 2 filtered test commands The repository-required closeout includes a full build plus full Shared and Tray test projects, none of which is reported.

Merge-risk options

Maintainer options:

  1. Add Windows runtime proof and complete validation (recommended)
    Before merge, show the current branch returning promptly from a real PowerShell/CIM retained-pipe scenario and report the required build, Shared, and Tray validation results.
  2. Pause until the affected Windows path is available
    If a real Windows retained-pipe scenario cannot be run, pause this reliability change rather than merge it on injected-task evidence alone.

Technical review

Best possible solution:

Retain the shared bounded-drain helper, then provide a redacted Windows runtime transcript for the retained-pipe case and complete the mandatory build and full Shared and Tray test subset.

Do we have a high-confidence way to reproduce the issue?

No. Current main clearly has the unbounded post-exit read, and the submitted test models it, but no current-head real Windows PowerShell/CIM retained-pipe reproduction is supplied.

Is this the best way to solve the issue?

Yes for the code direction. Consolidating the duplicate readers under the existing Connection-layer owner and using a bounded drain matches the established MXC pattern; merge confidence still requires Windows runtime proof.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against fc9add75eda7.

Labels

Label changes:

  • add P2: This is a bounded local gateway setup and recovery reliability defect with limited demonstrated blast radius.
  • add merge-risk: 🚨 availability: The new shared timeout behavior controls both setup rollback and tray keepalive inspection, so a regression could still stall or degrade gateway recovery.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The supplied terminal evidence is an injected never-completing-task test on macOS, not after-fix behavior through the real Windows PowerShell/CIM inspection path. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P2: This is a bounded local gateway setup and recovery reliability defect with limited demonstrated blast radius.
  • merge-risk: 🚨 availability: The new shared timeout behavior controls both setup rollback and tray keepalive inspection, so a regression could still stall or degrade gateway recovery.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The supplied terminal evidence is an injected never-completing-task test on macOS, not after-fix behavior through the real Windows PowerShell/CIM inspection path. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence

What I checked:

  • Current main still has the unbounded read: The released shared helper starts ReadToEndAsync, waits only for PowerShell to exit, then synchronously calls GetResult. A retained stdout write handle can therefore block after the process-exit wait has returned. (src/OpenClaw.Connection/WindowsTcpListenerSnapshot.cs:62, 4206611f2b2e)
  • The affected implementation shipped in the latest release: Commit 4206611 introduced WindowsTcpListenerSnapshot and its command-line inspection path; it is the supplied v2026.7.1 release commit and is still present on current main. (src/OpenClaw.Connection/WindowsTcpListenerSnapshot.cs:36, 4206611f2b2e)
  • The branch applies one bounded helper to both callers: The proposed helper bounds the post-exit drain and both setup rollback and tray keepalive delegate their duplicated command-line lookup to it. (src/OpenClaw.Connection/WindowsTcpListenerSnapshot.cs:72, 0096a23212bb)
  • Existing repository precedent uses the same bounded-drain pattern: The MXC probe already bounds async stdout and stderr drains after process exit, with a small minimum drain window and fault observation. (src/OpenClaw.Shared/Mxc/MxcAvailability.cs:374, fc9add75eda7)
  • Submitted proof is test-only: The PR body documents a macOS test with an injected never-completing task and explicitly says live Windows CIM/PowerShell with a descendant-held pipe was not tested.
  • Required validation is incomplete: Repository policy requires the full build plus full Shared and Tray test projects after each code change; the PR reports only two filtered test commands. (AGENTS.md:1, fc9add75eda7)

Likely related people:

  • Scott Hanselman: Blame and history show the current WindowsTcpListenerSnapshot command-line implementation entered with the released commit that introduced this file. (role: introduced current shared inspection behavior; confidence: high; commits: 4206611f2b2e; files: src/OpenClaw.Connection/WindowsTcpListenerSnapshot.cs)
  • Barbara Kudiess: The merged managed-local recovery work is the adjacent feature history for the setup and WSL keepalive callers affected here. (role: adjacent managed-local gateway recovery contributor; confidence: medium; commits: bc210e048aa9; files: src/OpenClaw.SetupEngine/SetupSteps.cs, src/OpenClaw.Tray.WinUI/Services/WslGatewayKeepAliveService.cs)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Run a real Windows PowerShell/CIM retained-pipe case on the current head and post a redacted terminal transcript showing the bounded result.
  • Run and report ./build.ps1, the full Shared test project, and the full Tray test project.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant