Repository navigation
fix(sandbox): stop blocking bash commands (e.g. servers) from hanging the turn - #3864
Conversation
Starting a server through the host bash tool (e.g. `python -m http.server`) could hang the whole turn for the full 600s timeout. `LocalSandbox.execute_command` used `subprocess.run(capture_output=True)`, whose captured pipes are inherited by any process the command spawns — so a backgrounded long-lived process (`server &`) keeps the read end open and blocks `communicate()` until the timeout fires, even though the foreground command already returned. Commands that read stdin blocked the same way, and on timeout only the direct child was killed, leaving orphaned process groups. Rework the POSIX path to capture stdout/stderr via temp files instead of pipes, take stdin from /dev/null, and run the command in its own session/process group: - Backgrounded long-lived processes (servers) now return immediately while the process keeps running. - A command reading stdin gets immediate EOF instead of blocking. - A genuinely blocking foreground command is bounded by a configurable wall-clock timeout; on timeout the whole process group is killed and the agent gets an explanatory notice telling it to background long-lived processes. The timeout is configurable via `sandbox.bash_command_timeout` (default 600). The Windows path is unchanged. Adds focused regression tests and updates docs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The code fix bounds a foreground server with a timeout, but the turn still waits the full timeout before the run continues. Add the prompt-side half: the bash tool description now tells the model to ALWAYS start long-lived processes (e.g. web servers) in the background with output redirected, so the tool returns immediately. The timeout notice points at the same readable workspace log path. Pins the guidance with a test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address automated review: the inner `except OSError: pass` in _terminate_process_group silently swallowed the case where the direct-child fallback kill found the process already gone. Make the intent explicit with a comment and a debug log instead of a bare pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
willem-bd
left a comment
There was a problem hiding this comment.
Solid fix for the pipe-inheritance hang — temp-file capture + start_new_session + group kill is the right shape for POSIX. Three inline notes below, the first (silent /tmp leak for un-redirected backgrounded processes) is the one I'd most want discussed before merge since it sits on the PR's headline use case.
| args, | ||
| shell=False, | ||
| stdin=subprocess.DEVNULL, | ||
| stdout=out_file, |
There was a problem hiding this comment.
Heads-up on a latent leak: these temp files are dup2'd into the child's fd 1/2, so a backgrounded process (python -m http.server &) inherits them. tempfile.TemporaryFile() on POSIX is anonymous (mkstemp + immediate unlink), so when the surrounding with block closes, the parent's fd is dropped but the background process still holds its dup — every byte it writes thereafter lands in an unnamed /tmp inode that can't be read, truncated, or discovered until the process exits.
That's a silent, unbounded disk leak on exactly the long-running-background case this PR enables, whenever the model omits the > log 2>&1 redirect (the tool docstring nudges it, but nothing enforces it). The old pipe behavior hung the turn; this trades it for a leak. Worth either calling the redirect out as load-bearing in the AGENTS.md note, or making the capture bounded — at minimum flagging it so it's a conscious tradeoff rather than a surprise.
| capture_output=True, | ||
| text=True, | ||
| timeout=600, | ||
| timeout=timeout, |
There was a problem hiding this comment.
On Windows this still calls subprocess.run(..., timeout=timeout), which raises TimeoutExpired uncaught on timeout — timed_out stays False, so the explanatory notice is never produced and the process group is never killed.
The PR now flows bash_command_timeout into this path (via command_timeout in bash_tool), so the contract here did change: a Windows user who lowers it (e.g. bash_command_timeout: 30) gets a raw TimeoutExpired → bash_tool's generic except Exception → "Error: Unexpected error executing command: …" instead of the bounded-with-notice behavior advertised for POSIX. The body says "Windows path is intentionally unchanged," but the knob now reaches it. Either don't pass command_timeout on Windows (keep the hardcoded 600), or catch TimeoutExpired and set timed_out=True to mirror the POSIX path.
| output += f"\nStd Error:\n{stderr}" if output else stderr | ||
| if timed_out: | ||
| notice = ( | ||
| f"Command timed out after {int(timeout)} seconds and was terminated. " |
There was a problem hiding this comment.
Minor: execute_command accepts float | None, but the notice does int(timeout), so a non-integral timeout renders truncated and mispluralized — e.g. timeout=1.5 → "Command timed out after 1 seconds and was terminated.". If float timeouts are meant to be supported (the tests pass 1), round it and handle the singular; otherwise document timeout as integral.
willem-bd
left a comment
There was a problem hiding this comment.
Solid fix for the pipe-inheritance hang — temp-file capture + start_new_session + group kill is the right shape for POSIX. Three inline notes below, the first (silent /tmp leak for un-redirected backgrounded processes) is the one I'd most want discussed before merge since it sits on the PR's headline use case.
… for bytedance#3864) Upstream bytedance#3864 (fe82552) made the shared bash tool call sandbox.execute_command(command, timeout=...) for EVERY provider and updated LocalSandbox to accept timeout, but did NOT update the community AioSandbox provider we run in production. Without this, every agent bash call would raise TypeError: unexpected keyword argument 'timeout'. Accept the kwarg for interface compatibility; the AIO sandbox already bounds commands via its own no_change_timeout, so the param is accepted but not separately applied. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fe82552 LocalSandbox.execute_command 改临时文件捕获+进程组终止+可配置 bash_command_timeout AGENTS.md 冲突取 fork 版(eai AGENTS.md 仅重定向 CLAUDE.md,不用上游完整版,#3770已跳过)
Conflict (1): community/aio_sandbox/aio_sandbox.py — resolved TAKE-UPSTREAM. Upstream bytedance#3871 reworked execute_command to (self, command, env=None, timeout=None), which SUPERSEDES our local timeout= shim (added 2026-06-30 for bytedance#3864) AND adds the new env= for request-scoped skill secrets. Dropping our shim — our fork divergence shrinks by one patch. Normal bash path (env=None) is byte-identical (legacy shell.exec_command); the timeout= call from tools.py no longer TypeErrors. The other 12 auto-merged clean. Notable: bytedance#3904 unified DB config (no-op for us — our checkpointer.type=postgres section takes precedence), bytedance#3903 recursion clamp (ceiling 1000 = our value → no-op), bytedance#3872 csrf cookie lifetime fix (positive), #4fcb subagent step history (persists to existing run_events table, no migration), bytedance#3871 request-scoped secrets. No new alembic migration → head stays 0002. i18n auto-merged, WRI branding preserved. Login/setup pages untouched.
… the turn (bytedance#3864) * fix(sandbox): stop blocking bash commands from hanging the turn Starting a server through the host bash tool (e.g. `python -m http.server`) could hang the whole turn for the full 600s timeout. `LocalSandbox.execute_command` used `subprocess.run(capture_output=True)`, whose captured pipes are inherited by any process the command spawns — so a backgrounded long-lived process (`server &`) keeps the read end open and blocks `communicate()` until the timeout fires, even though the foreground command already returned. Commands that read stdin blocked the same way, and on timeout only the direct child was killed, leaving orphaned process groups. Rework the POSIX path to capture stdout/stderr via temp files instead of pipes, take stdin from /dev/null, and run the command in its own session/process group: - Backgrounded long-lived processes (servers) now return immediately while the process keeps running. - A command reading stdin gets immediate EOF instead of blocking. - A genuinely blocking foreground command is bounded by a configurable wall-clock timeout; on timeout the whole process group is killed and the agent gets an explanatory notice telling it to background long-lived processes. The timeout is configurable via `sandbox.bash_command_timeout` (default 600). The Windows path is unchanged. Adds focused regression tests and updates docs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sandbox): instruct the agent to background long-lived processes The code fix bounds a foreground server with a timeout, but the turn still waits the full timeout before the run continues. Add the prompt-side half: the bash tool description now tells the model to ALWAYS start long-lived processes (e.g. web servers) in the background with output redirected, so the tool returns immediately. The timeout notice points at the same readable workspace log path. Pins the guidance with a test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sandbox): make fallback-kill exception explicit and observable Address automated review: the inner `except OSError: pass` in _terminate_process_group silently swallowed the case where the direct-child fallback kill found the process already gone. Make the intent explicit with a comment and a debug log instead of a bare pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sandbox): address bash timeout review feedback * fix(sandbox): document fd cleanup races --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
… the turn (bytedance#3864) * fix(sandbox): stop blocking bash commands from hanging the turn Starting a server through the host bash tool (e.g. `python -m http.server`) could hang the whole turn for the full 600s timeout. `LocalSandbox.execute_command` used `subprocess.run(capture_output=True)`, whose captured pipes are inherited by any process the command spawns — so a backgrounded long-lived process (`server &`) keeps the read end open and blocks `communicate()` until the timeout fires, even though the foreground command already returned. Commands that read stdin blocked the same way, and on timeout only the direct child was killed, leaving orphaned process groups. Rework the POSIX path to capture stdout/stderr via temp files instead of pipes, take stdin from /dev/null, and run the command in its own session/process group: - Backgrounded long-lived processes (servers) now return immediately while the process keeps running. - A command reading stdin gets immediate EOF instead of blocking. - A genuinely blocking foreground command is bounded by a configurable wall-clock timeout; on timeout the whole process group is killed and the agent gets an explanatory notice telling it to background long-lived processes. The timeout is configurable via `sandbox.bash_command_timeout` (default 600). The Windows path is unchanged. Adds focused regression tests and updates docs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sandbox): instruct the agent to background long-lived processes The code fix bounds a foreground server with a timeout, but the turn still waits the full timeout before the run continues. Add the prompt-side half: the bash tool description now tells the model to ALWAYS start long-lived processes (e.g. web servers) in the background with output redirected, so the tool returns immediately. The timeout notice points at the same readable workspace log path. Pins the guidance with a test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sandbox): make fallback-kill exception explicit and observable Address automated review: the inner `except OSError: pass` in _terminate_process_group silently swallowed the case where the direct-child fallback kill found the process already gone. Make the intent explicit with a comment and a debug log instead of a bare pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sandbox): address bash timeout review feedback * fix(sandbox): document fd cleanup races --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Problem
Starting a server through the host bash tool — e.g.
python -m http.server— could hang the entire turn for the full 600s timeout.LocalSandbox.execute_commandran commands viasubprocess.run(capture_output=True, timeout=600). The captured stdout/stderr pipes are inherited by every process the command spawns, so:server &) keeps the read end open →communicate()blocks until the timeout fires, even though the foreground command already returned and the user correctly backgrounded it.read x) blocked until the timeout because stdin was inherited rather than/dev/null.Fix
1. Code (
execute_command, POSIX path). Run commands with stdin from/dev/null, a dedicated session/process group, and bounded pipe-drain capture:server > log 2>&1 &(backgrounded)server &without redirectserver(foreground, no&)read x(reads stdin)The wall-clock timeout is configurable via
sandbox.bash_command_timeout(default 600, unchanged default behavior). Timeout notices now format fractional/singular values correctly.2. Windows timeout path. Windows still uses the existing
subprocess.run(...)call shape, butTimeoutExpiredis now caught so loweredbash_command_timeoutvalues produce the same explanatory timeout notice instead of bubbling into the generic unexpected-error path.3. Prompt/docs. The
bashtool description still instructs the model to background long-lived processes (e.g. servers) with output redirected, so logs stay inspectable and the tool returns immediately.README.md,backend/AGENTS.md, andconfig.example.yamldescribe the timeout/backgrounding behavior.Tests
backend/tests/test_local_sandbox_command_timeout.pycovers backgrounded prompt return, unredirected background stdout not inheriting deleted temp capture on Linux, foreground timeout notice, whole-process-group kill, stdin EOF, ordinary stdout/stderr/exit-code behavior, config default, bash-tool guidance, WindowsTimeoutExpired, and fractional/singular timeout wording.backend/tests/test_local_sandbox_provider_mounts.pyfor thePopen-based path.cd backend && make formatcd backend && make lintcd backend && PYTHONPATH=. uv run pytest tests/test_local_sandbox_command_timeout.py tests/test_local_sandbox_provider_mounts.py tests/test_local_sandbox_encoding.py tests/test_local_sandbox_virtual_path_contract.py -qcd backend && make test→5199 passed, 20 skippedDocs
Updated
README.md,backend/AGENTS.md, andconfig.example.yaml.🤖 Generated with Claude Code