Keep stdin open for can_use_tool and allow it with string prompts

[qing-ant]

Summary

Fixes #1105.

The can_use_tool permission callback is served over the control protocol just like hooks and SDK MCP servers: the CLI writes a control_request to stdout and blocks until the SDK writes the verdict back on stdin. But Query.wait_for_result_and_end_input() only held stdin open when hooks or SDK MCP servers were configured, so query() with a finite prompt and only can_use_tool closed stdin as soon as the prompt had been written. Every later permission request then failed CLI-side with Tool permission request failed: Error: Stream closed, the callback never ran, and the model burned turns retrying the tool. The documented workaround (register a no-op PreToolUse hook) worked only because it flipped this same condition.

Changes:

• Add can_use_tool to the hold condition, factored into Query._has_bidirectional_needs() (mirrors the TypeScript SDK's hasBidirectionalNeeds, which already includes canUseTool).
• Drop the "can_use_tool callback requires streaming mode" ValueError from query() and ClaudeSDKClient.connect(). String prompts have been streamed over stdin internally for a long time (the transport always runs --input-format stream-json), so with the hold in place they work with can_use_tool too — again matching the TypeScript SDK, which accepts canUseTool with a string prompt. This removes the trap where the only accepted query() shape was exactly the one that was broken.

Overlaps with #1106 (same one-clause core fix); this PR additionally lifts the string-prompt restriction and adds contract-level tests. Happy to rebase onto / defer to #1106 if preferred.

Behavior notes

• query(prompt=<str>) / query(prompt=<AsyncIterable>) without can_use_tool, hooks or SDK MCP servers: unchanged — stdin still closes as soon as the prompt is written.
• With can_use_tool: stdin now closes after the run-ending result (same lifecycle hooks/SDK MCP servers already use, including the in-flight background-task handling from SDK closes stdin on first result frame, breaking bidirectional control from nested Task/Agent subagents #1088).
• Docs follow-up: the "dummy hook keeps the stream open" workaround in the permissions / user-input guide can be removed once this ships.

Test plan

• New unit tests in tests/test_query.py:
  • TestNoTimeoutForHooksAndMcpServers::test_can_use_tool_waits_for_result — can_use_tool alone holds stdin until the result event.
  • TestCanUseToolKeepsStdinOpen — a mock transport that enforces the real CLI contract (the can_use_tool control request only arrives after the prompt is written; the assistant/result frames only arrive after the verdict is written; any write after end_input() raises). Covers both an AsyncIterable prompt and a string prompt. Both fail on main (RuntimeError: stdin closed / ValueError: ...requires streaming mode) and pass here.
• New e2e tests in e2e-tests/test_tool_permissions.py drive query() against the real CLI with can_use_tool as the only control-protocol consumer (Bash disallowed, Write to a path outside cwd so nothing can auto-approve it), for both prompt shapes. On main the AsyncIterable variant reproduces Tool permission request failed: AbortError: Stream closed and the callback log stays empty; with this change the callback fires for Write and the file is created.
• ruff check, ruff format, mypy src/, full pytest tests/ (1369 passed).

Review follow-up (76d4724)

• Extracted _configure_can_use_tool() / _hooks_to_internal_format() into types.py; query() and ClaudeSDKClient.connect() now share them (no behavior change).
• wait_for_result_and_end_input() docstring now states exactly what the hold detects and documents the pre-existing multi-message AsyncIterable limitation (one-shot event; tracked as a follow-up).

[claude]

Beyond the inline nits, I also checked that nothing else still depends on the removed "requires streaming mode" ValueError — no tests or docs in the repo reference it — and that dropping the check in ClaudeSDKClient.connect() is safe for the interactive client, since that path never closes stdin (only query()'s one-shot path does, and it now holds stdin open for can_use_tool).

Extended reasoning...

Two nit-level findings were reported inline (code duplication between client.py and _internal/client.py, and a pre-existing multi-message AsyncIterable limitation of the one-shot _first_result_event). Beyond those, I verified there are no stale references to the removed streaming-mode ValueError anywhere in tests or docs, and that the ClaudeSDKClient path is unaffected by the check removal because it keeps stdin open for the whole session regardless. Not approving: the change alters stdin-close lifecycle semantics on the core query path and relaxes a public-API restriction, and the author has explicitly asked maintainers whether to defer to the overlapping #1106 — a coordination decision for a human.

[claude]

I re-reviewed after the two follow-up commits and found no bugs — all earlier inline comments are now addressed in the code — but since this changes the tool-permission callback path and stdin lifecycle, a human look before merge is still worthwhile.

What was reviewed:

• The _configure_can_use_tool() / _hooks_to_internal_format() extraction in types.py — checked line-by-line against both deleted copies; behavior-preserving, and the string-prompt ValueError removal is the only intended difference.
• The reworked stream_input() failure paths — the written == 0 guard closes stdin immediately, and the raise-after-one-message path still completes the in-flight permission round-trip before closing.
• A candidate regression (an erroring connect(prompt=AsyncIterable) stream now closing a ClaudeSDKClient session's stdin) was investigated and ruled out — the normal-completion path of that shape already ended input before this PR.

Extended reasoning...

Overview

The PR (three commits since the last reviewed base) fixes the stdin lifecycle for the can_use_tool permission callback: Query._has_bidirectional_needs() in src/claude_agent_sdk/_internal/query.py now includes can_use_tool alongside hooks and SDK MCP servers, so stdin stays open until the run-ending result; stream_input() gained exception handling for failing prompt iterables with a written count guard mirroring the TypeScript SDK; the "requires streaming mode" ValueError was dropped so string prompts work with the callback; and the duplicated validation/hook-conversion logic in client.py and _internal/client.py was extracted into _configure_can_use_tool() and _hooks_to_internal_format() in types.py. New unit tests use a permission-gated mock transport that enforces the real CLI ordering contract, plus two e2e regression tests.

Security risks

The change touches the tool-permission path, which is security-relevant: can_use_tool is the SDK-side gate on tool execution. However, the change makes the callback more reliably invoked (previously it silently failed with "Stream closed" and the CLI fell back to its default permission behavior), and _configure_can_use_tool() preserves the mutual-exclusion check, the shadowing warning, and the forced permission_prompt_tool_name="stdio" routing verbatim from both deleted copies. No injection, auth-bypass, or data-exposure surface is introduced; the diff is limited to when stdin closes and where shared validation lives.

Level of scrutiny

Medium-high scrutiny is appropriate: async stdin lifecycle interacting with a subprocess control protocol is subtle, and this is the third revision of the PR. The bug-hunting run exited at max_rounds (bounded, not run-dry), which rules out approval on its own. That said, the two follow-up commits verifiably implement the earlier review feedback — the mock transport now has the if not user_message_written.is_set(): return fail-fast guard, stream_input logs the swallowed prompt-iterable error at error level and always reaches an end-input path, and the docstring now documents the pre-existing one-shot _first_result_event limitation for multi-message AsyncIterable prompts rather than overstating "run-ending result".

Other factors

This run investigated one candidate regression and ruled it out: an exception in a long-lived ClaudeSDKClient.connect(prompt=AsyncIterable) stream now triggers the close-stdin path, but the normal-completion path of that same shape already called wait_for_result_and_end_input() before this PR, so an unbounded-iterable session relying on stdin surviving an iterable crash was already outside the supported contract, and closing on error matches the TypeScript SDK. Test coverage is strong: the unit mock enforces the real ordering contract (permission request only after prompt write, frames only after verdict, writes after end_input() raise), and the e2e tests target a path outside cwd so nothing can auto-approve the Write. All prior review threads map to actual code changes, so nothing raised earlier remains outstanding; the deferral reflects only the bounded hunt and the sensitivity of the permission/stdin surface, not any known open issue.