Skip to content

Reject model streams that end before completing their turn - #785

Merged
WaylandYang merged 2 commits into
devfrom
land/truncated-model-stream
Sep 19, 2026
Merged

WaylandYang merged 2 commits into
devfrom
land/truncated-model-stream

Conversation

@WaylandYang

Copy link
Copy Markdown
Contributor

Maya-Kid's #768, rebased onto dev after #764 turned the streaming buffer into bytes. The conflict was mechanical: keep the Vec<u8> buffer, add their ended and got.

What it does. chat_tools_stream_with and chat_stream_raw get the completion check that #757 gave only to chat_at_streaming: a frame carrying finish_reason, or data: [DONE], marks the turn finished, and reaching EOF without either yields Interrupted { got } instead of a Turn or a silent end. Interrupted is already retryable in transient().

Reverting their source and keeping their tests shows what the check is for:

---- tests::a_tool_stream_cut_before_its_finish_is_an_error stdout ----
called `Result::unwrap_err()` on an `Ok` value: [Turn(AssistantTurn { content: None,
  tool_calls: [ToolCall { id: "call_1", name: "lookup", arguments: "{\"name\":" }] })]

Without it the agent receives a tool call whose arguments are the fragment {"name": and has to work out for itself that the JSON never finished.

One consequence worth stating, since the description did not. api/chat.rs streams chat_stream_raw and on an error yields an error event and returns before conversations::append_message. So a cut stream now shows the reader the partial text followed by an error, and the assistant turn is not saved at all; before, a short turn was saved with a clean done. Losing half an answer is the right call — half a thing is not a thing anywhere else in this ledger — but it is a decision, not an inheritance.

Their branch could not be rebased from here, so this carries their commit with authorship intact and #768 closes with it.

30 utopia-llm tests pass, clippy across utopia-llm and utopia-server is clean, fmt clean.

🤖 Generated with Claude Code

Signed-off-by: dada-yan <BinjunYann@gmail.com>
Signed-off-by: dada-yan <BinjunYann@gmail.com>
@WaylandYang
WaylandYang force-pushed the land/truncated-model-stream branch from 02a73fc to f01b6c8 Compare September 19, 2026 07:55
@WaylandYang
WaylandYang merged commit 53603da into dev Sep 19, 2026
@WaylandYang
WaylandYang deleted the land/truncated-model-stream branch September 19, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants