fix(client): fail closed on SSE UTF-8 split across HTTP/2 DATA — shared decoder + tail flush (#5374)

[Hmbown]

Summary

Closes #5374 — DeepSeek Flash on macOS showed garbled streaming agent text (U+FFFD / broken CJK) because HTTP/2 DATA frames can split a multi-byte UTF-8 character across chunks and the SSE readers decoded per chunk.

What main already had (19c4d1d): take_sse_line / flush_sse_line buffer raw bytes and decode only complete \n-terminated lines with strict str::from_utf8; a genuinely invalid line returns InvalidSseUtf8 instead of substituting U+FFFD.

What this PR adds on top of that baseline:

• decode_sse_line_bytes — the one shared strict decoder used by both take_sse_line and flush_sse_line; InvalidSseUtf8 slimmed to valid_up_to with a single Display form.
• next_sse_line(buffer, at_end) — complete-line-or-tail helper so every dialect flushes the unterminated stream-end tail through the same fail-closed path.
• Per-dialect propagation of the decode failure:
  • Chat Completions (chat.rs): new decode_failed flag; a UTF-8 error on a complete line or on the tail flush yields Err(InvalidSseUtf8) and skips the post-loop frame parse so nothing after the error reaches the transcript.
  • Anthropic (anthropic.rs) and Responses (responses.rs): read loop switched to an ended state so the stream-end tail is flushed via next_sse_line; decode errors yield Err and return.
• SseLineDecoder (#[cfg(test)]) — incremental raw-byte line assembler used by the new decoder tests.

Tests

• crates/tui/src/client.rs: take_sse_line_reassembles_cjk_and_rejects_invalid_bytes, flush_sse_line_reassembles_cjk_and_rejects_invalid_bytes, decode_sse_line_bytes_rejects_invalid_without_replacement; existing take/flush tests updated for the slimmed error type (byte-split positions computed via mid_char_split).
• crates/tui/src/client/chat/tests/stream_decoder.rs: decoder_reassembles_cjk_split_across_byte_chunks, decoder_reassembles_emoji_and_cjk_fed_one_byte_at_a_time, decoder_rejects_invalid_sse_bytes_without_replacement — drive the Chat Completions frame parser over raw byte chunks split mid-character.

Relation to #5404

Supersedes draft #5404 (cursor branch, based on 5ac75add4, before main's fail-closed baseline landed). This branch is rebased onto current main containing 19c4d1d, so it only carries the delta (shared decoder, tail flush, per-dialect propagation, tests). #5404 is left open for the owner to close.

Testing

• cargo fmt --all -- --check
• cargo clippy --workspace --all-targets --all-features --locked (CI)
• cargo test --workspace --all-features --locked (CI)

Checklist

• Updated docs or comments as needed
• Added or updated tests where relevant
• Verified TUI behavior manually if UI changes
• Harvested/co-authored credit uses a GitHub numeric noreply address (no co-author trailers)

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[claude]

Claude finished @Hmbown's task in 4m 11s —— View job

---

Reviewing PR #5468

• Gather context (PR body, diff, repo instructions)
• Read changed files (client.rs, anthropic.rs, chat.rs, chat/tests/stream_decoder.rs, responses.rs)
• Check correctness of shared decoder + tail flush logic
• Check provider/model/route safety implications (not applicable — no provider/model selection touched)
• Check for reuse/duplication issues
• Check test coverage
• Post final review

---

Summary

This is a solid, well-scoped fix. It adds a single shared strict decoder (decode_sse_line_bytes) used by both take_sse_line and flush_sse_line, and a next_sse_line(buffer, at_end) helper that lets Anthropic and Responses flush the unterminated stream-end tail through the same fail-closed path chat.rs already had. I traced the ended/done control flow in anthropic.rs and responses.rs and the decode_failed/saw_done gating in chat.rs; both are correct — no infinite loops, no double-processing, buffers are fully drained on error.

Notable genuine bug fix buried in the chat.rs refactor: previously, if flush_sse_line failed after the main loop, the code still fell through to if !line_buf.is_empty() { … yield events … } using stale pre-error line_buf content, i.e. it could yield transcript events after already yielding the fail-closed Err. The new decode_failed flag (crates/tui/src/client/chat.rs:1432) closes that gap by skipping the post-loop parse entirely once a decode error occurred, matching the PR description. Good catch.

Findings

1. Test coverage gap — Anthropic/Responses tail-flush path has no direct test (crates/tui/src/client/anthropic.rs:289, crates/tui/src/client/responses.rs stream loop)
The new ended flag + next_sse_line tail-flush behavior is the actual production code path for two of the three dialects (Anthropic, Responses), but only Chat Completions (chat.rs) got new end-to-end byte-chunk tests (stream_decoder.rs). next_sse_line itself (crates/tui/src/client.rs:3737) has zero direct unit tests — it's exercised only indirectly through take_sse_line/flush_sse_line tests, never in its own at_end=true combinator form. There's also no existing wiremock-based test that drives handle_anthropic_stream (or the Responses equivalent) over a real streamed body with a multi-byte character split at the very end of the stream — the existing #[tokio::test]s there only cover open/retry behavior, not body decoding. Given this PR's whole purpose is fixing exactly this class of bug for DeepSeek Flash (Anthropic-dialect), a direct test of the ended tail-flush for anthropic.rs/responses.rs (even just a next_sse_line unit test with at_end: true covering both a clean flush and an invalid-tail failure) would meaningfully raise confidence over relying solely on the shared low-level helper tests. Fix this →

Notes (not blocking)

• crates/tui/src/client/chat.rs's inner line loop still calls take_sse_line/flush_sse_line directly rather than next_sse_line, unlike anthropic.rs/responses.rs. This is justified — chat.rs needs the saw_done gate that doesn't fit next_sse_line's simple combinator — so not a real duplication issue, just noting the asymmetry is intentional.
• Pre-existing (unrelated to this PR): in chat.rs, after any break 'stream (including on decode error), execution still falls through to ContentBlockStop/MessageStop yields after the Err was already yielded. This predates this PR (the old code had the same break 'stream-then-fall-through structure) and isn't something this diff needs to fix, but is worth a follow-up if anthropic.rs/responses.rs's return-on-error pattern is meant to be the standard.
• No provider/model prefix inference issues — this PR only touches byte-level SSE decoding, not routing/model selection.
• Field slimming (InvalidSseUtf8 losing error_len) and the Display message simplification are clean; verified no other references to the removed error_len field remain in the crate.

I was unable to run cargo fmt --check, cargo test, or cargo clippy in this environment (git fetch / cargo invocations required interactive approval I don't have here); the PR's own checklist already defers clippy/test verification to CI, which is appropriate.
· branch codex/v099-issue-5374-sse-utf8