fix: hide Gemma 4 reasoning channel from CLI output

[inureyes]

Summary

Gemma 4 checkpoints open a <|channel>thought ... <channel|> reasoning channel inline with the answer. The CLI detokenizer decodes with skip_special_tokens = false, so the raw channel markers and the entire chain-of-thought printed straight to the terminal as if they were the answer (issue #884). The server chat-completions path already routed this into reasoning_content; only the CLI output path was unhandled. This PR hides the reasoning channel from the visible CLI output by default and adds an opt-in flag to surface it.

What changed

• New src/reasoning_stream.rs (mlxcel::reasoning_stream): a small streaming reasoning-channel splitter driven by the tokenizer's resolved ThinkingMarkers (<|channel>thought / <channel|> for Gemma 4, <think> / </think> for Qwen-style models). It is the CLI counterpart to the server's StreamFilter but deliberately narrower: it splits only the reasoning channel and leaves every other byte untouched, so an answer that opens no channel passes through byte-identically (no tool-call suppression, no turn-marker stripping).
• Because the open marker is the full <|channel>thought, the thought channel argument is consumed as part of the marker and never surfaces in the reasoning text (issue-required thought trim).
• Marker straddling two decoded fragments is held back until the next fragment resolves it; the end-of-stream flush keeps a never-closed thought block hidden without losing or duplicating any text.
• src/commands/chat.rs: the interactive REPL streaming callback now feeds each decoded fragment through the splitter; the detokenizer tail is flushed through the filter too, replacing the old reply.strip_prefix tail mechanism.
• src/commands/generate.rs: the one-shot generate flow feeds the whole decoded reply through the splitter before printing, at every autoregressive print site.
• src/main.rs: adds the --show-reasoning flag (default off) on the shared generation options so it reaches both mlxcel run and mlxcel generate; ChatOptions carries it into the REPL.

Behavior

• Default (flag off): a Gemma 4 checkpoint that opens a thought channel prints only the final answer; no <|channel> / <channel|> / thought text appears. A direct answer that opens no channel is byte-identical to before (the filter is an inert passthrough for non-thinking models).
• --show-reasoning: the reasoning is also printed (dimmed on a TTY). The raw channel markers never print either way.

New CLI flag

--show-reasoning (default off), on both mlxcel run and mlxcel generate.

Test plan

• cargo test --lib reasoning_stream (14 unit tests): thought-then-answer (only the answer visible, reasoning captured), answer-only passthrough, an unclosed thought (reasoning hidden, no marker leak), token-by-token feeding with markers split across fragments, and the --show-reasoning surface without raw markers.
• cargo check --lib and cargo check --bin mlxcel: clean.
• cargo clippy --lib --tests: clean.
• End-to-end model-run verification on the GB10 CUDA host (gemma-4-e4b-it-qat-4bit) left to the orchestrator, per the build constraints on this host.

Scope note

This PR addresses the CLI output leak, which is the primary bug. The minor server-side reasoning_content thought prefix noted in the issue is left untouched to avoid changing the server's StreamFilter reasoning contract and its existing tests.

Closes #884

[inureyes]

Implementation Review Summary

Intent

Hide a reasoning-capable checkpoint's chain-of-thought channel from visible CLI output (raw <|channel>thought / <channel|> / <think> / </think> markers must never print), with an opt-in --show-reasoning flag, wired into both mlxcel run and mlxcel generate.

The core Gemma-4 fix (issue #884's primary target) is correct, complete, and well-tested. Review turned up one real leak in the PR's own claimed Qwen-style support, which I auto-fixed on the branch.

Findings Addressed (auto-fixed, commit ea22e60)

• Prompt-primed open thinking leaked reasoning + raw close marker (HIGH). The CLI enables thinking for Qwen-style models by rendering a prompt that ends with an open marker (<think>\n, or <|channel>thought\n for a thinking-on Gemma-4 channel), see load_cli_prompt and the server's OPEN_THINKING_SUFFIXES. That open marker lives in the prompt, not the generated tokens, so the generated text begins already inside the reasoning block and the first marker the model emits is the close. ReasoningFilter always started in Content, never matched the absent open marker, and printed the whole chain-of-thought plus the raw </think> / <channel|> to the terminal, exactly the leak this PR fixes for a model-opened channel, but for the common Qwen3 CLI path. Fix mirrors the server's is_prompt_primed_open_thinking / StreamFilter::new_primed_open_thinking: added ReasoningFilter::new_primed_open_thinking + prompt_primed_open_thinking, wired into stream_turn (REPL) and filter_reasoning_for_display (one-shot, both print sites). Gemma-4 thinking-OFF prompts end with the closed scaffold and are not detected as primed, so the primary bug: Gemma 4 reasoning channel leaks into CLI output as raw tokens #884 path is unchanged. Added 7 unit tests.

Remaining Items (left as-is)

• --show-reasoning can reorder content that precedes a reasoning block (LOW). In a single drain, ReasoningSplit lumps all outside text into content and render_visible emits reasoning before it, so an input like prefix<think>reason</think>answer renders as reasonprefixanswer when reasoning is shown. Only affects the opt-in --show-reasoning display, and only when content precedes the first open marker in one drain, which does not occur for the target models (Gemma 4 opens the channel first; a primed-open Qwen generation starts in the reasoning state). Default-off suppression is byte-correct. A fix needs an ordered-segment representation instead of two lumped strings; deferred to keep the change focused and low-risk.
• Server reasoning_content still carries the leading thought prefix (issue AC, by design). The issue's server-side acceptance criterion is intentionally out of scope per the PR's scope note (the server StreamFilter uses <|channel> as its open marker and its reasoning-content contract/tests are deliberately untouched). Consider keeping bug: Gemma 4 reasoning channel leaks into CLI output as raw tokens #884 open for the server part or filing a follow-up, since the PR body says "Closes bug: Gemma 4 reasoning channel leaks into CLI output as raw tokens #884".
• Block-diffusion (DiffusionGemma) and LLaDA-2 MoE CLI print paths bypass the filter (informational). These separate drivers print tokens directly and do not route through the reasoning splitter. Not affected by bug: Gemma 4 reasoning channel leaks into CLI output as raw tokens #884's autoregressive Gemma-4 target; noted for completeness only.

Verification

• All stated requirements implemented (default-off suppression, --show-reasoning, no raw markers, byte-identical passthrough)
• No placeholder/mock code
• Integrated into project code flow: mlxcel run (REPL stream_turn) and mlxcel generate -p (run_generate_once, all print sites); --show-reasoning reaches both via GenerationOptions flatten + chat_options_from_args
• Project conventions followed (thiserror/anyhow n/a; module docs, tokenizer marker reuse)
• Existing modules reused where practical; the narrower CLI filter is a deliberate, documented divergence from the server StreamFilter (which also does tool-call suppression, turn-marker stripping, and primed-open handling and keeps the thought prefix)
• No unintended structural changes; server path untouched, its StreamFilter reasoning-content contract intact
• Tests pass: cargo test --lib reasoning_stream (21 passed) and cargo clippy --lib --tests -- -D warnings clean

Not run per this GB10 host's build constraints: release/CUDA build, bare cargo test, end-to-end model-run verification (left to the orchestrator).

[inureyes]

Security & performance review (auto-fix)

Scope: src/reasoning_stream.rs and the CLI output wiring (src/commands/chat.rs, src/commands/generate.rs, src/main.rs). CLI-only local text transform, no network/auth/injection surface, so the review focused on the three memory/correctness concerns for a streaming splitter.

Result: no CRITICAL / HIGH / MEDIUM findings. No changes required.

1. Unbounded memory / buffer growth: safe

ReasoningFilter::buffer is bounded by marker length in every state. drain() emits everything except a trailing run that is a proper prefix of the marker it is currently scanning for, so after each feed the buffer holds at most marker.len() - 1 bytes. This holds in Thinking state (hidden-reasoning default) exactly as in Content. Hidden reasoning is emitted per fragment as the reasoning field and then dropped by the caller (render_visible with show_reasoning=false), never accumulated. A never-closed thought stays bounded: the buffer holds only a partial marker and flush() releases it once into the (discarded) reasoning side. Generation length itself remains bounded by max_tokens, unchanged.

2. UTF-8 boundary safety: safe

No slice can land mid-codepoint. Fragments reach the filter only from StreamingDecodeState::on_token / flush, which return Option<String> (always complete UTF-8; the detokenizer holds back incomplete trailing sequences). Inside the filter, find(marker) and drain(..pos + marker.len()) land on match boundaries, and safe_emit_len returns an index from char_indices() (always a char boundary). Since the markers are ASCII, a multi-byte character in the tail can never be a prefix of a marker, so safe_emit_len never holds one back and never splits it.

3. No output loss / duplication: safe

The drain + flush invariant accounts for every fed byte (emitted as content, emitted as reasoning, consumed as a marker, or held then released at flush). The rewritten chat.rs tail path flushes StreamingDecodeState (offset-tracked, no overlap with already-emitted deltas) and then the filter, in that order. For a non-thinking model the filter is an inert passthrough, so output is byte-identical to the pre-fix path. Verified by cargo test --lib reasoning_stream (21 passing, including token-by-token marker splits and angle-bracket passthrough) plus cargo clippy --lib --tests -- -D warnings (clean).

LOW (informational, not changed)

• Under --show-reasoning (opt-in, off by default), if a single fed fragment contains content that precedes the open marker, render_visible prints that fragment's reasoning before its leading content, since it emits reasoning then content per split. The close-marker case (reasoning then answer) is correctly ordered, and the default hidden path is unaffected (no loss or duplication, display order only). A fix would require ordered-segment emission; not worth the regression risk for a cosmetic non-default path.
• safe_emit_len walks char_indices() from index 0 with a skip to the tail region. Linear overall and negligible at CLI output sizes.

[inureyes]

PR Finalization Complete

Summary

• Tests: src/reasoning_stream.rs went from 21 to 23 unit tests. Added multiple_reasoning_blocks_in_one_stream_all_hidden (the state machine must re-enter Thinking on a second open marker within one stream, not just once per filter lifetime) and flush_is_idempotent_after_holding_back_a_partial_marker (a held-back partial close marker is released exactly once by flush, and a second flush call is a no-op). All 23 pass (cargo test --lib reasoning_stream).
• Documentation: --show-reasoning had no documentation outside --help. Added a short paragraph to README.md's "Run a model" section (the default model is a thinking-capable Gemma 4 checkpoint, so this is directly relevant to first-run output) and a new "CLI reasoning display (--show-reasoning)" subsection in docs/supported-models.md, alongside the existing server-side reasoning_content / enable_thinking documentation, with a note distinguishing the two (one controls whether the model generates reasoning, the other whether the CLI displays it). No Korean translation exists yet for either doc (docs/ko/ is not populated in this repo), so no translation update was needed.
• Lint/Format: cargo fmt on the touched file, cargo clippy --lib --tests -- -D warnings clean, cargo check --lib --tests --bin mlxcel clean.

All checks passing. Ready for merge.