Fix tool calls being silently dropped on DeepSeek V4 and Cohere2MoE - #593
Draft
fgheorghe wants to merge 1 commit into
Draft
Fix tool calls being silently dropped on DeepSeek V4 and Cohere2MoE#593fgheorghe wants to merge 1 commit into
fgheorghe wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tested for DeepSeek V4 Flash 0713 - Cohere2MoE is reported by Claude.
Fix supplied by GLM 5.2.
The issue
When DeepSeek V4 emitted a tool call, the API response came back with
finish_reason: "tool_calls"and an emptytool_callsarray. The model calledthe tool, the daemon parsed it correctly — the call was then thrown away by the
CLI serve layer on its way out to the client. Cohere2MoE has the same bug.
The serve layer folds the daemon's event stream one of two ways, chosen by
whether
gen_startadvertisescontract_version: 2. The V2 fold reads toolcalls from the
callsarray staged on the terminal payload. The older folddoesn't — it only ever picked them up from a separate
tool_callsevent.That worked as long as those two things lined up, and for most archs they do.
DeepSeek V4 doesn't advertise contract 2 (by design), so it lands on the older
fold — but all four of its generate paths (AR, spec, EP, heterogeneous) deliver
calls exclusively by staging them on the terminal, and never emit the separate
event the old fold was waiting for. Nothing errored; the buffer was just empty
when the response was built. Cohere2MoE delivers calls the same way, so it was
dropping them too.
Only tool turns on those archs are affected. Anything on the V2 fold, and any
generation that doesn't call a tool, behaves exactly as before.
The fix
Teach the older fold to read the staged
callsarray, the same way the V2 foldalready does. It runs at
commit_ready, which is where the terminal is actuallydelivered — the post-commit
donehandler is never reached on that path.A staged array replaces the buffer rather than appending to it, so an arch that
both stages and emits the separate event doesn't end up with each call twice.
This matches what the V2 fold does, for the same reason. A terminal with no
callsfield leaves the buffer alone, so archs that only emit the separateevent keep working untouched.
Malformed entries — a non-array
calls, or a call missing its name — now failthe request instead of being skipped. Quietly discarding them is what caused
this bug in the first place.