Skip to content

Commit e8e7579

Browse files
committed
[Bug] Fix gpt-oss missing tool content
The changes include adding 'with_recipient' and the Assistant's 'analysis' content. Without adding this content, there was an issue where the gpt-oss model had a higher probability of outputting abnormal tokens when calling tools. Signed-off-by: kyt <eluban4532@gmail.com>
1 parent 784c231 commit e8e7579

File tree

1 file changed

+7
-2
lines changed

1 file changed

+7
-2
lines changed

vllm/entrypoints/harmony_utils.py

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -241,6 +241,11 @@ def parse_chat_input(chat_msg) -> list[Message]:
241241
tool_calls = chat_msg.get("tool_calls")
242242
if role == "assistant" and tool_calls:
243243
msgs: list[Message] = []
244+
content = chat_msg.get("content") or ""
245+
analysis_msg = Message.from_role_and_content(Role.ASSISTANT, content)
246+
analysis_msg = analysis_msg.with_channel("analysis")
247+
msgs.append(analysis_msg)
248+
244249
for call in tool_calls:
245250
func = call.get("function", {})
246251
name = func.get("name", "")
@@ -266,8 +271,8 @@ def parse_chat_input(chat_msg) -> list[Message]:
266271
)
267272

268273
msg = Message.from_author_and_content(
269-
Author.new(Role.TOOL, f"functions.{name}"), content
270-
).with_channel("commentary")
274+
Author.new(Role.TOOL, f"functions.{name}"),
275+
content).with_channel("commentary").with_recipient("assistant")
271276
return [msg]
272277

273278
# Default: user/assistant/system messages with content

0 commit comments

Comments
 (0)