[New Model] Add Seed-Oss model #23241

FoolPlayer · 2025-08-20T08:04:24Z

Purpose

Add Seed-Oss model.
transformers see: huggingface/transformers#40272

Test Plan

Test Result

(Optional) Documentation Update

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Signed-off-by: jiabin.00 <jiabin.00@bytedance.com>

gemini-code-assist

Code Review

This pull request adds support for the Seed-Oss model, including the model implementation, a tool parser, and necessary registry updates. The implementation looks mostly solid, but I've identified a few critical issues that should be addressed before merging. These include leftover debugging print statements, a security vulnerability from using eval() on model output, and silent error handling that could obscure bugs. I've provided specific suggestions for each of these points.

gemini-code-assist · 2025-08-20T08:05:43Z

vllm/entrypoints/openai/tool_parsers/seed_oss_tool_parser.py

+                    param_value = eval(param_value)
+                except:
+                    logger.warning(
+                        f"Parsed value '{param_value}' of parameter '{param_name}' cannot be converted via Python `eval()` in tool '{func_name}', degenerating to string."
+                    )


Using eval() on model output is a significant security vulnerability. Model output is untrusted input and can be manipulated to execute arbitrary code on the server. Please replace eval() with ast.literal_eval and add import ast at the top of the file.

Suggested change

param_value = eval(param_value)

except:

logger.warning(

f"Parsed value '{param_value}' of parameter '{param_name}' cannot be converted via Python `eval()` in tool '{func_name}', degenerating to string."

)

param_value = ast.literal_eval(param_value)

except (ValueError, SyntaxError):

logger.warning(

f"Parsed value '{param_value}' of parameter '{param_name}' cannot be converted via Python `ast.literal_eval()` in tool '{func_name}', degenerating to string."

)

gemini-code-assist · 2025-08-20T08:05:44Z

vllm/model_executor/model_loader/utils.py

+    print(architectures)
+    print(model_config)


These print statements appear to be leftover debugging code. They should be removed before merging to avoid cluttering logs and potential performance degradation in production environments.

Makes sense

gemini-code-assist · 2025-08-20T08:05:44Z

vllm/entrypoints/openai/tool_parsers/seed_oss_tool_parser.py

+                    except Exception:
+                        pass  # Ignore parsing errors during streaming


Silently ignoring all exceptions with except Exception: pass is dangerous. It can hide bugs and make debugging very difficult. If an error occurs during argument parsing, it should at least be logged as a warning. This will help identify issues with model output or the parser logic itself.

except Exception: logger.warning( "Failed to parse tool arguments during streaming.", exc_info=True)

github-actions · 2025-08-20T08:06:22Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

youkaichao · 2025-08-20T08:15:23Z

cc @DarkLight1337 who can help take a look?

jeejeelee · 2025-08-20T08:16:48Z

vllm/model_executor/models/seed_oss.py

+        return loaded_params
+
+
+class SeedOssForCausalLM(nn.Module, SupportsLoRA, SupportsPP):


It looks like this model structure is consistent with llama. Can we directly inherit from llama, just like phi3 does?

We also need to update the supported_models and test

jeejeelee · 2025-08-20T08:18:07Z

vllm/entrypoints/openai/tool_parsers/seed_oss_tool_parser.py

+            rf"{tool_start_re}(.*?){tool_end_re}|{tool_start_re}(.*?)$",
+            re.DOTALL)
+
+        # 其余 regex 与原版一致


We should add comment in english

jeejeelee · 2025-08-20T08:28:21Z

vllm/entrypoints/openai/tool_parsers/seed_oss_tool_parser.py

@@ -0,0 +1,632 @@
+# SPDX-License-Identifier: Apache-2.0


We need to add test for this tool parser, see: https://github.com/vllm-project/vllm/tree/main/tests/tool_use

ZJY0516 · 2025-08-21T02:04:43Z

Now that Seed-OSS is released, I was wondering if there's any progress on this PR.

FoolPlayer · 2025-08-21T02:39:37Z

Now that Seed-OSS is released, I was wondering if there's any progress on this PR.

thanks for review, i'll fix the comments these days

Signed-off-by: jiabin.00 <jiabin.00@bytedance.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

jeejeelee

Already confirmed with @FoolPlayer, both the generation and tool calls are working fine, so I'm adding my stamp.
@FoolPlayer Thank you for contribuion

prnake · 2025-08-21T16:06:34Z

In addition to the documents and test files, the current PR is lacking some necessary elements, such as ReasoningParser.

The current PR does not handle the xml tags such as seed:think.

{"id":"chatcmpl-7f45361eec5e483c96a119683efdadd7","object":"chat.completion","created":1755792348,"model":"Seed-OSS-36B-Instruct","choices":[{"index":0,"message":{"role":"assistant","content":"<seed:think>\nGot it, let's see. The user wrote \"Ciallo\". First, I need to figure out what that means. \"Ciallo\" is a term I've heard before, maybe from Japanese or anime? Wait, let me think. Oh right, \"ciao\" is Italian for \"hello\" or \"goodbye\", but \"ciallo\"—maybe it's a misspelling or a different language. Wait, no, actually, in some anime or Japanese contexts, \"ciallo\" is sometimes used as a casual way to say \"hello\" or \"hi\", maybe influenced by \"ciao\"? Or is it a typo? Wait, let me check. Alternatively, maybe it's a name? Or a slang term. Wait, another angle: \"ciallo\" could be a transliteration of a Japanese sound or a term. Let me think... Oh, right! There's a character named Ciallo? Or maybe it's a fanservice term. Wait, no, actually, I remember that \"ciallo\" is sometimes used in online communities, especially among anime fans, as a friendly greeting, similar to \"hi\" or \"hello\". It might have originated from a mishearing or a playful twist on \"ciao\". Alternatively, could it be a typo for \"ciao\"? But the user wrote \"ciallo\" with two Ls. Let me confirm. Looking it up quickly in my mind: yes, \"ciallo\" is often used in casual, friendly contexts, especially in anime or otaku communities, as a way to say hello. So the user probably just said \"hello\" in a casual, maybe anime-influenced way. So the response should be friendly, acknowledging that. Maybe say \"Hi there! Ciallo is a cute greeting, right? How are you doing today?\" or something like that. Keep it warm and welcoming.</seed:think>Ciallo! It’s such a cheerful, friendly little greeting—often seen in casual anime or otaku circles, like a playful \"hi\" or \"hello.\"  \n\nHow are you feeling today? 😊","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":10,"total_tokens":441,"completion_tokens":431,"prompt_tokens_details":null},"prompt_logprobs":null,"kv_transfer_params":null}

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: jiabin.00 <jiabin.00@bytedance.com> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> Signed-off-by: root <xwq391974@alibaba-inc.com>

Signed-off-by: jiabin.00 <jiabin.00@bytedance.com> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: jiabin.00 <jiabin.00@bytedance.com> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> Signed-off-by: Xiao Yu <xiao.yu@amd.com>

Signed-off-by: jiabin.00 <jiabin.00@bytedance.com> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

FoolPlayer added 2 commits August 19, 2025 21:43

support seed-oss model in vllm

2c353b4

Signed-off-by: jiabin.00 <jiabin.00@bytedance.com>

add seed-oss tool parser

1662dda

FoolPlayer requested a review from aarnphm as a code owner August 20, 2025 08:04

mergify bot added frontend new-model Requests to new models tool-calling labels Aug 20, 2025

github-project-automation bot added this to Tool Calling Aug 20, 2025

gemini-code-assist bot reviewed Aug 20, 2025

View reviewed changes

jeejeelee reviewed Aug 20, 2025

View reviewed changes

FoolPlayer added 2 commits August 21, 2025 17:59

fix seed-oss parser and add test

8dbb665

Signed-off-by: jiabin.00 <jiabin.00@bytedance.com>

updata readme

5b16d11

Signed-off-by: jiabin.00 <jiabin.00@bytedance.com>

FoolPlayer requested a review from hmellor as a code owner August 21, 2025 10:18

mergify bot added the documentation Improvements or additions to documentation label Aug 21, 2025

jeejeelee added 2 commits August 21, 2025 22:31

Merge branch 'main' into seed-oss

f7de5f7

Fix lint

ac77333

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

jeejeelee approved these changes Aug 21, 2025

View reviewed changes

jeejeelee added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 21, 2025

jeejeelee enabled auto-merge (squash) August 21, 2025 16:05

jeejeelee added 2 commits August 22, 2025 00:09

Merge branch 'main' into seed-oss

aea5318

Fix test

1eaf817

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

jeejeelee requested review from DarkLight1337 and ywang96 as code owners August 22, 2025 01:36

Fix config

76254a3

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

jeejeelee force-pushed the seed-oss branch from 843baa1 to 76254a3 Compare August 22, 2025 02:42

jeejeelee merged commit 5964069 into vllm-project:main Aug 22, 2025
42 checks passed

github-project-automation bot moved this to Done in Tool Calling Aug 22, 2025

SkyAndCloud mentioned this pull request Aug 23, 2025

Update README.md ByteDance-Seed/seed-oss#8

Closed

AlpinDale mentioned this pull request Aug 23, 2025

feat: add support for Seed-Oss model aphrodite-engine/aphrodite-engine#1430

Merged

bfroemel mentioned this pull request Aug 24, 2025

[Model] Added reasoning parser for Seed-Oss model #23486

Closed

4 tasks

Jun-Howie mentioned this pull request Aug 28, 2025

FEAT: Seed-OSS-36B xorbitsai/inference#4020

Merged

		except Exception:
		pass # Ignore parsing errors during streaming

		return loaded_params


		class SeedOssForCausalLM(nn.Module, SupportsLoRA, SupportsPP):

Uh oh!

[New Model] Add Seed-Oss model #23241

[New Model] Add Seed-Oss model #23241

Uh oh!

Conversation

FoolPlayer commented Aug 20, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

(Optional) Documentation Update

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

jeejeelee Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Aug 20, 2025

Uh oh!

youkaichao commented Aug 20, 2025

Uh oh!

jeejeelee Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

jeejeelee Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

jeejeelee Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

jeejeelee Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

ZJY0516 commented Aug 21, 2025

Uh oh!

FoolPlayer commented Aug 21, 2025

Uh oh!

jeejeelee left a comment

Choose a reason for hiding this comment

Uh oh!

prnake commented Aug 21, 2025

Uh oh!

Uh oh!

Uh oh!

FoolPlayer commented Aug 20, 2025 •

edited by github-actions bot

Loading