Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support custom models #112

Closed
WoosukKwon opened this issue May 20, 2023 · 2 comments
Closed

Support custom models #112

WoosukKwon opened this issue May 20, 2023 · 2 comments
Labels
feature request

Comments

@WoosukKwon
Copy link
Collaborator

We need to provide clean abstractions and interfaces so that users can easily plug in their custom models.
@zhuohan123 zhuohan123 mentioned this issue Jun 27, 2023
Closed
76 tasks
@hmellor
Copy link
Collaborator

hmellor commented Mar 6, 2024

Can this be considered done now that there is a documentation page detailing how to do it https://docs.vllm.ai/en/latest/models/adding_model.html?

@DarkLight1337 DarkLight1337 added the enhancement New feature or request label May 31, 2024
@hmellor hmellor added feature request and removed enhancement New feature or request labels May 31, 2024
jikunshang pushed a commit to jikunshang/vllm that referenced this issue Aug 15, 2024
@kzawora-intel @michalkuligowski
Add option to skip tokenizer (vllm-project#112)
73b97de 
* Disable tokenizer

* Update protocol.py

* Update serving_completion.py

* Detect value of skip_tokenizer_init cmd arg

* support skipping tokenizer for streaming scenario

* remove debug print

---------

Co-authored-by: Michał Kuligowski <michal.kuligowski@intel.com>
@DarkLight1337
Copy link
Member

DarkLight1337 commented Oct 26, 2024 •
edited
Loading

We now support automatic detection of generation/embedding task (vllm/models/interfaces_base.py) as well as additional features like multimodality and PP (vllm/models/interfaces.py) for both built-in and OOT models. It's safe to say that this issue has been resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hmellor @DarkLight1337 @WoosukKwon