Skip to content

Commit dcaa192

Browse files
DarkLight1337amitm02
authored andcommitted
[Doc] Update OOT model docs (vllm-project#18742)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: amit <amit.man@gmail.com>
1 parent f919fa9 commit dcaa192

File tree

2 files changed

+19
-18
lines changed

2 files changed

+19
-18
lines changed

docs/contributing/model/registration.md

Lines changed: 15 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -23,33 +23,32 @@ Finally, update our [list of supported models][supported-models] to promote your
2323

2424
## Out-of-tree models
2525

26-
You can load an external model using a plugin without modifying the vLLM codebase.
27-
28-
!!! info
29-
[vLLM's Plugin System][plugin-system]
26+
You can load an external model [using a plugin][plugin-system] without modifying the vLLM codebase.
3027

3128
To register the model, use the following code:
3229

3330
```python
34-
from vllm import ModelRegistry
35-
from your_code import YourModelForCausalLM
36-
ModelRegistry.register_model("YourModelForCausalLM", YourModelForCausalLM)
31+
# The entrypoint of your plugin
32+
def register():
33+
from vllm import ModelRegistry
34+
from your_code import YourModelForCausalLM
35+
36+
ModelRegistry.register_model("YourModelForCausalLM", YourModelForCausalLM)
3737
```
3838

3939
If your model imports modules that initialize CUDA, consider lazy-importing it to avoid errors like `RuntimeError: Cannot re-initialize CUDA in forked subprocess`:
4040

4141
```python
42-
from vllm import ModelRegistry
43-
44-
ModelRegistry.register_model(
45-
"YourModelForCausalLM",
46-
"your_code:YourModelForCausalLM"
47-
)
42+
# The entrypoint of your plugin
43+
def register():
44+
from vllm import ModelRegistry
45+
46+
ModelRegistry.register_model(
47+
"YourModelForCausalLM",
48+
"your_code:YourModelForCausalLM"
49+
)
4850
```
4951

5052
!!! warning
5153
If your model is a multimodal model, ensure the model class implements the [SupportsMultiModal][vllm.model_executor.models.interfaces.SupportsMultiModal] interface.
5254
Read more about that [here][supports-multimodal].
53-
54-
!!! note
55-
Although you can directly put these code snippets in your script using `vllm.LLM`, the recommended way is to place these snippets in a vLLM plugin. This ensures compatibility with various vLLM features like distributed inference and the API server.

docs/design/plugin_system.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,10 @@ def register():
3030
from vllm import ModelRegistry
3131

3232
if "MyLlava" not in ModelRegistry.get_supported_archs():
33-
ModelRegistry.register_model("MyLlava",
34-
"vllm_add_dummy_model.my_llava:MyLlava")
33+
ModelRegistry.register_model(
34+
"MyLlava",
35+
"vllm_add_dummy_model.my_llava:MyLlava",
36+
)
3537
```
3638

3739
For more information on adding entry points to your package, please check the [official documentation](https://setuptools.pypa.io/en/latest/userguide/entry_point.html).

0 commit comments

Comments
 (0)