[Installation]: How to deploy docling model on vllm

[nagarajsv3]

Your current environment

I have downloaded the docling model files. I have it in the local machine path. I want to serve the docling model through vllm. Not finding any documentation around serving custom models on vllm. Would appreciate any input or solution or documentation on this.

How you are installing vllm

pip install -vvv vllm

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[nagarajsv3]

I need to serve various custom models . How to do that

[jeejeelee]

Maybe you can refer to: https://docs.vllm.ai/en/latest/design/plugin_system.html