Feature request: Support for embedding models

[mantrakp04]

Support for embedding models such as instructor or sentence-transformers

[iguy0]

+1

[jianhuaz]

Supporting Embedding is a prerequisite for LLM to build a knowledge base. I think VLLM needs to be implemented as soon as possible. Otherwise, it will fall behind others in the new round of AI Agent technology.

[iguy0]

#310

[michaelfeil]

After buidling https://github.com/michaelfeil/infinity, I‘d be happy to support, if there is a better breakdown on what to work on.

[DavidPeleg6]

+1

[michaelfeil]

As the Roadmap is still missing Embedding models, I'd think it would be time to start supporting an initial backend with a stable (OpenAI-compatible API), similar to what e.g. Ollama did recently. I don't have the time to building everything from scratch and would prefer to a shallow integration of infinity, without any additional dependencies (other than torch that ships with vllm and python 3.9-3.12).

@WoosukKwon @Yard1 WDYT?

[DarkLight1337]

Closed as completed by #3734.

[yichuan520030910320]

Can we support model like facebook/contriever this one?

[DarkLight1337]

Just try it. If it's not supported, please open a new issue.