[Bug]: api_server.py: error: unrecognized arguments: --task embedding

[javiplav]

Your current environment

I'm running in a Kubernetes environment vllm docker image vllm/vllm-openai:latest .

The latest version is supposed to support --task embedding flag. https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html

Model Input Dumps

No response

🐛 Describe the bug

I start the server with the following command: python3 -m vllm.entrypoints.openai.api_server and arguments: --served-model-name BAAI/bge-m3 --enforce-eager --task embedding .

The pod fails to run. Error message: api_server.py: error: unrecognized arguments: --task embedding

Any suggestions please?

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

I think --task embedding (#9424) has not been released outside of main branch yet. You'll have to clone the repository and build it from source to use it. Make sure that you have updated your Docker image to use the latest version (as in the latest, latest build).

[javiplav]

@DarkLight1337 thanks for your reply. I'm using https://hub.docker.com/layers/vllm/vllm-openai/latest/images/sha256-facbbd4a92c1754675b239a5f22a281ed3aa8bde64662db8919d85d670673aa7?context=explore .

[DarkLight1337]

It looks like the "latest" Docker image there refers to the most recently released version, so it's not the one from main branch.

[DarkLight1337]

Please follow these instructions for the latest code.

[javiplav]

Thanks @DarkLight1337 . Managed to run the server with the main branch.

Unfortunately I didn't succeed in running bge-m3 as it seems that vllm doesn't support xlm-roberta architectures.

[DarkLight1337]

Oh, I completely missed that you were trying to run bge-m3. Can you move over to #9847 ?