[Usage]: how to use tools/tool_choice while using open api endpoints

[shubham-bnxt]

Your current environment

The output of `python collect_env.py`

How would you like to use vllm

I want to run inference of a mistral 8X7B model.I want to utilize function calling / tools while inferencing using open ai compatible API endpoints but I am not able to get details around it. is this supported? if not what is the timeline of having this.

[br3no]

Tool use is not supported yet. In #3237 there is an effort to bring this feature to vLLM.

[hmellor]

Closing as this is not yet supported and is not a feature request ticket.

For the feature request, see #1869.