Support custom OpenAI-compatible endpoints like VLLM

[bitnom]

Is it currently possible to set a custom openai endpoint? https://vllm.readthedocs.io/en/latest/getting_started/quickstart.html#openai-compatible-server

If not, it would be a good thing.

[bitnom]

Looks like this will have to happen first: vllm-project/vllm#379

[0xc1c4da]

duplicate of #143

[lbeurerkellner]

Yes, please refer to #143 to track vLLM support in the future. As said there, we are definitely willing to add support but are still waiting out their integration of logit biasing.