Using Outlines with vLLM/ FastChat

[rounak610]

How can we use outlines with vLLM or FastChat?

[brandonwillard]

We already have an open issue for vLLM, #163, so feel free to follow and/or add to that.

Aside from that, exactly what kind of integrations with FastChat are you proposing? It looks like they already have some HF integrations, so using those models with outlines could be a straightforward user-level integration. If you can construct a minimal example demonstrating what you would like to be able to do, we can work from there.

[rlouf]

It is now possible to use with vLLM. Documentation coming.