Endpoints code requires an OpenAI API compatible server (SGLANG incompatability)

[merlinf31]

[merlinf31]

here is the output when trying to run endpoints against a server running SGLANG.

[merlinf31]

SGLANG does not (yet) support the full OPENAI API hence the incompatability.

[arekay-nv]

@merlinf31 can you add reproducing steps - please include server command with sglang version, and the client config that you used. We can document the specific requirements that is missing.

[attafosu]

@merlinf31 #158 should cover that. See src/inference_endpoint/openai/types.py::refusal

[arekay-nv]

Closing for now - please reopen if issue still persists.