Skip to content

Add support for a Python vLLM backend #178

@rgreenberg1

Description

@rgreenberg1

Description:
Right now, GuideLLM only works w/ GuideLLM through the OpenAI compatible API backend to take the target URI and perform benchmarking on a vLLM server. We want to also add support for the python vLLM backend so that we can create scripts for benchmarks w/ vLLM and use it in-line with Python.

Acceptance Criteria:

Metadata

Metadata

Assignees

Labels

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions