Skip to content

Improve Cuda compatibility of vllm-openai image #2393

Closed as not planned
Closed as not planned
@RonanKMcGovern

Description

@RonanKMcGovern

Currently the 'https://hub.docker.com/r/vllm/vllm-openai/' image uses Cuda 12.1 - this runs into a lot of cuda versioning issues depending on the drivers used on the underlying GPU.

This makes the image an inconsistent starting point for running on services like vast ai or runpod.

Could the docker image be updated to more dynamically support cuda versions from 11.8 and up?

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature requestNew feature or requeststaleOver 90 days of inactivity

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions