vllm-serving

Here are 3 public repositories matching this topic...

The Continuous Verification and Optimization layer for self-hosted LLMs

self-hosted mlops vllm llm-inference vllm-serving

A simple UI and config generator to run vLLM with Docker, GPU settings, model config parsing, memory estimation, and OpenAI-compatible test clients.

python docker flask docker-compose cuda flask-application model-serving mlops huggingface llm vllm vllm-ui vllm-serve vllm-server vllm-serving

面向国产算力（沐曦 C500 / MXMACA）的多通道 AI Agent 运行底座，核心架构实现“上层能力不变，下层算力可换”与混合推理。

To associate your repository with the vllm-serving topic, visit your repo's landing page and select "manage topics."