#
sglang
Here are 40 public repositories matching this topic...
GPU cluster manager for optimized AI model deployment
cuda inference openai llama maas rocm ascend llm llm-serving vllm genai llm-inference qwen deepseek sglang distributed-inference high-performance-inference mindie
-
Updated
Dec 7, 2025 - Python
MOSS-TTSD is a spoken dialogue generation model that enables expressive dialogue speech synthesis in both Chinese and English, supporting zero-shot multi-speaker voice cloning, and long-form speech generation.
-
Updated
Nov 28, 2025 - Python
LLM model quantization (compression) toolkit with hw acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.
-
Updated
Dec 5, 2025 - Python
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
serverless inference-engine llm llm-serving vllm llm-inference ollama llm-framework sglang kvcache gpu-sharing kvcached gpu-mutiplexing kvcache-optimization elastic-kvcache online-offline-coserve
-
Updated
Nov 30, 2025 - Python
基于SparkTTS、OrpheusTTS等模型，提供高质量中文语音合成与声音克隆服务。
-
Updated
May 18, 2025 - Python
OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)
k8s llama oracle-cloud model-serving model-as-a-service multi-node-kubernetes llm llm-inference deepseek sglang kimi-k2 sgalng
-
Updated
Dec 7, 2025 - Go
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
kubernetes inference huggingface llm modelscope llamacpp vllm text-generation-inference ollama sglang inference-platform
-
Updated
Nov 24, 2025 - Go
High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.
golang ai amd proxy intel self-hosted nvidia mlx llamacpp llama-cpp vllm llm-inference local-ai ollama llm-proxy lmstudio sglang llm-router llm-routing self-hosted-ai
-
Updated
Nov 23, 2025 - Go
A tool for benchmarking LLMs on Modal
-
Updated
Aug 29, 2025 - Python
Arks is a cloud-native inference framework running on Kubernetes
-
Updated
Nov 20, 2025 - Go
A high-performance RDMA distributed file system for fast LLM Inference and GPU Training
python big-data cpp gpu cuda rdma infiniband distributed-cache kv-cache ucx llm-serving vllm llm-framework sglang gpu-multiplexing
-
Updated
Nov 25, 2025 - C++
Boosting GPU utilization for LLM serving via dynamic spatial-temporal prefill & decode orchestration
-
Updated
Dec 2, 2025 - Python
DeepSeek-V3, R1 671B on 8xH100 Throughput Benchmarks
-
Updated
Mar 13, 2025 - Python
A guide to structured generation using constrained decoding
-
Updated
Jun 9, 2024 - Jupyter Notebook
Improve this page
Add a description, image, and links to the sglang topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the sglang topic, visit your repo's landing page and select "manage topics."