Popular repositories Loading

1. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

2. multi-part-simulator multi-part-simulator Public

   Go

3. gateway-api-inference-extension gateway-api-inference-extension Public

   Forked from kubernetes-sigs/gateway-api-inference-extension

   Gateway API Inference Extension

   Go

4. kvcached kvcached Public

   Forked from ovg-project/kvcached

   kvcached: Elastic KV cache for dynamic GPU sharing and efficient multi-LLM inference.

   Python

5. amazon-bedrock-agentcore-samples amazon-bedrock-agentcore-samples Public

   Forked from awslabs/amazon-bedrock-agentcore-samples

   Amazon Bedrock Agentcore accelerates AI agents into production with the scale, reliability, and security, critical to real-world deployment.

   Jupyter Notebook

6. llm-d-infra llm-d-infra Public

   Forked from llm-d-incubation/llm-d-infra

   llm-d helm charts and deployment examples

   Shell