gpu-sharing

Here are 3 public repositories matching this topic...

kvcached: Elastic KV cache for dynamic GPU sharing and efficient multi-LLM inference.

serverless inference-engine llm llm-serving vllm llm-inference ollama llm-framework sglang kvcache gpu-sharing kvcached gpu-mutiplexing kvcache-optimization

Boosting GPU utilization for LLM serving via dynamic spatial-temporal prefill & decode orchestration

inference llm llm-serving sglang gpu-sharing

GPUs unite using secure and private crypto transactions to distribute compute to decentralized nodes.

Add a description, image, and links to the gpu-sharing topic page so that developers can more easily learn about it.

To associate your repository with the gpu-sharing topic, visit your repo's landing page and select "manage topics."