Popular repositories Loading

1. vllm vllm Public

   Forked from kaln27/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

2. inference-benchmarker inference-benchmarker Public

   Forked from huggingface/inference-benchmarker

   Inference server benchmarking tool

   Rust

3. LLM_Sizing_Guide LLM_Sizing_Guide Public

   Forked from qoofyk/LLM_Sizing_Guide

   A calculator to estimate the memory footprint, capacity, and latency on VMware Private AI with NVIDIA.

   Python