FlashInfer: Kernel Library for LLM Serving

Grok open release

A high-throughput and memory-efficient inference and serving engine for LLMs

AICI: Prompts as (Wasm) Programs

MSCCL++: A GPU-driven communication stack for scalable AI applications

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

Updated C version of the Test Suite for Vectorising Compilers

NCCL Profiling Kit

TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches

Microsoft Collective Communication Library

Graph-structured Indices for Scalable, Fast, Fresh and Filtered Approximate Nearest Neighbor Search

Development repository for the Triton language and compiler

Synthesizer for optimal collective communication algorithms