Quantized Attention on GPU

A Python-embedded modeling language for convex optimization problems.

MLIR For Beginners tutorial

The most Obsidian-native PDF annotation, viewing & editing tool ever. Comes with optional Vim keybindings.

DLRover: An Automatic Distributed Deep Learning System

Next-Token Prediction is All You Need

Organize Your GitHub Stars With Ease

Vulkan/CUDA/HIP/OpenCL/Level Zero/Metal Fast Fourier Transform library

compilerbook

Decoding Attention is specially optimized for multi head attention (MHA) using CUDA core for the decoding stage of LLM inference.

A Minimal, Header only Modern c++ library for terminal goodies 💄✨

FlashInfer: Kernel Library for LLM Serving

Examples of CUDA implementations by Cutlass CuTe

Examples demonstrating available options to program multiple GPUs in a single node or a cluster

Quantized Attention that achieves speedups of 2.1x and 2.7x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

Standalone Flash Attention v2 kernel without libtorch dependency

Bloaty: a size profiler for binaries

Puzzles for learning Triton

my cs notes

PyTorch bindings for CUTLASS grouped GEMM.

An acceleration library that supports arbitrary bit-width combinatorial quantization operations

FlagGems is an operator library for large language models implemented in Triton Language.

A fast communication-overlapping library for tensor parallelism on GPUs.

Open deep learning compiler stack for Kendryte AI accelerators ✨

Fast inference from large lauguage models via speculative decoding

Tile primitives for speedy kernels