The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

Official release of InternLM2.5 base and chat models. 1M context support

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

📚Modern CUDA Learn Notes with PyTorch: Tensor/CUDA Cores, 📖150+ CUDA Kernels with PyTorch bindings, 📖HGEMM/SGEMM (95%~99% cuBLAS performance), 📖100+ LLM/CUDA Blogs.

FlashInfer: Kernel Library for LLM Serving

InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.

The official CLIP training codebase of Inf-CL: "Breaking the Memory Barrier: Near Infinite Batch Size Scaling for Contrastive Loss". A super memory-efficiency CLIP training scheme.

Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.

Triton implementation of FlashAttention2 that adds Custom Masks.

Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

Decoding Attention is specially optimized for multi head attention (MHA) using CUDA core for the decoding stage of LLM inference.

Python package for rematerialization-aware gradient checkpointing

Utilities for efficient fine-tuning, inference and evaluation of code generation models

Fast and memory efficient PyTorch implementation of the Perceiver with FlashAttention.

Flash Attention Implementation with Multiple Backend Support and Sharding This module provides a flexible implementation of Flash Attention with support for different backends (GPU, TPU, CPU) and platforms (Triton, Pallas, JAX).

An simple pytorch implementation of Flash MultiHead Attention

Long term project about a custom AI architecture. Consist of cutting-edge technique in machine learning such as Flash-Attention, Group-Query-Attention, ZeRO-Infinity, BitNet, etc.

Poplar implementation of FlashAttention for IPU

Toy Flash Attention implementation in torch