CUDA Library Samples

CUDA Templates for Linear Algebra Subroutines

🎉 Modern CUDA Learn Notes with PyTorch: CUDA Cores, Tensor Cores, fp32/tf32, fp16/bf16, fp8/int8, flash_attn, rope, sgemm, hgemm, sgemv, warp/block reduce, elementwise, softmax, layernorm, rmsnorm.

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

PaddlePaddle custom device implementaion. (『飞桨』自定义硬件接入实现)

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters

SGLang is a fast serving framework for large language models and vision language models.

Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)

A large-scale simulation framework for LLM inference

AISystem 主要是指AI系统，包括AI芯片、AI编译器、AI推理和训练框架等AI全栈底层技术

Samples for CUDA Developers which demonstrates features in CUDA Toolkit

A high-throughput and memory-efficient inference and serving engine for LLMs

TensorFlow and PyTorch Reference models for Gaudi(R)

Provides the examples to write and build Habana custom kernels using the HabanaTools

Large Language Model Text Generation Inference on Habana Gaudi

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型