A large-scale simulation framework for LLM inference

A low-latency & high-throughput serving engine for LLMs

A high-throughput and memory-efficient inference and serving engine for LLMs

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

Source code of paper ''KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing''

Transformer Explained Visually: Learn How LLM Transformer Models Work with Interactive Visualization

LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale

LLM Inference analyzer for different hardware platforms

Ongoing research training transformer models at scale

ASTRA-sim 是一个分布式机器学习系统模拟器。它可以系统地研究现代深度学习系统所面临的挑战，探索瓶颈问题，并为未来不同平台上开发大型 DNN 模型提供高效的方法。

ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale

PALM: A Efficient Performance Simulator for Tiled Accelerators with Large-scale Model Training

Artifacts for our ASPLOS'23 paper ElasticFlow

Inference code for Llama models

手把手带你实战 Huggingface Transformers 课程视频同步更新在B站与YouTube

ChatGPT Plus 共享方案。ChatGPT Plus / OpenAI API sharing solution.

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）