PyTorch native quantization and sparsity for training and inference

personal website

FlashInfer: Kernel Library for LLM Serving

Code for the paper "Evaluating Large Language Models Trained on Code"

SGLang is yet another fast serving framework for large language models and vision language models.

Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

Structured Text Generation

A curated list for Efficient Large Language Models

Stanford DSPy: The framework for programming—not prompting—foundation models

Ongoing research training transformer language models at scale, including: BERT & GPT-2

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.

Benchmarking large language models' complex reasoning ability with chain-of-thought prompting

gnark-crypto provides elliptic curve and pairing-based cryptography on BN, BLS12, BLS24 and BW6 curves. It also provides various algorithms (algebra, crypto) of particular interest to zero knowledg…

This repository contains integer operators on GPUs for PyTorch.

An Open Source Machine Learning Framework for Everyone

Training and serving large-scale neural networks

Tutorial for writing constraints in the `arkworks` framework

End to end Tensor IR/DSL stack for deploying deep learning workloads to hardwares

Mesh TensorFlow: Model Parallelism Made Easier

A distributed deep learning framework that supports flexible parallelization strategies.

Backprop with Low-Precision Activations

MONeT framework for reducing memory consumption of DNN training

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Dynamic Tensor Rematerialization prototype (modified PyTorch) and simulator. Paper: https://arxiv.org/abs/2006.09616