The official PyTorch implementation of Google's Gemma models

Cramming the training of a (BERT-type) language model into limited compute.

Code for the accepted CVPR 2023 workshop paper.

Mamba SSM architecture

Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"

State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

Ongoing research training transformer models at scale

Distributed preprocessing and data loading for language datasets