8-bit Adafactor Optimizer with Fused CUDA Kernels
-
Updated
Jul 12, 2026 - Python
8-bit Adafactor Optimizer with Fused CUDA Kernels
A from-scratch 336M-parameter language model built with JAX and Equinox
reproduce Adam, AdamW, Adafactor optimizors with pytorch, and introduce popular optimizers in the training of the LLMs.
Add a description, image, and links to the adafactor topic page so that developers can more easily learn about it.
To associate your repository with the adafactor topic, visit your repo's landing page and select "manage topics."