Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers
-
Updated
Jan 6, 2022 - Python
#
adamw
Here are 30 public repositories matching this topic...
Implements https://arxiv.org/abs/1711.05101 AdamW optimizer, cosine learning rate scheduler and "Cyclical Learning Rates for Training Neural Networks" https://arxiv.org/abs/1506.01186 for PyTorch framework
scheduler clr optimizer pytorch restarts triangular adamw cosine-annealing cyclical-learning-rate adamw-optimizer
-
Updated
Jul 14, 2019 - Python
Newton-Muon + Preconditioned Optimizers for MoE Training at scale, with out-of-the-box support for MuP and FSDP support for Muon, built on top of Megatron-LM and TransformerEngine.
amd optimization cuda transformers pytorch parallelism transformer muon gpt adam preconditioning kfac megatron adamw optimizers foof large-language-models psgd newton-muon newton-muo
-
Updated
Jun 17, 2026
Quasi Hyperbolic Rectified DEMON Adam/Amsgrad with AdaMod, Gradient Centralization, Lookahead, iterative averaging and decorrelated Weight Decay
optimizer ranger adam demon lookahead amsgrad adamw radam adamod gradient-centralization qhadam decay-momentum iterate-averaging qhranger adaptive-optimizer hypergradient-descent hd-adam hd-sgd nosadam nostalgic-adam
-
Updated
Sep 23, 2020 - Python
Pytorch implementation of lookahead optimizer(https://arxiv.org/pdf/1907.08610.pdf) and RAdam(https://arxiv.org/pdf/1908.03265.pdf)
python deep-learning neural-network optimizer pytorch lookahead adamw radam lookahead-optim lookahead-pytorch
-
Updated
Jun 25, 2022 - Jupyter Notebook
Nadir: Cutting-edge PyTorch optimizers for simplicity & composability! 🔥🚀💻
machine-learning optimization pytorch lion adam-optimizer adamax sgd-optimizer amsgrad adamw radam adamp adabelief
-
Updated
Jun 15, 2024 - Python
Literature survey of convex optimizers and optimisation methods for deep-learning; made especially for optimisation researchers with ❤️
-
Updated
Mar 9, 2023
[ICLR 2026] LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning
-
Updated
Feb 28, 2026 - Python
SCAO is a sparse, second-order PyTorch optimizer designed as a high-throughput, drop-in replacement for AdamW.
-
Updated
May 22, 2026 - Python
GYRO is an optimizer for deep neural networks that augments Adam with a geometric rotation step applied to the gradient before momentum buffers are updated.
deep-learning optimizer pytorch gradient-descent optimization-algorithms adam adam-optimizer adamw adamw-optimizer
-
Updated
May 4, 2026 - Python
Clean-room GPT-2/GPT-3 implementation: tokenizers, architecture blocks, training loop with AdamW + cosine decay, CLI scripts, inference tools, and pytest suite. Covers OpenWebText-10k & WikiText-103 workflows. Designed as an academic reference for understanding and scaling decoder-only transformers
nlp transformers pytorch gpu-acceleration language-model adamw gpt2 gpt3 cosine-decay decoder-only educational-implementation
-
Updated
Feb 18, 2026 - Python
Lightweight, zero-dependency C++ Feedforward & Recurrent Neural Network library with native Python bindings (via pybind11).
learning machine-learning deep-learning cplusplus neural-network cpp artificial-intelligence python-wrapper python-bindings sgd deep-learning-library feedforward-neural-network no-dependencies backpropagation from-scratch adam feedforward pybind11 educational-project adamw
-
Updated
Jul 13, 2026 - C++
Kaggle's plant disease image classification competition. Finetuning pre-trained CNN models, loss functions, and optimizers in order to achieve better results.
machine-learning computer-vision deep-learning scheduler cnn plant-pathology resnext adamw efficientnet asymmetric-loss
-
Updated
Aug 4, 2021 - HTML
Computational Graph Library for Neural Network Training
machine-learning deep-learning neural-network automatic-differentiation neural-networks tensor backpropagation computational-graphs loss-functions stochastic-gradient-descent sigmoid adam-optimizer initialization relu schedulers cross-entropy-loss adamw optimizers xavier-initialization computational-graph
-
Updated
Jun 20, 2026 - Java
🕳️ BlackHole Optimizer - A General Relativity approach to optimization. Beats Adam/AdamW via Schwarzschild metric, Hawking radiation, Kerr frame dragging, Penrose process, and Bekenstein-Hawking entropy.
machine-learning deep-learning optimization optimizer pytorch neural-networks gradient-descent general-relativity adam schwarzschild adamw superradiance kaluza-klein penrose-process hawking-radiation physics-inspired kerr-metric blackhole-optimizer bekenstein-hawking
-
Updated
Jul 13, 2026 - Python
reproduce Adam, AdamW, Adafactor optimizors with pytorch, and introduce popular optimizers in the training of the LLMs.
-
Updated
Mar 24, 2025 - Python
Drop-in PyTorch optimizer that beats AdamW with lower variance
-
Updated
Feb 11, 2026 - Python
Improve this page
Add a description, image, and links to the adamw topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the adamw topic, visit your repo's landing page and select "manage topics."