PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538
-
Updated
Apr 19, 2024 - Python
#
sparsely-gated-mixture-of-experts
Here are 2 public repositories matching this topic...
[ICLR 2025] Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization
-
Updated
Oct 5, 2025 - Shell
Improve this page
Add a description, image, and links to the sparsely-gated-mixture-of-experts topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the sparsely-gated-mixture-of-experts topic, visit your repo's landing page and select "manage topics."