Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
-
Updated
Jun 17, 2024 - Python
Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
[NeurIPS 24] MoE Jetpack: From Dense Checkpoints to Adaptive Mixture of Experts for Vision Tasks
Code for Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference (CVPR2020)
Mixture of Experts from scratch
Google DeepMind: Mixture of Depths Unofficial Implementation.
Topological Manifold Control: A multi-paradigm (PyTorch/JAX) geometric morphing engine utilizing differentiable soft-gating, optimized via XLA fused kernels and autograd-isolated non-blocking pipelines.
通过基于“滞后梯度博弈”的内生演化压力,替代传统 MoE 的外在负载均衡约束,实现神经网络拓扑结构的自组织特化与层级化
Curated papers on dynamic neural networks, pruning, growing architectures, sparse training, and adaptive computation.
Research code for testing whether causal token surprisal can guide adaptive computation, sparse refinement, and learned compute allocation in byte-level language models.
Research pilot (dormant): can attention-residual signal learn per-input depth routing?
Add a description, image, and links to the conditional-computation topic page so that developers can more easily learn about it.
To associate your repository with the conditional-computation topic, visit your repo's landing page and select "manage topics."