megatron

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research.

A high-performance framework for training LLMs, VLMs, diffusion, and embodied models on NVIDIA GPUs and Kunlun XPUs.

Megatron was a telegram file management bot that helped a lot of users, specially movie channel managers to upload their files to telegram by just providing a link to it. The project initially started as roanuedhuru_bot which lately retired and came back as Megatron which was a side project of the famous Maldivian Telegram community - @baivaru u…

MCore-Bridge: Providing Megatron-Core model definitions for state-of-the-art large models and making Megatron training as simple as Transformers — with support for 300+ large language models (Qwen3-Next, GLM-5.3, Deepseek-V4, MiniMax-2.7, ...) and 200+ multimodal large models (Qwen3.8, Qwen3-Omni, Gemma4, ...).

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

Newton-Muon + Preconditioned Optimizers for MoE Training at scale, with out-of-the-box support for MuP and FSDP support for Muon, built on top of Megatron-LM and TransformerEngine.

Tiny-Megatron, a minimalistic re-implementation of the Megatron library

[EMNLP 2026] MDM-Prime-v2: Binary Encoding and Index Shuffling Enable Scaling of Diffusion Language Models

A LLaMA1/LLaMA12 Megatron implement.

Source-code level analysis of LLM RL training infra: async RL, weight sync, FP8, MoE routing | LLM RL 训练基础设施源码级分析

Running Large Language Model easily.

A minimal, hackable pre-training stack for GPT-style language models

A minimal, hackable pre-training stack for GPT-style language models

🛠️ Enhance programming workflows with a curated library of agent skills for tasks like code review, debugging, and planning for efficient project execution.

Megatron was a telegram file management bot that helped a lot of users, specially movie channel managers to upload their files to telegram by just providing a link to it. The project initially started as roanuedhuru_bot which lately retired and came back as Megatron which was a side project of the famous Maldivian Telegram community -

Automate Microsoft Account OAuth registration and token imports with this Chrome extension for the Microsoft Account Manager API.

Frex: A high-performance, Unified Virtual Memory (UVM) Deep Learning framework. Engineered for efficient scaling with page-based memory spilling and fused-operator kernels.

Sync vs fully-async agentic RL on verl: multi-turn GRPO, long-tail rollout profiling, staleness ablations — quantifying when async pays off.

A course which takes you through the entire process of setting up a Hugo project with Bootstrap. It also teaches you the basics of bootstrap to get you up and running.