Repository navigation

All pull requests

fixed args deepspeed_config for gpt pretraining
#485QuPengfei opened Jan 22, 2026
spelling fix: contiguous checkpointing was spelled contigious
#484dalongbao opened Nov 4, 2025
Fix AttributeError in hf2megads weight converter
#483KeitaW opened Sep 27, 2025
fix pretrain gpt and llama bug
#4769LLPPLL6 opened May 8, 2025
2
FastPersist rebase
#467tjruwase opened Feb 25, 2025
support split qkv linear and sp overlap comm
#415inkcherry opened Jul 5, 2024
10
fix --use-cpu-initialization error when expert is not tensor-parallel
#413taozhiwei opened Jul 3, 2024
3
fix NAN loss of rope long context training
#399inkcherry opened Jun 5, 2024
1
convert mds checkpoint to Hf Llama model
#394vksastry opened May 31, 2024
1
ds-sequence-parallel(ulysses) for rope.
#392inkcherry opened May 30, 2024
1
add HFTokenizer option for preprocess_data
#388Jianhong-Zhang opened May 17, 2024
1
Add layer norm weight plus 1
#378Yejing-Lai opened Apr 18, 2024
4
Fix ConstantGradScaler and loss-scale argument not match
#376BeingGod opened Apr 12, 2024
1
Support Llama2Tokenizer
#375jinyouzhi opened Apr 11, 2024
collect grad_norm for non pipeline path
#370inkcherry opened Mar 21, 2024
optimize the generation of attention mask
#331imh966 opened Jan 13, 2024
Enable torch.compile
#322tohtana opened Dec 28, 2023
Simplify SP - Opportunity to improve SP scalability
#301RezaYazdaniAminabadi opened Nov 28, 2023
support transfer llama hf weight to megatron weight
#246uygnef opened Sep 12, 2023
27
add vit training with TP/PP
#146etoilestar opened Jun 9, 2023
3
attempt at pipelining
#78siddharth9820 opened Aug 18, 2022
Add support for DS comms
#50Quentin-Anthony opened Jun 13, 2022
1

Pull requests