GitHub · Where software is built

Preview

Labels Milestones New issue

[REQUEST] An option to only save the model state_dict when save_checkpoint(), and how to manually save & load the model state_dict when using ZERO3

#2304

· BlinkDL opened on Sep 8, 2022

Schema for ds_config.json

#2355

· timoklimmer opened on Sep 24, 2022

[QUESTION] Pipeline explaination needed

#2435

· bruno-darochac opened on Oct 20, 2022

[Question] why are overlap and contiguous grads meaningless in stage 1 and are ignored

#2295

· woolpeeker opened on Sep 6, 2022

DeepSpeed still gives CUDA-out-of-memory issue

#2302

· buttercutter opened on Sep 7, 2022

[BUG] load_checkpoint should load directly to gpu

#1971

· stas00 opened on May 21, 2022

[BUG] AttributeError: 'NoneType' object has no attribute 'swap_folder'

#4998

· maywind23 opened on Jan 23, 2024

[REQUEST] Expert Choice Routing for MoE

#2517

· clumsy opened on Nov 17, 2022

[REQUEST] how to Wrap normalization layers like LayerNorm in FP32 when use zero (fp16 or bf16)?

#2667

· xiaohu2015 opened on Jan 4, 2023

[REQUEST] How to use `Flops Profiler` to test `model.generate()`

#2514

· CaffreyR opened on Nov 17, 2022

[BUG] need some asserts instead of warnings or silence

#2658

· stas00 opened on Dec 30, 2022

[REQUEST] `zero.Init` and silent skipping of custom `_init_weights` functions

#2650

· stas00 opened on Dec 25, 2022