5 Pull requests merged by 4 people

• Protect import for device_mesh

  #3742 merged Aug 22, 2025

• Feat: add to_json

  #3743 merged Aug 22, 2025

• fix: CPU RAM efficient loading for nd or HSDP parallelisms

  #3740 merged Aug 21, 2025

• Fix convert LayerNorm without bias to fp8

  #3725 merged Aug 18, 2025

• feat: add ignored_params support for fsdp2

  #3731 merged Aug 18, 2025

4 Pull requests opened by 3 people

• [ND Parallel] Update examples, cleanup

  #3737 opened Aug 18, 2025

• [WIP] Upstreaming FSDP2 changes

  #3739 opened Aug 19, 2025

• Specify device_ids in torch.distributed.barrier for PartialState

  #3744 opened Aug 22, 2025

• [Context Parallel] Experimental support for flex attention

  #3745 opened Aug 23, 2025

3 Issues closed by 2 people

• Accelerate launch with Zero3, Model Parallelism in the same machine when multinode train.

  #3738 closed Aug 22, 2025

• The Socket Timeout is displayed when a PPO is trained on multiple machines

  #3658 closed Aug 18, 2025

• weight is not tied within `init_empty_weights`

  #3668 closed Aug 18, 2025

1 Issue opened by 1 person

• accelerate.prepare() will consume much NPU memory

  #3741 opened Aug 21, 2025

13 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Deferred initialization support

  #3666 commented on Aug 17, 2025 • 0 new comments

• Transformer Engine memory-efficient initialization to convert_model for large models

  #3652 commented on Aug 17, 2025 • 0 new comments

• dtype issue when using accelerate

  #3685 commented on Aug 18, 2025 • 0 new comments

• Can devices be specified in distributed training

  #3726 commented on Aug 20, 2025 • 0 new comments

• CUDA Out of Memory using 2 GPUs

  #3735 commented on Aug 21, 2025 • 0 new comments

• Advice with GPU computations in collator/pre-process

  #3691 commented on Aug 21, 2025 • 0 new comments

• [DeepSpeed] GPU VRAM usage increases each validation step

  #3690 commented on Aug 21, 2025 • 0 new comments

• What will happened if I use mix_precision but don't add with accelerator.autocast():

  #3689 commented on Aug 21, 2025 • 0 new comments

• Multi-XPU launch error

  #3664 commented on Aug 21, 2025 • 0 new comments

• logger message "dataset had no length" confusing when `drop_last=True`

  #3693 commented on Aug 22, 2025 • 0 new comments

• bugs in accelerator.unwrap_model

  #3683 commented on Aug 22, 2025 • 0 new comments

• [WIP] optimize infer_auto_device_map for multi-GPU allocation

  #3321 commented on Aug 22, 2025 • 0 new comments

• Unwrap before saving/loading sharded model

  #3733 commented on Aug 22, 2025 • 0 new comments