Skip to content
New issue

v0.4.0

Closed Jun 16, 2021 100% complete

Lazy new features

  • 2-D SBP: Support hierarchical N-D SBP Parallel for super large scale distributed training with data/model parallel at the same time.
  • Pipeline Parallelism
  • Checkpointing
  • ZeRO

Eager new features

  • Ease of use align PyTorch with Eager exec in single device. （in flow.experimental.xx namespace）
  • nn.Module for ResNet50, BERT, etc.

Lazy new features

  • 2-D SBP: Support hierarchical N-D SBP Parallel for super large scale distributed training with data/model parallel at the same time.
  • Pipeline Parallelism
  • Checkpointing
  • ZeRO

Eager new features

  • Ease of use align PyTorch with Eager exec in single device. （in flow.experimental.xx namespace）
  • nn.Module for ResNet50, BERT, etc.

This milestone is closed.

No open issues remain. View closed issues or see open milestones in this repository.