DeepSpeed v0.6.0

DeepSpeed v0.6.0

Release notes

New features

• Advancing MoE inference and training to power next-generation AI scale
  • MoE inference
  • PR-MoE model support
• AMD support (#1430)
• Various ZeRO Stage3 Optimizations + Improvements (#1453)

Special thanks to our contributors in this release

@stas00, @jithunnair-amd, @rraminen, @jeffdaily, @okakarpa, @jfc4050, @raamjad, @aphedges, @SeanNaren, @liamcli, @andriyor, @manuelciosici