Updating MPU docs (#92)

microsoft · Feb 20, 2020 · 2abef1e · 2abef1e
1 parent bca2305
commit 2abef1e
Show file tree

Hide file tree

Showing 2 changed files with 3 additions and 2 deletions.
diff --git a/deepspeed/__init__.py b/deepspeed/__init__.py
@@ -54,7 +54,7 @@ def initialize(args,
             step(), state_dict(), and load_state_dict() methods
 
         mpu: Optional: A model parallelism unit object that implements
-            get_model/data_parallel_group/rank/size()
+            get_{model,data}_parallel_{rank,group,world_size}()
 
         dist_init_required: Optional: Initializes torch.distributed
 

diff --git a/docs/features.md b/docs/features.md
@@ -68,10 +68,11 @@ mpu.get_model_parallel_rank()
 mpu.get_model_parallel_group()
 mpu.get_model_parallel_world_size()
 
-mpu.get_data_parallel_rank/group/world_size()
+mpu.get_data_parallel_rank()
 mpu.get_data_parallel_group()
 mpu.get_data_parallel_world_size()
 ```
+
 ### Integration with Megatron-LM
 DeepSpeed is fully compatible with [Megatron](https://github.com/NVIDIA/Megatron-LM).
 Please see the [Megatron-LM tutorial](tutorials/MegatronGPT2Tutorial.md) for details.