update-benchmarks

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>

doc/source/ray-air/benchmarks.rst

@@ -104,14 +104,16 @@ XGBoost parameters were kept as defaults for xgboost==1.6.1 this task.
 
 
 GPU image batch prediction
-----------------------------------------------------
+--------------------------
 
 This task uses the BatchPredictor module to process different amounts of data
 using a Pytorch pre-trained ResNet model.
 
 We test out the performance across different cluster sizes and data sizes.
 
 - `GPU image batch prediction script`_
+- `GPU training small cluster configuration`_
+- `GPU training large cluster configuration`_
 
 .. list-table::
 
@@ -134,14 +136,16 @@ We test out the performance across different cluster sizes and data sizes.
 
 
 GPU image training
-------------------------
+------------------
 
 This task uses the TorchTrainer module to train different amounts of data
 using an Pytorch ResNet model.
 
 We test out the performance across different cluster sizes and data sizes.
 
 - `GPU image training script`_
+- `GPU training small cluster configuration`_
+- `GPU training large cluster configuration`_
 
 .. note::
 
@@ -169,10 +173,36 @@ We test out the performance across different cluster sizes and data sizes.
       - `python pytorch_training_e2e.py --data-size-gb=100 --num-workers=16`
 
 
+Pytorch Training Parity
+-----------------------
+
+This task checks the performance parity between native Pytorch Distributed and
+Ray Train's distributed TorchTrainer.
+
+We demonstrate that the performance is similar between the two frameworks.
+
+.. list-table::
+
+    * - **Cluster Setup**
+      - **Dataset**
+      - **Performance**
+      - **Command**
+    * - 4 m5.2xlarge nodes (4 workers)
+      - FashionMNIST
+      - 144.75 s (vs 154.35 s Pytorch)
+      - `python workloads/torch_benchmark.py run --num-runs 3 --num-epochs 20 --num-workers 4 --cpus-per-worker 8`
+    * - 4 g4dn.12xlarge node (16 workers)
+      - FashionMNIST
+      - TODO
+      - `python workloads/torch_benchmark.py run --num-runs 3 --num-epochs 20 --num-workers 16 --cpus-per-worker 4 --use-gpu`
+
+
 .. _`Bulk Ingest Script`: https://github.com/ray-project/ray/blob/a30bdf9ef34a45f973b589993f7707a763df6ebf/release/air_tests/air_benchmarks/workloads/data_benchmark.py#L25-L40
 .. _`Bulk Ingest Cluster Configuration`: https://github.com/ray-project/ray/blob/a30bdf9ef34a45f973b589993f7707a763df6ebf/release/air_tests/air_benchmarks/data_20_nodes.yaml#L6-L15
 .. _`XGBoost Training Script`: https://github.com/ray-project/ray/blob/a241e6a0f5a630d6ed5b84cce30c51963834d15b/release/air_tests/air_benchmarks/workloads/xgboost_benchmark.py#L40-L58
 .. _`XGBoost Prediction Script`: https://github.com/ray-project/ray/blob/a241e6a0f5a630d6ed5b84cce30c51963834d15b/release/air_tests/air_benchmarks/workloads/xgboost_benchmark.py#L63-L71
 .. _`XGBoost Cluster Configuration`: https://github.com/ray-project/ray/blob/a241e6a0f5a630d6ed5b84cce30c51963834d15b/release/air_tests/air_benchmarks/xgboost_compute_tpl.yaml#L6-L24
 .. _`GPU image batch prediction script`: https://github.com/ray-project/ray/blob/cec82a1ced631525a4d115e4dc0c283fa4275a7f/release/air_tests/air_benchmarks/workloads/gpu_batch_prediction.py#L18-L49
-.. _`GPU image training script`: https://github.com/ray-project/ray/blob/cec82a1ced631525a4d115e4dc0c283fa4275a7f/release/air_tests/air_benchmarks/workloads/pytorch_training_e2e.py#L95-L106
+.. _`GPU image training script`: https://github.com/ray-project/ray/blob/cec82a1ced631525a4d115e4dc0c283fa4275a7f/release/air_tests/air_benchmarks/workloads/pytorch_training_e2e.py#L95-L106
+.. _`GPU training small cluster configuration`: https://github.com/ray-project/ray/blob/master/release/air_tests/air_benchmarks/compute_gpu_1.yaml#L6-L24
+.. _`GPU training large cluster configuration`: https://github.com/ray-project/ray/blob/master/release/air_tests/air_benchmarks/compute_gpu_16.yaml#L5-L25