Skip to content

Commit 15a7ddf

Browse files
mrshenlisoumith
authored andcommitted
Highlight that Model Parallel Best Practices performance numbers are environment dependent (#518)
1 parent 6308024 commit 15a7ddf

File tree

1 file changed

+9
-0
lines changed

1 file changed

+9
-0
lines changed

intermediate_source/model_parallel_tutorial.py

+9
Original file line numberDiff line numberDiff line change
@@ -342,3 +342,12 @@ def forward(self, x):
342342
# sub-network structures require different stream management strategies. As no
343343
# general multi-stream solution works for all model parallel use cases, we will
344344
# not discuss it in this tutorial.
345+
#
346+
# **Note:**
347+
#
348+
# This post shows several performance measurements. You might see different
349+
# numbers when running the same code on your own machine, because the result
350+
# depends on the underlying hardware and software. To get the best performance
351+
# for your environment, a proper approach is to first generate the curve to
352+
# figure out the best split size, and then use that split size to pipeline
353+
# inputs.

0 commit comments

Comments
 (0)