add reproducible compilation environment (#3943)

* add reproducible compilation environment * fix ci * fix typo for formatting check * Fix casing for format --------- Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com> Co-authored-by: Michael Wyatt <mrwyattii@gmail.com> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com> Co-authored-by: Logan Adams <loadams@microsoft.com>
microsoft · Jul 31, 2023 · f763b93 · f763b93
1 parent 8a63754
commit f763b93
Show file tree

Hide file tree

Showing 2 changed files with 30 additions and 0 deletions.
diff --git a/docs/_tutorials/advanced-install.md b/docs/_tutorials/advanced-install.md
@@ -123,6 +123,16 @@ fail. Therefore, if you need to you can override the default location with the h
  TORCH_EXTENSIONS_DIR=./torch-extensions deepspeed ...
 ```
 
+### Conda environment for building from source
+
+If you encounter difficulties during compilation using the default system environment, you can try the conda environment provided, which includes the necessary compilation toolchain and PyTorch.
+
+```bash
+conda env create -n deepspeed -f environment.yml --force
+```
+
+and try above install commands after activating it.
+
 ## Building for the correct architectures
 
 If you're getting the following error:

diff --git a/environment.yml b/environment.yml
@@ -0,0 +1,20 @@
+channels:
+  - nvidia/label/cuda-11.8.0
+  - pytorch # or pytorch-nightly
+  - conda-forge
+dependencies:
+  - pytorch
+  - torchvision
+  - torchaudio
+  - cuda
+  - pytorch-cuda=11.8
+  - compilers
+  - sysroot_linux-64==2.17
+  - gcc==11.4
+  - ninja
+  - py-cpuinfo
+  - libaio
+  - ca-certificates
+  - certifi
+  - openssl
+  - python=3.10