Skip to content

Commit

Permalink
Enable Python-3.11 binary builds across the board (pytorch#94430)
Browse files Browse the repository at this point in the history
Most of the work is outside of repositories and consists of cloning projects https://github.com/AnacondaRecipes/ and building:
- [typing_extensions](https://github.com/AnacondaRecipes/typing_extensions-feedstock)
- [pyyaml](https://github.com/AnacondaRecipes/pyyaml-feedstock)
- [setuptools](https://github.com/AnacondaRecipes/setuptools-feedstock) v 59.8.0, needed to build `numpy`. Trick here is to add `add_pip_as_python_dependency: off` to ones `.condarc`
- [cython](https://github.com/AnacondaRecipes/cython-feedstock)
- [mkl-service](https://github.com/AnacondaRecipes/mkl-service-feedstock)
- [numpy-base](https://github.com/AnacondaRecipes/numpy-feedstock) (against mkl-2021.4), i.e. add `blas_impl: "mkl"` and `mkl: ">=2021.4.0,<2022.0a0"` to ones `conda_build_config.yaml`
- [mkl_random](https://github.com/AnacondaRecipes/mkl_random-feedstock)
- [mkl_fft](https://github.com/AnacondaRecipes/mkl_fft-feedstock)
- [numpy](https://github.com/AnacondaRecipes/numpy-feedstock)
- [mpmath](https://github.com/AnacondaRecipes/mpmath-feedstock)
- [sympy](https://github.com/AnacondaRecipes/sympy-feedstock)

Anaconda build system is really modern, so in order to be able to build:
- x86 MacOS packages, one need to install Macos 10.10 SDK from 2014, still available at https://github.com/phracker/MacOSX-SDKs/releases and reference it as conda build sysroot, as follows: `CONDA_BUILD_SYSROOT: /Library/Developer/CommandLineTools/SDKs/MacOSX10.10.sdk`
- Windows packages "MSVC v141 - VS 2017 C++ x64/86 build tools (v14.16)" is needed, which likely is still available as Visual Studio component

As well as make a pretty trivial tweak to build rules in pytorch/builder@cf4fa89
Pull Request resolved: pytorch#94430
Approved by: https://github.com/seemethere, https://github.com/weiwangmeta, https://github.com/albanD, https://github.com/atalman
  • Loading branch information
malfet authored and pytorchmergebot committed Feb 10, 2023
1 parent a1f15fb commit 89df0e4
Show file tree
Hide file tree
Showing 4 changed files with 917 additions and 9 deletions.
12 changes: 3 additions & 9 deletions .github/scripts/generate_binary_build_matrix.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ def arch_type(arch_version: str) -> str:
("cpu", CXX11_ABI): "pytorch/libtorch-cxx11-builder:cpu",
}

FULL_PYTHON_VERSIONS = ["3.8", "3.9", "3.10"]
FULL_PYTHON_VERSIONS = ["3.8", "3.9", "3.10", "3.11"]


def translate_desired_cuda(gpu_arch_type: str, gpu_arch_version: str) -> str:
Expand All @@ -89,11 +89,7 @@ def list_without(in_list: List[str], without: List[str]) -> List[str]:
def generate_conda_matrix(os: str) -> List[Dict[str, str]]:
ret: List[Dict[str, str]] = []
arches = ["cpu"]
python_versions = list(FULL_PYTHON_VERSIONS)
if os == "linux":
# NOTE: We only build 3.11 on linux right now as many dependencies
# are yet not available on conda
python_versions.append("3.11")
python_versions = FULL_PYTHON_VERSIONS
if os == "linux" or os == "windows":
arches += CUDA_ARCHES
for python_version in python_versions:
Expand Down Expand Up @@ -180,9 +176,7 @@ def generate_wheels_matrix(os: str,
package_type = "manywheel"

if python_versions is None:
# Define default python version
python_versions = list(FULL_PYTHON_VERSIONS)
python_versions.append("3.11")
python_versions = FULL_PYTHON_VERSIONS

if arches is None:
# Define default compute archivectures
Expand Down
112 changes: 112 additions & 0 deletions .github/workflows/generated-macos-arm64-binary-conda-nightly.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

112 changes: 112 additions & 0 deletions .github/workflows/generated-macos-binary-conda-nightly.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 89df0e4

Please sign in to comment.