Skip to content

Commit

Permalink
[Nova] Add GHA Linux CPU Unittests for Torchvision (#6759)
Browse files Browse the repository at this point in the history
* [Nova][WIP] Add Linux CPU Unittests for Torchvision

* use conda-builder image since conda installation is needed

* install torch dep with conda instead

* use circleCI command to run tests

* larger instance to avoid OOM issues

* proper syntax for self-hosted runners

* 4xlarge instance

* 8xlarge

* 12xlarge

* use setup-miniconda job

* add back PATH change to help setup py detect conda

* run conda shell script

* install other deps up front

* git config and undo path change

* revert to local conda install

* conda-builder image

* support for whole python version matrix

* clean up the conda env once we are done with the job
  • Loading branch information
osalpekar authored Oct 17, 2022
1 parent decb191 commit 0610b13
Showing 1 changed file with 69 additions and 0 deletions.
69 changes: 69 additions & 0 deletions .github/workflows/test-linux-cpu.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
name: Unit-tests on Linux CPU

on:
pull_request:
push:
branches:
- nightly
- main
- release/*
workflow_dispatch:

env:
CHANNEL: "nightly"

jobs:
tests:
name: "Unit-tests on Linux CPU"
runs-on: [self-hosted, linux.12xlarge]
container:
image: pytorch/conda-builder:cpu
strategy:
matrix:
py_vers: ["3.7", "3.8", "3.9", "3.10"]

steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Set Release CHANNEL (for release)
if: ${{ (github.event_name == 'pull_request' && startsWith(github.base_ref, 'release')) || startsWith(github.ref, 'refs/heads/release') }}
run: |
echo "CHANNEL=test" >> "$GITHUB_ENV"
- name: Setup Conda
shell: bash -l {0}
env:
ENV_NAME: conda-env-${{ github.run_id }}
PY_VERS: ${{ matrix.py_vers }}
run: |
git config --global --add safe.directory /__w/vision/vision
. ~/miniconda3/etc/profile.d/conda.sh
conda create -yp ${ENV_NAME} python=${PY_VERS} numpy libpng jpeg scipy
echo "CONDA_RUN=conda run -p ${ENV_NAME}" >> "$GITHUB_ENV"
- name: Install TorchVision
shell: bash -l {0}
env:
VERSION: cpu
CUDATOOLKIT: cpuonly
run: |
# Needed for JPEG library detection as setup.py detects conda presence
# by running `shutil.which('conda')`
export PATH=~/miniconda3/bin:$PATH
set -ex
${CONDA_RUN} conda install \
--yes \
-c "pytorch-${CHANNEL}" \
-c nvidia "pytorch-${CHANNEL}"::pytorch[build="*${VERSION}*"] \
"${CUDATOOLKIT}"
${CONDA_RUN} python3 setup.py develop
${CONDA_RUN} python3 -m pip install pytest pytest-mock av
- name: Run tests
shell: bash -l {0}
env:
ENV_NAME: conda-env-${{ github.run_id }}
PY_VERS: ${{ matrix.py_vers }}
run: |
. ~/miniconda3/etc/profile.d/conda.sh
set -ex
${CONDA_RUN} python3 -m torch.utils.collect_env
${CONDA_RUN} python3 -m pytest --junitxml=test-results/junit.xml -v --durations 20
conda env remove -p ${ENV_NAME}

0 comments on commit 0610b13

Please sign in to comment.