Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mpi tests to LGPU #519

Merged
merged 44 commits into from
Oct 18, 2023
Merged
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
f5f5b3f
Initial commit mpi tests
vincentmr Oct 13, 2023
17e0557
Remove label guards
vincentmr Oct 13, 2023
fe2270d
Fix PL_DEVICE
vincentmr Oct 13, 2023
b585943
Install lightning_qubit.
vincentmr Oct 13, 2023
1ed0a4a
Fix ENABLE_MPI
vincentmr Oct 13, 2023
eb6983a
print cuquantum
vincentmr Oct 13, 2023
1502d0c
export cu_sdk
vincentmr Oct 13, 2023
804ed24
revert define
vincentmr Oct 13, 2023
86fca09
Debug cpp tests.
vincentmr Oct 13, 2023
d0a9a79
Debug cpp tests.
vincentmr Oct 13, 2023
5cf7870
Fix cmake options.
vincentmr Oct 13, 2023
9e151de
Compile with mpicxx
vincentmr Oct 16, 2023
2969555
Specify backend.
vincentmr Oct 16, 2023
e4e24b0
Specify backend.
vincentmr Oct 16, 2023
ab08397
Remove obsolete line.
vincentmr Oct 16, 2023
1463b89
Specify cov backend
vincentmr Oct 16, 2023
4ef07c4
Merge test/cov & try simplifying python
vincentmr Oct 16, 2023
f0c90c6
if-no-files-found: error and fix python tests.
vincentmr Oct 16, 2023
b5fcc1f
Fix mpi find
vincentmr Oct 16, 2023
884aa75
Install real lightning.
vincentmr Oct 16, 2023
9115107
Revert python tests.
vincentmr Oct 16, 2023
7636d80
Hardcode backend values in python tests
vincentmr Oct 16, 2023
cd8c67b
Install lightning_qubit with gpu in python tests
vincentmr Oct 16, 2023
2327eb4
Remove explicit mpich mentions.
vincentmr Oct 16, 2023
0ed2bee
Parametrize mpilib name.
vincentmr Oct 16, 2023
42f9d2f
Add openmpi tests.
vincentmr Oct 16, 2023
3a27060
Build only openmpi python tests.
vincentmr Oct 16, 2023
d7bfd80
Add timeouts
vincentmr Oct 16, 2023
eb17997
test/test_apply.py
vincentmr Oct 16, 2023
a3628fb
Revert pull triggers.
vincentmr Oct 16, 2023
f6e0e4d
Clean gpu-mpi test workflows.
vincentmr Oct 16, 2023
8202ce3
Revert to 804ed24.
vincentmr Oct 17, 2023
4027615
Revert back.
vincentmr Oct 17, 2023
f6e1eea
Update tests_linux_x86_mpi.yml [ci skip]
vincentmr Oct 17, 2023
3be766a
Add jobs dep.
vincentmr Oct 17, 2023
dbd251d
Remove module unload
vincentmr Oct 17, 2023
532364d
Simplify mpi-gpu tests.
vincentmr Oct 17, 2023
816af11
trigger CI
vincentmr Oct 18, 2023
0326c06
unset CFLAGS.
vincentmr Oct 18, 2023
f46a6aa
set CFLAGS
vincentmr Oct 18, 2023
76f8e2d
Revert triggers.
vincentmr Oct 18, 2023
e9589ae
Fix pull_request: [skip ci]
vincentmr Oct 18, 2023
9bae63b
trigger CI
vincentmr Oct 18, 2023
be49289
Rename test_gpu_cu11.yml -> tests_gpu_cu11.yml [skip ci]
vincentmr Oct 18, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix PL_DEVICE
  • Loading branch information
vincentmr committed Oct 13, 2023
commit fe2270dbcfb71952358f41ff92f648fee4ed5191
45 changes: 23 additions & 22 deletions .github/workflows/tests_linux_x86_mpich.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ on:
pull_request:

env:
COVERAGE_FLAGS: "--cov=pennylane_lightning_gpu --cov-report=term-missing --cov-report=xml:./coverage.xml --no-flaky-report -p no:warnings --tb=native"
COVERAGE_FLAGS: "--cov=pennylane_lightning --cov-report=term-missing --cov-report=xml:./coverage.xml --no-flaky-report -p no:warnings --tb=native"
GCC_VERSION: 11
OMP_NUM_THREADS: "2"
CI_CUDA_ARCH: 86
Expand All @@ -28,7 +28,7 @@ concurrency:

jobs:
cpp_mpich_tests:
# if: contains(github.event.pull_request.labels.*.name, 'ci:use-multi-gpu-runner') || (inputs.lightning-gpu-version != '' && inputs.pennylane-version != '')
if: contains(github.event.pull_request.labels.*.name, 'ci:use-multi-gpu-runner') || (inputs.lightning-gpu-version != '' && inputs.pennylane-version != '')

runs-on:
- self-hosted
Expand All @@ -44,8 +44,7 @@ jobs:
- name: Checkout
uses: actions/checkout@v3
with:
# Checkout entire git-history if workflow_call passes 'stable' as we need to find the most recent git-tag
fetch-depth: ${{ inputs.lightning-gpu-version == 'stable' && 0 || 1 }}
fetch-tags: true

- name: Switch to stable build of Lightning-GPU
if: inputs.lightning-gpu-version == 'stable'
Expand Down Expand Up @@ -131,23 +130,23 @@ jobs:
module use /opt/modules/
module load mpich
module unload mpich
export CUQUANTUM_SDK=$(python -c "import site; print( f'{site.getsitepackages()[0]}/cuquantum/lib')")
export PATH=$PATH:/usr/local/cuda/bin:/opt/mpi/mpich/include:/opt/mpi/mpich/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/opt/mpi/mpich/lib:$(python -c "import site; print( f'{site.getsitepackages()[0]}/cuquantum/lib')")
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/opt/mpi/mpich/lib:$CUQUANTUM_SDK
cmake . -BBuild \
-DCMAKE_PREFIX_PATH=/opt/mpi/mpich/ \
-DPLLGPU_ENABLE_MPI=On \
-DPLLGPU_ENABLE_MPI=ON \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DPLLGPU_BUILD_TESTS=ON \
-DCMAKE_CXX_COMPILER="$(which g++-${{ env.GCC_VERSION }})" \
-DLIGHTNING_RELEASE_TAG="master" \
-DCMAKE_CUDA_COMPILER="/usr/local/cuda/bin/nvcc" \
-DCMAKE_CUDA_ARCHITECTURES="86" \
-DPython_EXECUTABLE:FILE="${{ steps.python_path.outputs.python }}" \
-G Ninja
cmake --build ./Build
cd ./Build
mkdir -p ./tests/results
/opt/mpi/mpich/bin/mpirun -np 2 ./pennylane_lightning_gpu/src/tests/mpi_runner --order lex --reporter junit --out ./tests/results/report_${{ github.job }}.xml
/opt/mpi/mpich/bin/mpirun -np 2 ./pennylane_lightning/src/tests/mpi_runner --order lex --reporter junit --out ./tests/results/report_${{ github.job }}.xml

- name: Upload test results
uses: actions/upload-artifact@v3
Expand All @@ -162,24 +161,24 @@ jobs:
module use /opt/modules/
module load mpich
module unload mpich
export CUQUANTUM_SDK=$(python -c "import site; print( f'{site.getsitepackages()[0]}/cuquantum/lib')")
export PATH=$PATH:/usr/local/cuda/bin:/opt/mpi/mpich/include:/opt/mpi/mpich/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/opt/mpi/mpich/lib:$(python -c "import site; print( f'{site.getsitepackages()[0]}/cuquantum/lib')")
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/opt/mpi/mpich/lib:$CUQUANTUM_SDK
cmake . -BBuildCov \
-DCMAKE_PREFIX_PATH=/opt/mpi/mpich/ \
-DPLLGPU_ENABLE_MPI=On \
-DPLLGPU_ENABLE_MPI=ON \
-DCMAKE_BUILD_TYPE=Debug \
-DPLLGPU_BUILD_TESTS=ON \
-DPLLGPU_ENABLE_COVERAGE=ON \
-DCMAKE_CXX_COMPILER="$(which g++-${{ env.GCC_VERSION }})" \
-DLIGHTNING_RELEASE_TAG="master" \
-DCMAKE_CUDA_COMPILER="/usr/local/cuda/bin/nvcc" \
-DCMAKE_CUDA_ARCHITECTURES="86" \
-DPython_EXECUTABLE:FILE="${{ steps.python_path.outputs.python }}" \
-G Ninja
cmake --build ./BuildCov
cd ./BuildCov
/opt/mpi/mpich/bin/mpirun -np 2 ./pennylane_lightning_gpu/src/tests/mpi_runner
lcov --directory . -b ../pennylane_lightning_gpu/src --capture --output-file coverage.info
/opt/mpi/mpich/bin/mpirun -np 2 ./pennylane_lightning/src/tests/mpi_runner
lcov --directory . -b ../pennylane_lightning/src --capture --output-file coverage.info
lcov --remove coverage.info '/usr/*' --output-file coverage.info
mv coverage.info coverage-${{ github.job }}.info

Expand Down Expand Up @@ -217,12 +216,13 @@ jobs:
- name: Checkout pennyLane-lightning-gpu
uses: actions/checkout@v3
with:
# Checkout entire git-history if workflow_call passes 'stable' as we need to find the most recent git-tag
fetch-depth: ${{ inputs.lightning-gpu-version == 'stable' && 0 || 1 }}
fetch-tags: true

- name: Switch to stable build of Lightning-GPU
if: inputs.lightning-gpu-version == 'stable'
run: git checkout $(git tag | sort -V | tail -1)
run: |
git fetch tags --force
git checkout $(git tag | sort -V | tail -1)

- uses: actions/setup-python@v4
id: setup_python
Expand Down Expand Up @@ -273,27 +273,28 @@ jobs:
module use /opt/modules/
module load mpich
module unload mpich
export CUQUANTUM_SDK=$(python -c "import site; print( f'{site.getsitepackages()[0]}/cuquantum/lib')")
export PATH=$PATH:/usr/local/cuda/bin:/opt/mpi/mpich/include:/opt/mpi/mpich/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/opt/mpi/mpich/lib:$(python -c "import site; print( f'{site.getsitepackages()[0]}/cuquantum/lib')")
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/opt/mpi/mpich/lib:$CUQUANTUM_SDK
python -m pip install pip~=22.0
python -m pip install ninja cmake custatevec-cu11 pytest pytest-mock flaky pytest-cov mpi4py
# Sync with latest master branches
python -m pip install git+https://github.com/PennyLaneAI/pennylane-lightning.git@master --force-reinstall --no-deps
# python -m pip install git+https://github.com/PennyLaneAI/pennylane-lightning.git@master --force-reinstall --no-deps

- name: Build and install package (MPICH backend)
env:
CUQUANTUM_SDK: $(python -c "import site; print( f'{site.getsitepackages()[0]}/cuquantum/lib')")
run: |
python setup.py build_ext -i --define="CMAKE_CXX_COMPILER=$(which g++-${{ env.GCC_VERSION }});CMAKE_PREFIX_PATH=/opt/mpi/mpich;PLLGPU_ENABLE_MPI=On;LIGHTNING_RELEASE_TAG=master;CMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc;CMAKE_CUDA_ARCHITECTURES=${{ env.CI_CUDA_ARCH }};Python_EXECUTABLE=${{ steps.python_path.outputs.python }}"
python -m pip install -e . --verbose
PL_BACKEND=lightning_gpu python setup.py build_ext -i --define="CMAKE_CXX_COMPILER=$(which g++-${{ env.GCC_VERSION }});CMAKE_PREFIX_PATH=/opt/mpi/mpich;PLLGPU_ENABLE_MPI=ON;CMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc;CMAKE_CUDA_ARCHITECTURES=${{ env.CI_CUDA_ARCH }};Python_EXECUTABLE=${{ steps.python_path.outputs.python }}"
PL_BACKEND=lightning_gpu python -m pip install -e . --verbose

- name: Run PennyLane-Lightning-GPU unit tests (MPICH backend)
run: |
source /etc/profile.d/modules.sh
module use /opt/modules/
module load mpich
/opt/mpi/mpich/bin/mpirun -np 2 python -m pytest ./mpitests/
python -m pytest ./tests/
PL_DEVICE=lightning.gpu /opt/mpi/mpich/bin/mpirun -np 2 python -m pytest ./mpitests/
PL_DEVICE=lightning.gpu python -m pytest ./tests/

- name: Cleanup
if: always()
Expand Down