Add v0.33 rc changes

.coveragerc

@@ -5,6 +5,9 @@ omit =
     tests/*
     mpitests/*
 
+[coverage:run]
+parallel = true
+
 [report]
 # Regexes for lines to exclude from consideration
 exclude_lines =

.github/CHANGELOG.md

@@ -6,6 +6,9 @@
 
 ### Improvements
 
+* Add MPI synchronization in places to safely handle communicated data.
+  [(#538)](https://github.com/PennyLaneAI/pennylane-lightning/pull/538)
+
 * Add release option in compatibility cron jobs to test the release candidates of PennyLane and the Lightning plugins against one another.
   [(#531)] (https://github.com/PennyLaneAI/pennylane-lightning/pull/531)
 
@@ -16,18 +19,22 @@
 
 ### Bug fixes
 
+* Fix MPI Python unit tests for the adjoint method.
+  [(#538)](https://github.com/PennyLaneAI/pennylane-lightning/pull/538)
+
 ### Contributors
 
 This release contains contributions from (in alphabetical order):
 
-Vincent Michaud-Rioux
-
----
+Vincent Michaud-Rioux, Shuli Shu
 
 # Release 0.33.0
 
 ### New features since last release
 
+* Add documentation updates for the `lightning.gpu` backend.
+  [(#525)] (https://github.com/PennyLaneAI/pennylane-lightning/pull/525)
+
 * Add `SparseHamiltonian` support for Lightning-Qubit and Lightning-GPU.
   [(#526)] (https://github.com/PennyLaneAI/pennylane-lightning/pull/526)
 
@@ -40,7 +47,7 @@ Vincent Michaud-Rioux
 * Integrate the distributed C++ backend of Lightning-GPU into the Lightning monorepo.
   [(#514)] (https://github.com/PennyLaneAI/pennylane-lightning/pull/514)
 
-* Integrate Lightning-GPU into the Lightning monorepo. The new backend is named `lightning_gpu` and includes all single-GPU features.
+* Integrate Lightning-GPU into the Lightning monorepo. The new backend is named `lightning.gpu` and includes all single-GPU features.
   [(#499)] (https://github.com/PennyLaneAI/pennylane-lightning/pull/499)
 
 * Build Linux wheels for Lightning-GPU (CUDA-11).
@@ -54,10 +61,10 @@ Vincent Michaud-Rioux
 
 ### Breaking changes
 
-* Add `tests_gpu.yml` workflow to test the Lightning-Kokkos backend with CUDA-12. 
+* Add `tests_gpu.yml` workflow to test the Lightning-Kokkos backend with CUDA-12.
   [(#494)](https://github.com/PennyLaneAI/pennylane-lightning/pull/494)
 
-* Implement `LM::GeneratorDoubleExcitation`, `LM::GeneratorDoubleExcitationMinus`, `LM::GeneratorDoubleExcitationPlus` kernels. L-Qubit default kernels are now strictly from the `LM` implementation, which requires less memory and is faster for large state vectors.  
+* Implement `LM::GeneratorDoubleExcitation`, `LM::GeneratorDoubleExcitationMinus`, `LM::GeneratorDoubleExcitationPlus` kernels. Lightning-Qubit default kernels are now strictly from the `LM` implementation, which requires less memory and is faster for large state vectors.
   [(#512)](https://github.com/PennyLaneAI/pennylane-lightning/pull/512)
 
 * Add workflows validating compatibility between PennyLane and Lightning's most recent stable releases and development (latest) versions.
@@ -70,7 +77,7 @@ Vincent Michaud-Rioux
 * Cast integral-valued arrays to the device's complex type on entry in `_preprocess_state_vector` to ensure the state is correctly represented with floating-point numbers.
   [(#501)](https://github.com/PennyLaneAI/pennylane-lightning/pull/501)
 
-* Update DefaultQubit to DefaultQubitLegacy on Lightning fallback.
+* Update `DefaultQubit` to `DefaultQubitLegacy` on Lightning fallback.
   [(#500)](https://github.com/PennyLaneAI/pennylane-lightning/pull/500)
 
 * Enums defined in `GateOperation.hpp` start at `1` (previously `0`). `::BEGIN` is introduced in a few places where it was assumed `0` accordingly.
@@ -87,16 +94,16 @@ Vincent Michaud-Rioux
 * Add support for `pip install pennylane-lightning[kokkos]` for the OpenMP backend.
   [(#515)](https://github.com/PennyLaneAI/pennylane-lightning/pull/515)
 
-* Update setup.py to allow for multi-package co-existence. The PennyLane_Lightning package now is the responsible for the core functionality, and will be depended upon by all other extensions.
+* Update `setup.py` to allow for multi-package co-existence. The `PennyLane_Lightning` package now is the responsible for the core functionality, and will be depended upon by all other extensions.
   [(#504)] (https://github.com/PennyLaneAI/pennylane-lightning/pull/504)
 
-* Refactor LKokkos `StateVectorKokkos` class to use Kokkos `RangePolicy` together with special functors in `applyMultiQubitOp` to apply 1- to 4-wire generic unitary gates. For more than 4 wires, the general implementation using Kokkos `TeamPolicy` is employed to yield the best all-around performance.
+* Redesign Lightning-Kokkos `StateVectorKokkos` class to use Kokkos `RangePolicy` together with special functors in `applyMultiQubitOp` to apply 1- to 4-wire generic unitary gates. For more than 4 wires, the general implementation using Kokkos `TeamPolicy` is employed to yield the best all-around performance.
   [(#490)] (https://github.com/PennyLaneAI/pennylane-lightning/pull/490)
 
-* Refactor LKokkos `Measurements` class to use Kokkos `RangePolicy` together with special functors to obtain the expectation value of 1- to 4-wire generic unitary gates. For more than 4 wires, the general implementation using Kokkos `TeamPolicy` is employed to yield the best all-around performance.
+* Redesign Lightning-Kokkos `Measurements` class to use Kokkos `RangePolicy` together with special functors to obtain the expectation value of 1- to 4-wire generic unitary gates. For more than 4 wires, the general implementation using Kokkos `TeamPolicy` is employed to yield the best all-around performance.
   [(#489)] (https://github.com/PennyLaneAI/pennylane-lightning/pull/489)
 
-* Add tests to increase LKokkos coverage.
+* Add tests to increase Lightning-Kokkos coverage.
   [(#485)](https://github.com/PennyLaneAI/pennylane-lightning/pull/485)
 
 * Add memory locality tag reporting and adjoint diff dispatch for `lightning.qubit` statevector classes.
@@ -112,13 +119,16 @@ Vincent Michaud-Rioux
 
 ### Bug fixes
 
+* Fix CI issues running python-cov with MPI. 
+  [(#535)](https://github.com/PennyLaneAI/pennylane-lightning/pull/535)
+
 * Re-add support for `pip install pennylane-lightning[gpu]`.
   [(#515)](https://github.com/PennyLaneAI/pennylane-lightning/pull/515)
 
-* Switch most L-Qubit default kernels to `LM`. Add `LM::multiQubitOp` tests, failing when targeting out-of-order wires clustered close to `num_qubits-1`. Fix the `LM::multiQubitOp` kernel implementation by introducing a generic `revWireParity` routine and replacing the `bitswap`-based implementation. Mimic the changes fixing the corresponding `multiQubitOp` and `expval` functors in L-Kokkos.
+* Switch most Lightning-Qubit default kernels to `LM`. Add `LM::multiQubitOp` tests, failing when targeting out-of-order wires clustered close to `num_qubits-1`. Fix the `LM::multiQubitOp` kernel implementation by introducing a generic `revWireParity` routine and replacing the `bitswap`-based implementation. Mimic the changes fixing the corresponding `multiQubitOp` and `expval` functors in Lightning-Kokkos.
   [(#511)](https://github.com/PennyLaneAI/pennylane-lightning/pull/511)
 
-* Fix RTD builds by removing unsupported `sytem_packages` configuration option.
+* Fix RTD builds by removing unsupported `system_packages` configuration option.
   [(#491)](https://github.com/PennyLaneAI/pennylane-lightning/pull/491)
 
 ### Contributors
@@ -133,7 +143,7 @@ Ali Asadi, Amintor Dusko, Vincent Michaud-Rioux, Lee J. O'Riordan, Shuli Shu
 
 ### New features since last release
 
-* The `lightning_kokkos` backend supports Nvidia GPU execution (with Kokkos v4 and CUDA v12).
+* The `lightning.kokkos` backend supports Nvidia GPU execution (with Kokkos v4 and CUDA v12).
   [(#477)](https://github.com/PennyLaneAI/pennylane-lightning/pull/477)
 
 * Complete overhaul of repository structure to facilitates integration of multiple backends. Refactoring efforts we directed to improve development performance, code reuse and decrease overall overhead to propagate changes through backends. New C++ modular build strategy allows for faster test builds restricted to a module. Update CI/CD actions concurrency strategy. Change minimal Python version to 3.9.

.github/workflows/tests_gpu_cu11.yml

@@ -79,7 +79,6 @@ jobs:
         uses: actions/checkout@v3
         with:
           path: main
-          fetch-depth: 2
 
       - uses: actions/setup-python@v4
         name: Install Python

.github/workflows/tests_linux_x86_mpi_gpu.yml

@@ -91,13 +91,15 @@ jobs:
 
       - name: Install required packages
         run: |
-          python -m pip install ninja cmake custatevec-cu11
+          python -m pip install -r requirements-dev.txt
+          python -m pip install cmake custatevec-cu11
 
       - name: Validate GPU version and installed compiler
         run: |
           source /etc/profile.d/modules.sh && module use /opt/modules && module load cuda/11.8
           which -a nvcc
           nvcc --version
+
       - name: Validate Multi-GPU packages
         run: |
           source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}
@@ -107,9 +109,6 @@ jobs:
           which -a mpicxx
           mpicxx --version
           module unload ${{ matrix.mpilib }}
-      - name: Install Latest PennyLane
-        if: inputs.pennylane-version == 'latest'
-        run: python -m pip install git+https://github.com/PennyLaneAI/pennylane.git@master
 
       - name: Build and run unit tests
         run: |
@@ -222,16 +221,11 @@ jobs:
           echo "PIP Path => $pip_path"
           echo "pip=$pip_path" >> $GITHUB_OUTPUT
 
-      - name: Install Latest PennyLane
-        # We want to install the latest PL on non workflow_call events
-        if: inputs.pennylane-version == 'latest'  || inputs.pennylane-version == ''
-        run: python -m pip install git+https://github.com/PennyLaneAI/pennylane.git@master
-
       - name: Install required packages
         run: |
           source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}
-          python -m pip install pip~=22.0
-          python -m pip install ninja cmake custatevec-cu11 pytest pytest-mock flaky pytest-cov mpi4py openfermionpyscf
+          python -m pip install -r requirements-dev.txt
+          python -m pip install custatevec-cu11 mpi4py openfermionpyscf
           SKIP_COMPILATION=True PL_BACKEND=lightning_qubit python -m pip install -e . -vv
 
       - name: Build and install package
@@ -242,12 +236,15 @@ jobs:
           CMAKE_ARGS="-DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx -DENABLE_MPI=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc -DCMAKE_CUDA_ARCHITECTURES=${{ env.CI_CUDA_ARCH }} -DPython_EXECUTABLE=${{ steps.python_path.outputs.python }}" \
           PL_BACKEND=lightning_gpu python -m pip install -e . --verbose
 
+      # There are issues running py-cov with MPI. A solution is to use coverage as reported
+      # [here](https://github.com/pytest-dev/pytest-cov/issues/237#issuecomment-544824228)
       - name: Run unit tests for MPI-enabled lightning.gpu device
         run: |
           source /etc/profile.d/modules.sh && module use /opt/modules/ && module load ${{ matrix.mpilib }}
-          PL_DEVICE=lightning.gpu /opt/mpi/${{ matrix.mpilib }}/bin/mpirun -np 2 python -m pytest ./mpitests $COVERAGE_FLAGS
-          mv coverage.xml coverage-${{ github.job }}-lightning_gpu_${{ matrix.mpilib }}-main.xml
-          # PL_DEVICE=lightning.gpu /opt/mpi/${{ matrix.mpilib }}/bin/mpirun --oversubscribe -n 4 pytest -s -x mpitests/test_device.py -k test_create_device $COVERAGE_FLAGS
+          PL_DEVICE=lightning.gpu /opt/mpi/${{ matrix.mpilib }}/bin/mpirun -np 2 \
+          coverage run --rcfile=.coveragerc --source=pennylane_lightning -p -m mpi4py -m pytest ./mpitests --tb=native
+          coverage combine
+          coverage xml -o coverage-${{ github.job }}-lightning_gpu_${{ matrix.mpilib }}-main.xml
 
       - name: Upload code coverage results
         uses: actions/upload-artifact@v3

.github/workflows/wheel_linux_x86_64_cu11.yml

@@ -87,7 +87,7 @@ jobs:
             LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:$CUQUANTUM_SDK \
             PL_BACKEND="${{ matrix.pl_backend }}"
 
-          CIBW_REPAIR_WHEEL_COMMAND_LINUX: "auditwheel repair -w {dest_dir} {wheel}"
+          CIBW_REPAIR_WHEEL_COMMAND_LINUX: "./bin/auditwheel repair -w {dest_dir} {wheel}"
 
           CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
 
@@ -114,6 +114,10 @@ jobs:
 
   upload-pypi:
     needs: linux-wheels-x86-64
+    strategy:
+      matrix:
+        arch: [x86_64]
+        pl_backend: ["lightning_gpu"]
     runs-on: ubuntu-latest
     if: ${{ github.event_name == 'release' || github.ref == 'refs/heads/master'}}
     steps:

.github/workflows/wheel_noarch.yml

@@ -22,7 +22,7 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest]
-        pl_backend: ["lightning_kokkos", "lightning_qubit"]
+        pl_backend: ["lightning_gpu", "lightning_kokkos", "lightning_qubit"]
     timeout-minutes: 30
     name: ${{ matrix.os }} - Pure Python wheels - ${{ matrix.pl_backend }} (Python 3.9)
     runs-on: ${{ matrix.os }}
@@ -46,41 +46,66 @@ jobs:
           python -m pip install --upgrade cmake ninja
 
       - name: Build wheels
+        if: ${{ matrix.pl_backend == 'lightning_qubit'}}
         run: |
           python -m pip install --upgrade pip wheel
           cd main
-          python setup.py bdist_wheel
+          PL_BACKEND="${{ matrix.pl_backend }}" python setup.py bdist_wheel
         env:
           SKIP_COMPILATION: True
 
       - name: Validate wheels
+        if: ${{ matrix.pl_backend == 'lightning_qubit'}}
         run: |
           python -m pip install twine
           python -m twine check main/dist/*.whl
 
       - uses: actions/upload-artifact@v3
-        if: ${{ github.event_name == 'release' || github.ref == 'refs/heads/master' }}
+        if: ${{ matrix.pl_backend == 'lightning_qubit' && (github.event_name == 'release' || github.ref == 'refs/heads/master') }}
         with:
           name: pure-python-wheels-${{ matrix.pl_backend }}.zip
           path: main/dist/*.whl
 
+      - name: Build source dist
+        if: ${{ matrix.pl_backend != 'lightning_qubit'}}
+        run: |
+          python -m pip install --upgrade pip wheel
+          cd main
+          PL_BACKEND="${{ matrix.pl_backend }}" python setup.py sdist
+        env:
+          SKIP_COMPILATION: True
+
+      - uses: actions/upload-artifact@v3
+        if: ${{ matrix.pl_backend != 'lightning_qubit' && (github.event_name == 'release' || github.ref == 'refs/heads/master') }}
+        with:
+          name: pure-source-dist-${{ matrix.pl_backend }}.tar.gz
+          path: main/dist/*.tar.gz
+
   upload-pypi:
     needs: build-pure-python-wheel
     strategy:
       matrix:
-        pl_backend: ["lightning_qubit"]
+        pl_backend: ["lightning_gpu", "lightning_kokkos", "lightning_qubit"]
     runs-on: ubuntu-latest
 
-    if: ${{ github.event_name == 'release' }}
     steps:
       - uses: actions/download-artifact@v3
+        if: ${{ matrix.pl_backend == 'lightning_qubit' && github.event_name == 'release' }}
         with:
           name: pure-python-wheels-${{ matrix.pl_backend }}.zip
           path: dist
 
       - name: Upload wheels to PyPI
+        if: ${{ matrix.pl_backend == 'lightning_qubit' && github.event_name == 'release' }}
         uses: pypa/gh-action-pypi-publish@release/v1
         with:
           user: __token__
           password: ${{ secrets.TEST_PYPI_API_TOKEN }}
-          repository-url: https://test.pypi.org/legacy/
+          repository-url: https://test.pypi.org/legacy/
+
+      - uses: actions/download-artifact@v3
+        if: ${{ matrix.pl_backend != 'lightning_qubit' && github.event_name == 'release' }}
+        with:
+          name: pure-source-dist-${{ matrix.pl_backend }}.tar.gz
+          path: dist
+

.readthedocs.yml

@@ -21,8 +21,11 @@ build:
     - libopenblas-base
     - libopenblas-dev
     - graphviz
+    - nvidia-cuda-toolkit
   jobs:
     pre_install:
       - echo "setuptools~=66.0\npip~=22.0" >> ci_build_requirements.txt
     post_install:
-      - PL_BACKEND="lightning_kokkos" pip install -e . -vv
+      - rm -rf ./build && PL_BACKEND="lightning_kokkos" python setup.py bdist_wheel
+      - rm -rf ./build && PL_BACKEND="lightning_gpu" python setup.py build_ext --define="PL_DISABLE_CUDA_SAFETY=1" && PL_BACKEND="lightning_gpu" python setup.py bdist_wheel
+      - python -m pip install ./dist/*.whl