Pennylane is compatible with numpy 2.0 #6061

EmilianoG-byte · 2024-08-01T16:00:59Z

Context:

We want to make Pennylane compatible with Numpy 2.0. After several discussions, we decided to test NumPy 2.0 on the CI by default in every PR (testing both Python versions would have been to slow). Some jobs still downgrade automatically to Numpy 1.x, since some interfaces (such as Tensorflow) still do not support NumPy 2.0.

Description of the Change:

We can distinguish the changes into 3 main categories:

Changes to workflows

None in the final version

Changes to requirements and setup files

Unpin the Numpy version in setup.py (now we also allow Numpy 2.0).
Update requirements-ci.txt to include Scipy 1.13 (this adds support for Numpy 2.0).

Changes to the source code

Change np.NaN to np.nan.
Use legacy printing representation in tests, contrary to the new numpy representation of scalars, e.g. np.float64(3.0) rather than just 3.0.
Update probabilities warning to be case insensitive and check for a partial match, since this warning was changed in Numpy 2.0.
Check the datatype of np.exp from the Global phase only for Numpy 1.x, since this gets promoted to complex128 in Numpy 2.x. https://numpy.org/neps/nep-0050-scalar-promotion.html#schema-of-the-new-proposed-promotion-rules.

Benefits:

Make Pennylane compatible with Numpy 2.0.

Possible Drawbacks:

We need to create a separate workflow to keep testing PennyLane with NumPy 1.x, since we still want to maintain compatibility with previous NumPy versions. This will be done in a separate PR.
We are not testing Numpy 2.x for the interfaces that implicitly require Numpy 1.x. These currently seem to be tensorflow and openfermionpyscf (notice that tensorflow is required in some code sections like qcut). In particular, openfermionpyscf causes an error:

AttributeError: np.string_ was removed in the NumPy 2.0 release. Use np.bytes_ instead.

in the qchem tests. The attribute np.string_ is not used in the PL source code, so it is a problem with the package itself.

[sc-61399]
[sc-66548]

github-actions · 2024-08-01T16:01:15Z

Hello. You may have forgotten to update the changelog!
Please edit doc/releases/changelog-dev.md with:

A one-to-two sentence description of the change. You may include a small working example for new features.
A link back to this PR.
Your name (or GitHub username) in the contributors section.

….nan

…aN to np.nan" This reverts commit b3ab335. Reverting changes from numpy formating and NAN

revert changes from numpy printing settings

attempt to use the legacy printing option

…ion 1.21

codecov · 2024-08-28T17:23:55Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.58%. Comparing base (d0344b0) to head (591671d).
Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6061      +/-   ##
==========================================
- Coverage   99.59%   99.58%   -0.01%     
==========================================
  Files         443      443              
  Lines       42255    42255              
==========================================
- Hits        42082    42078       -4     
- Misses        173      177       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

astralcai

Is this PR only fixing tests?

.github/workflows/install_deps/action.yml

pennylane/numpy/random.py

tests/data/attributes/operator/test_operator.py

mlxd

Thanks @EmilianoG-byte for all of the work
A few questions from my side:

.github/workflows/install_deps/action.yml

requirements-ci.txt

tests/data/attributes/operator/test_operator.py

tests/devices/test_default_qubit_legacy.py

mudit2812

Looks good, I left a few comments and suggestions.

.github/workflows/install_deps/action.yml

.github/workflows/interface-unit-tests.yml

.gitignore

tests/data/attributes/test_dict.py

tests/devices/test_default_qubit_legacy.py

PietropaoloFrisoni · 2024-09-13T13:28:56Z

Update: now, Codecov project complains since it detects a -0.07% coverage compared to the commit associated with this PR. As specified in the PR itself, Code cov failures are in default.qubit.legacy and QubitDevice, so can be ignored.

…o compatible-np-2.0

… is already in `setup.py`

…o compatible-np-2.0

**Context:** We want to make Pennylane compatible with Numpy 2.0. After several discussions, we decided to test NumPy 2.0 on the CI by default in every PR (testing both Python versions would have been to slow). Some jobs still downgrade automatically to Numpy 1.x, since some interfaces (such as Tensorflow) still do not support NumPy 2.0. **Description of the Change:** We can distinguish the changes into 3 main categories: *Changes to workflows* - None in the final version *Changes to requirements and setup files* - Unpin the Numpy version in `setup.py` (now we also allow Numpy 2.0). - Update `requirements-ci.txt` to include Scipy 1.13 (this adds support for Numpy 2.0). - Pin Numpy in `requirements-ci.txt` to 2.0. *Changes to the source code* - Change `np.NaN` to `np.nan`. - Use legacy printing representation in tests, contrary to the new numpy representation of scalars, e.g. np.float64(3.0) rather than just 3.0. - Update probabilities warning to be case insensitive and check for a partial match, since this warning was changed in Numpy 2.0. - Check the datatype of np.exp from the Global phase only for Numpy 1.x, since this gets promoted to complex128 in Numpy 2.x. https://numpy.org/neps/nep-0050-scalar-promotion.html#schema-of-the-new-proposed-promotion-rules. **Benefits:** Make Pennylane compatible with Numpy 2.0. **Possible Drawbacks:** - We need to create a separate workflow to keep testing PennyLane with NumPy 1.x, since we still want to maintain compatibility with previous NumPy versions. This will be done in a separate PR. - We are not testing Numpy 2.x for the interfaces that implicitly require Numpy 1.x. These currently seem to be `tensorflow` and `openfermionpyscf` (notice that `tensorflow` is required in some code sections like qcut). In particular, `openfermionpyscf` causes an error: ``` AttributeError: np.string_ was removed in the NumPy 2.0 release. Use np.bytes_ instead. ``` in the qchem tests. The attribute `np.string_` is not used in the PL source code, so it is a problem with the package itself. [sc-61399] [sc-66548] --------- Co-authored-by: PietropaoloFrisoni <pietropaolo.frisoni@xanadu.ai> Co-authored-by: Pietropaolo Frisoni <pietropfrisoni@gmail.com>

**Context:** As part of the effort to make PL compatible with Numpy 2.0 (see #6061), we need to upgrade JAX to 0.4.26+ since such a version introduced the support for Numpy 2.0. We opted for JAX 0.4.28 since it is the same version used by Catalyst. **Description of the Change:** As above. **Benefits:** PL is compatible with Numpy 2.0 and Jax 0.4.28. **Possible Drawbacks:** - From JAX 0.4.27, in `jax.jit`, passing invalid static_argnums or static_argnames now leads to an error rather than a warning. In PL, this breaks every test where we set `shots` in the `QNode` call with `static_argnames=["shots"]`. At this stage, we decided to mark such tests with `pytest.xfail` to allow the upgrade. **Related GitHub Issues:** None. **Related Shortcut Stories**: [sc-61389] --------- Co-authored-by: dwierichs <david.wierichs@xanadu.ai>

**Context:** We want to make Pennylane compatible with Numpy 2.0. After several discussions, we decided to test NumPy 2.0 on the CI by default in every PR (testing both Python versions would have been to slow). Some jobs still downgrade automatically to Numpy 1.x, since some interfaces (such as Tensorflow) still do not support NumPy 2.0. **Description of the Change:** We can distinguish the changes into 3 main categories: *Changes to workflows* - None in the final version *Changes to requirements and setup files* - Unpin the Numpy version in `setup.py` (now we also allow Numpy 2.0). - Update `requirements-ci.txt` to include Scipy 1.13 (this adds support for Numpy 2.0). - Pin Numpy in `requirements-ci.txt` to 2.0. *Changes to the source code* - Change `np.NaN` to `np.nan`. - Use legacy printing representation in tests, contrary to the new numpy representation of scalars, e.g. np.float64(3.0) rather than just 3.0. - Update probabilities warning to be case insensitive and check for a partial match, since this warning was changed in Numpy 2.0. - Check the datatype of np.exp from the Global phase only for Numpy 1.x, since this gets promoted to complex128 in Numpy 2.x. https://numpy.org/neps/nep-0050-scalar-promotion.html#schema-of-the-new-proposed-promotion-rules. **Benefits:** Make Pennylane compatible with Numpy 2.0. **Possible Drawbacks:** - We need to create a separate workflow to keep testing PennyLane with NumPy 1.x, since we still want to maintain compatibility with previous NumPy versions. This will be done in a separate PR. - We are not testing Numpy 2.x for the interfaces that implicitly require Numpy 1.x. These currently seem to be `tensorflow` and `openfermionpyscf` (notice that `tensorflow` is required in some code sections like qcut). In particular, `openfermionpyscf` causes an error: ``` AttributeError: np.string_ was removed in the NumPy 2.0 release. Use np.bytes_ instead. ``` in the qchem tests. The attribute `np.string_` is not used in the PL source code, so it is a problem with the package itself. [sc-61399] [sc-66548] --------- Co-authored-by: PietropaoloFrisoni <pietropaolo.frisoni@xanadu.ai> Co-authored-by: Pietropaolo Frisoni <pietropfrisoni@gmail.com>

**Context:** As part of the effort to make PL compatible with Numpy 2.0 (see #6061), we need to upgrade JAX to 0.4.26+ since such a version introduced the support for Numpy 2.0. We opted for JAX 0.4.28 since it is the same version used by Catalyst. **Description of the Change:** As above. **Benefits:** PL is compatible with Numpy 2.0 and Jax 0.4.28. **Possible Drawbacks:** - From JAX 0.4.27, in `jax.jit`, passing invalid static_argnums or static_argnames now leads to an error rather than a warning. In PL, this breaks every test where we set `shots` in the `QNode` call with `static_argnames=["shots"]`. At this stage, we decided to mark such tests with `pytest.xfail` to allow the upgrade. **Related GitHub Issues:** None. **Related Shortcut Stories**: [sc-61389] --------- Co-authored-by: dwierichs <david.wierichs@xanadu.ai>

**Context:** After [Pennylane support numpy 2.0](PennyLaneAI/pennylane#6061), catalyst needs to add this support. **Description of the Change:** 1. Bump pennylane version to include the numpy 2.0 support 2. Since programs built using numpy 1.x would not readily work with numpy 2.0, we upgraded reuirements.txt to install numpy2.0. The reason is that wrapper.cpp uses the numpy C-API that should be built using numpy 2.x. 3. pyblind11 dependency was upgraded to 2.12.0 which adds support to numpy 2.0 4. scipy dependency includes 1.13 which adds support for numpy 2.0 5. removed optax dependancy from requirements (it was a TODO and unrelated to numpy but would cause CI delay) **Benefits:** Adds support for numpy 2.0 **Related GitHub Issues:** [sc-61401]

**Context:** In #6061, we started using by default NumPy 2 (instead of NumPy 1) on the CI to test every PR targeting the master branch. Some jobs (for example, those requiring `tensorflow`) still automatically downgrade to NumPy 1.26 since not all interfaces are supported at the current stage. This PR aims to create another workflow to test some jobs with NumPy 1.26 (those that don't automatically downgrade to NumPy 1 during the day). Such a workflow should run at night to ensure PennyLane is still compatible with NumPy 1.26. **We'll keep this workflow only temporarily** (probably a quarter, but this still has to be established). After that, we'll get rid of it. Finally, It has been verified (see [here](https://github.com/PennyLaneAI/pennylane/actions/runs/10927439038)) that the tests introduced with this PR are green. This has been verified a second time (see [here](https://github.com/PennyLaneAI/pennylane/actions/runs/10946347640) before merging this PR into master). **Description of the Change:** As above. **Benefits:** We ensure that PennyLane is still compatible with NumPy 1.26. **Possible Drawbacks:** - This issue was discovered accidentally (and potentially fixed) during this PR. The `TestSinglePrecision` class in the `test_jax_jit_qnode.py` file contains tests that change the JAX precision with `jax.config.update("jax_enable_x64", False)`. When running the jax tests using multiple workers, we can face race conditions if some workers change jax configuration, because other workers in other test classes downgrade to a single precision causing numerous failures. To avoid this, we add [--dist=loadscope](https://pytest-xdist.readthedocs.io/en/latest/distribution.html) as additional pytest arg for jax tests, grouping tests by module for test functions and by class for test methods. - The new workflow is it is to a large extent a (very simplified) copy of `interface-unit-tests.yml`. This is against the principle of avoiding code duplication whenever possible. Still, this is just a temporary workflow that lasts for a few months at most (and runs only at night), and it may be convenient to disentangle it as much as possible from the usual test interface. **Related GitHub Issues:** None. **Related Shortcut Stories:** [sc-73744]

**Context:** We want to make Pennylane compatible with Numpy 2.0. After several discussions, we decided to test NumPy 2.0 on the CI by default in every PR (testing both Python versions would have been to slow). Some jobs still downgrade automatically to Numpy 1.x, since some interfaces (such as Tensorflow) still do not support NumPy 2.0. **Description of the Change:** We can distinguish the changes into 3 main categories: *Changes to workflows* - None in the final version *Changes to requirements and setup files* - Unpin the Numpy version in `setup.py` (now we also allow Numpy 2.0). - Update `requirements-ci.txt` to include Scipy 1.13 (this adds support for Numpy 2.0). - Pin Numpy in `requirements-ci.txt` to 2.0. *Changes to the source code* - Change `np.NaN` to `np.nan`. - Use legacy printing representation in tests, contrary to the new numpy representation of scalars, e.g. np.float64(3.0) rather than just 3.0. - Update probabilities warning to be case insensitive and check for a partial match, since this warning was changed in Numpy 2.0. - Check the datatype of np.exp from the Global phase only for Numpy 1.x, since this gets promoted to complex128 in Numpy 2.x. https://numpy.org/neps/nep-0050-scalar-promotion.html#schema-of-the-new-proposed-promotion-rules. **Benefits:** Make Pennylane compatible with Numpy 2.0. **Possible Drawbacks:** - We need to create a separate workflow to keep testing PennyLane with NumPy 1.x, since we still want to maintain compatibility with previous NumPy versions. This will be done in a separate PR. - We are not testing Numpy 2.x for the interfaces that implicitly require Numpy 1.x. These currently seem to be `tensorflow` and `openfermionpyscf` (notice that `tensorflow` is required in some code sections like qcut). In particular, `openfermionpyscf` causes an error: ``` AttributeError: np.string_ was removed in the NumPy 2.0 release. Use np.bytes_ instead. ``` in the qchem tests. The attribute `np.string_` is not used in the PL source code, so it is a problem with the package itself. [sc-61399] [sc-66548] --------- Co-authored-by: PietropaoloFrisoni <pietropaolo.frisoni@xanadu.ai> Co-authored-by: Pietropaolo Frisoni <pietropfrisoni@gmail.com>

**Context:** As part of the effort to make PL compatible with Numpy 2.0 (see #6061), we need to upgrade JAX to 0.4.26+ since such a version introduced the support for Numpy 2.0. We opted for JAX 0.4.28 since it is the same version used by Catalyst. **Description of the Change:** As above. **Benefits:** PL is compatible with Numpy 2.0 and Jax 0.4.28. **Possible Drawbacks:** - From JAX 0.4.27, in `jax.jit`, passing invalid static_argnums or static_argnames now leads to an error rather than a warning. In PL, this breaks every test where we set `shots` in the `QNode` call with `static_argnames=["shots"]`. At this stage, we decided to mark such tests with `pytest.xfail` to allow the upgrade. **Related GitHub Issues:** None. **Related Shortcut Stories**: [sc-61389] --------- Co-authored-by: dwierichs <david.wierichs@xanadu.ai>

**Context:** In #6061, we started using by default NumPy 2 (instead of NumPy 1) on the CI to test every PR targeting the master branch. Some jobs (for example, those requiring `tensorflow`) still automatically downgrade to NumPy 1.26 since not all interfaces are supported at the current stage. This PR aims to create another workflow to test some jobs with NumPy 1.26 (those that don't automatically downgrade to NumPy 1 during the day). Such a workflow should run at night to ensure PennyLane is still compatible with NumPy 1.26. **We'll keep this workflow only temporarily** (probably a quarter, but this still has to be established). After that, we'll get rid of it. Finally, It has been verified (see [here](https://github.com/PennyLaneAI/pennylane/actions/runs/10927439038)) that the tests introduced with this PR are green. This has been verified a second time (see [here](https://github.com/PennyLaneAI/pennylane/actions/runs/10946347640) before merging this PR into master). **Description of the Change:** As above. **Benefits:** We ensure that PennyLane is still compatible with NumPy 1.26. **Possible Drawbacks:** - This issue was discovered accidentally (and potentially fixed) during this PR. The `TestSinglePrecision` class in the `test_jax_jit_qnode.py` file contains tests that change the JAX precision with `jax.config.update("jax_enable_x64", False)`. When running the jax tests using multiple workers, we can face race conditions if some workers change jax configuration, because other workers in other test classes downgrade to a single precision causing numerous failures. To avoid this, we add [--dist=loadscope](https://pytest-xdist.readthedocs.io/en/latest/distribution.html) as additional pytest arg for jax tests, grouping tests by module for test functions and by class for test methods. - The new workflow is it is to a large extent a (very simplified) copy of `interface-unit-tests.yml`. This is against the principle of avoiding code duplication whenever possible. Still, this is just a temporary workflow that lasts for a few months at most (and runs only at night), and it may be convenient to disentangle it as much as possible from the usual test interface. **Related GitHub Issues:** None. **Related Shortcut Stories:** [sc-73744]

bring changes from prep_for_np2 branch

c471f5b

EmilianoG-byte and others added 5 commits August 2, 2024 15:12

change from rc to released version

c3e011e

Merge branch 'master' into compatible-np-2.0

6ffb5e5

add scipy<= 1.13 on ci requirements to become np 2.0 compatible

c8b7d91

pin autograd to major 1.7.0 version

e54eeef

Merge branch 'master' into compatible-np-2.0

0c0d5a6

EmilianoG-byte marked this pull request as ready for review August 27, 2024 15:04

EmilianoG-byte added 11 commits August 27, 2024 13:56

set numpy print options to legacy for scalars and change np.NaN to np…

b3ab335

….nan

Revert "set numpy print options to legacy for scalars and change np.N…

ef47609

…aN to np.nan" This reverts commit b3ab335. Reverting changes from numpy formating and NAN

change NaN for nan

9a88660

revert changes from numpy printing settings

increase version of scipy used with jax

f523d32

attempt to use the legacy printing option

change to use context manager for legacy and check against older vers…

236b3f5

…ion 1.21

add legacy print option context manager to torch and prepselprep tests

d94d30c

add legacy printing solution to data-tests

2d8db14

change regex for probabilities not adding 1 from numpy

f9322a7

use legacy context manager on default qubit legacy

50711df

Merge branch 'master' into compatible-np-2.0

e0bcd0a

check numpy version to check dtype in test_apply_global_phase

1793a03

astralcai reviewed Aug 28, 2024

View reviewed changes

.github/workflows/install_deps/action.yml Outdated Show resolved Hide resolved

pennylane/numpy/random.py Outdated Show resolved Hide resolved

astralcai reviewed Aug 28, 2024

View reviewed changes

tests/data/attributes/operator/test_operator.py Outdated Show resolved Hide resolved

EmilianoG-byte changed the title ~~np.sign is compatible with PennyLane~~ Pennylane is compatible with numpy 2.0 Aug 28, 2024

Alex-Preciado requested review from mudit2812 and albi3ro August 29, 2024 16:34

mlxd reviewed Aug 29, 2024

View reviewed changes

move context manager to only affect asserts related to representation

d55a614

mudit2812 reviewed Aug 29, 2024

View reviewed changes

EmilianoG-byte and others added 2 commits August 29, 2024 17:33

ping numpy in requirements

e728c25

Resolving CodeFactor issues

f1ff64e

PietropaoloFrisoni added 2 commits September 12, 2024 13:22

Removing empty spaces (causing issues with CI)

6ce96cd

changing number of runners

0f6ec70

PietropaoloFrisoni added the ci:run-full-test-suite Run the full test-suite on the pull request label Sep 12, 2024

Running full test CI suite

eca800f

PietropaoloFrisoni mentioned this pull request Sep 13, 2024

PennyLane is compatible with JAX 0.4.28 #6255

Merged

PietropaoloFrisoni removed the ci:run-full-test-suite Run the full test-suite on the pull request label Sep 13, 2024

PietropaoloFrisoni and others added 5 commits September 13, 2024 12:26

Increasing core runners

9e48ecc

Merge branch 'master' of https://github.com/PennyLaneAI/pennylane int…

7964bc9

…o compatible-np-2.0

Merge branch 'master' of https://github.com/PennyLaneAI/pennylane int…

8b09bc3

…o compatible-np-2.0

Removing double Numpy tests

3604c45

Updating changelog

3f9567e

PietropaoloFrisoni approved these changes Sep 16, 2024

View reviewed changes

Removing pin in requirements-ci.txt (not useful since the pin <=2.0…

5aa49f8

… is already in `setup.py`

PietropaoloFrisoni enabled auto-merge (squash) September 16, 2024 12:47

Merge branch 'master' of https://github.com/PennyLaneAI/pennylane int…

591671d

…o compatible-np-2.0

PietropaoloFrisoni merged commit 94f067a into master Sep 16, 2024
36 checks passed

PietropaoloFrisoni deleted the compatible-np-2.0 branch September 16, 2024 13:08

PietropaoloFrisoni mentioned this pull request Sep 17, 2024

Create a workflow to test PennyLane with NumPy 1.26 #6275

Merged

mehrdad2m mentioned this pull request Sep 18, 2024

catalyst works with numpy 2.0 PennyLaneAI/catalyst#1119

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pennylane is compatible with numpy 2.0 #6061

Pennylane is compatible with numpy 2.0 #6061

EmilianoG-byte commented Aug 1, 2024 •

edited by PietropaoloFrisoni

Loading

github-actions bot commented Aug 1, 2024

codecov bot commented Aug 28, 2024 •

edited

Loading

astralcai left a comment

mlxd left a comment

mudit2812 left a comment

PietropaoloFrisoni commented Sep 13, 2024

Pennylane is compatible with numpy 2.0 #6061

Pennylane is compatible with numpy 2.0 #6061

Conversation

EmilianoG-byte commented Aug 1, 2024 • edited by PietropaoloFrisoni Loading

github-actions bot commented Aug 1, 2024

codecov bot commented Aug 28, 2024 • edited Loading

Codecov Report

astralcai left a comment

Choose a reason for hiding this comment

mlxd left a comment

Choose a reason for hiding this comment

mudit2812 left a comment

Choose a reason for hiding this comment

PietropaoloFrisoni commented Sep 13, 2024

EmilianoG-byte commented Aug 1, 2024 •

edited by PietropaoloFrisoni

Loading

codecov bot commented Aug 28, 2024 •

edited

Loading