Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebuild for numpy 2.0 #271

Merged

Conversation

regro-cf-autotick-bot
Copy link
Contributor

This PR has been triggered in an effort to update numpy2.

Notes and instructions for merging this PR:

  1. Please merge the PR only after the tests have passed.
  2. Feel free to push to the bot's branch to update this PR if needed.

Please note that if you close this PR we presume that the feedstock has been rebuilt, so if you are going to perform the rebuild yourself don't close this PR until the your rebuild has been merged.


Here are some more details about this specific migrator:

TL;DR: The way we build against numpy has changed as of numpy 2.0. This bot
PR has updated the recipe to account for the changes (see below for details).
The numpy 2.0 package itself is currently only available from a special release
channel (conda-forge/label/numpy_rc) and will not be available on the main
conda-forge channel until the release of numpy 2.0 GA.

The biggest change is that we no longer need to use the oldest available numpy
version at build time in order to support old numpy version at runtime - numpy
will by default use a compatible ABI for the oldest still-supported numpy versions.

Additionally, we no longer need to use {{ pin_compatible("numpy") }} as a
run requirement - this has been handled for more than two years now by a
run-export on the numpy package itself. The migrator will therefore remove
any occurrences of this.

However, by default, building against numpy 2.0 will assume that the package
is compatible with numpy 2.0, which is not necessarily the case. You should
check that the upstream package explicitly supports numpy 2.0, otherwise you
need to add a - numpy <2 run requirement until that happens (check numpy
issue 26191 for an overview of the most important packages).

Note that the numpy release candidate promises to be ABI-compatible with the
final 2.0 release. This means that building against 2.0.0rc1 produces packages
that can be published to our main channels.

If you already want to use the numpy 2.0 release candidate yourself, you can do

conda config --add channels conda-forge/label/numpy_rc

or add this channel to your .condarc file directly.

To-Dos:

  • Match run-requirements for numpy (i.e. check upstream pyproject.toml or however the project specifies numpy compatibility)
    • If upstream is not yet compatible with numpy 2.0, add numpy <2 upper bound under run:.
    • If upstream is already compatible with numpy 2.0, nothing else should be necessary in most cases.
    • If upstream requires a minimum numpy version newer than 1.19, you can add numpy >=x.y under run:.
  • Remove any remaining occurrences of {{ pin_compatible("numpy") }} that the bot may have missed.

PS. If the build does not compile anymore, this is almost certainly a sign that
the upstream project is not yet ready for numpy 2.0; do not close this PR until
a version compatible with numpy 2.0 has been released upstream and on this
feedstock (in the meantime, you can keep the bot from reopening this PR in
case of git conflicts by marking it as a draft).


If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/9233114471 - please use this URL for debugging.

TL;DR: The way we build against numpy has changed as of numpy 2.0. This bot
PR has updated the recipe to account for the changes (see below for details).
The numpy 2.0 package itself is currently only available from a special release
channel (`conda-forge/label/numpy_rc`) and will not be available on the main
`conda-forge` channel until the release of numpy 2.0 GA.

The biggest change is that we no longer need to use the oldest available numpy
version at build time in order to support old numpy version at runtime - numpy
will by default use a compatible ABI for the oldest still-supported numpy versions.

Additionally, we no longer need to use `{{ pin_compatible("numpy") }}` as a
run requirement - this has been handled for more than two years now by a
run-export on the numpy package itself. The migrator will therefore remove
any occurrences of this.

However, by default, building against numpy 2.0 will assume that the package
is compatible with numpy 2.0, which is not necessarily the case. You should
check that the upstream package explicitly supports numpy 2.0, otherwise you
need to add a `- numpy <2` run requirement until that happens (check numpy
issue 26191 for an overview of the most important packages).

Note that the numpy release candidate promises to be ABI-compatible with the
final 2.0 release. This means that building against 2.0.0rc1 produces packages
that can be published to our main channels.

If you already want to use the numpy 2.0 release candidate yourself, you can do
```
conda config --add channels conda-forge/label/numpy_rc
```
or add this channel to your `.condarc` file directly.

### To-Dos:
  * [ ] Match run-requirements for numpy (i.e. check upstream `pyproject.toml` or however the project specifies numpy compatibility)
    * If upstream is not yet compatible with numpy 2.0, add `numpy <2` upper bound under `run:`.
    * If upstream is already compatible with numpy 2.0, nothing else should be necessary in most cases.
    * If upstream requires a minimum numpy version newer than 1.19, you can add `numpy >=x.y` under `run:`.
  * [ ] Remove any remaining occurrences of `{{ pin_compatible("numpy") }}` that the bot may have missed.

PS. If the build does not compile anymore, this is almost certainly a sign that
the upstream project is not yet ready for numpy 2.0; do not close this PR until
a version compatible with numpy 2.0 has been released upstream and on this
feedstock (in the meantime, you can keep the bot from reopening this PR in
case of git conflicts by marking it as a draft).
@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@jakirkham
Copy link
Member

Raised upstream issue: gnuradio/gnuradio#7378

Converting to draft

@jakirkham jakirkham marked this pull request as draft May 31, 2024 01:04
@ryanvolz ryanvolz marked this pull request as ready for review June 6, 2024 15:27
@ryanvolz ryanvolz merged commit a4f879d into conda-forge:main Jun 6, 2024
32 checks passed
@regro-cf-autotick-bot regro-cf-autotick-bot deleted the rebuild-numpy2-0-1_he1f575 branch June 6, 2024 15:38
@jakirkham
Copy link
Member

Thanks Ryan! 🙏

@jakirkham
Copy link
Member

jakirkham commented Jun 6, 2024

It looks like the Windows builds are failing

Here is one CI job that fails:

(%BUILD_PREFIX%) %SRC_DIR%\build>cmake -G "Ninja"     -DCMAKE_INSTALL_PREFIX:PATH="%PREFIX%\Library"     -DCMAKE_PREFIX_PATH:PATH="%PREFIX%\Library"     -DCMAKE_BUILD_TYPE:STRING=Release     -DPYTHON_EXECUTABLE:PATH="%PREFIX%\python.exe"     -DBoost_NO_BOOST_CMAKE=ON     -DGR_PYTHON_DIR:PATH="%PREFIX%\Lib\site-packages"     -DENABLE_COMMON_PCH=OFF     -DENABLE_CTRLPORT_THRIFT=OFF     -DENABLE_DOXYGEN=OFF     -DENABLE_EXAMPLES=ON     -DENABLE_GNURADIO_RUNTIME=ON     -DENABLE_GR_ANALOG=ON     -DENABLE_GR_AUDIO=ON     -DENABLE_GR_BLOCKS=ON     -DENABLE_GR_BLOCKTOOL=ON     -DENABLE_GR_CHANNELS=ON     -DENABLE_GR_CTRLPORT=ON     -DENABLE_GR_DIGITAL=ON     -DENABLE_GR_DTV=ON     -DENABLE_GR_FEC=ON     -DENABLE_GR_FFT=ON     -DENABLE_GR_FILTER=ON     -DENABLE_GR_MODTOOL=ON     -DENABLE_GR_NETWORK=ON     -DENABLE_GR_QTGUI=ON     -DENABLE_GR_SOAPY=ON     -DENABLE_GR_TRELLIS=ON     -DENABLE_GR_UHD=ON     -DENABLE_GR_UTILS=ON     -DENABLE_GR_VIDEO_SDL=ON     -DENABLE_GR_VOCODER=ON     -DENABLE_GR_WAVELET=ON     -DENABLE_GR_ZEROMQ=ON     -DENABLE_GRC=ON     -DENABLE_POSTINSTALL=OFF     -DENABLE_PYTHON=ON     -DENABLE_TESTING=ON     .. 
CMake Error at D:/bld/gnuradio_1717687888310/_build_env/Library/share/cmake-3.29/Modules/CMakeDetermineCXXCompiler.cmake:48 (message):
  Could not find compiler set in environment variable CXX:

  cl.exe.

Call Stack (most recent call first):
  CMakeLists.txt:26 (project)


CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

Edit: Asking on Matrix

@ryanvolz
Copy link
Contributor

ryanvolz commented Jun 6, 2024

Yeah, that seems like a wider conda-forge problem/change since the build passed yesterday. Maybe something changed in the Windows Azure image?

@ryanvolz
Copy link
Contributor

ryanvolz commented Jun 6, 2024

Yeah, that seems like a wider conda-forge problem/change since the build passed yesterday. Maybe something changed in the Windows Azure image?

Likely the change to the included Visual Studio version as seen in the latest release: https://github.com/actions/runner-images/releases/tag/win22%2F20240603.1

@jakirkham
Copy link
Member

It looks like we are already using VS 2022 though

c_compiler: # [win]
- vs2022 # [win]
cxx_compiler: # [win]
- vs2022 # [win]

@ryanvolz
Copy link
Contributor

ryanvolz commented Jun 6, 2024

I think it's more about the version change from 17.9.34902.65 to 17.10.34928.147. The vc feedstock probably needs an update to look for the new versions, like conda-forge/vc-feedstock#76. It seems that the way that Microsoft is handling these Windows images is becoming more fragile.

@ryanvolz
Copy link
Contributor

ryanvolz commented Jun 6, 2024

Or maybe more extensive changes are needed, like what's in progress at conda-forge/vc-feedstock#79.

@jakirkham
Copy link
Member

Hmm...ok. Should we raise a new vc issue to discuss next steps?

@h-vetinari
Copy link
Member

Hmm...ok. Should we raise a new vc issue to discuss next steps?

Next steps are merging the referenced PR (I'm not a maintainer there, otherwise I'd have merged already). It's hard to time because we basically need to match our upgrade with whenever the new version gets deployed in Azure images, which we don't know in advance.

@jakirkham
Copy link
Member

Thanks Axel! 🙏

Looks like your fix is in

So have restarted the Windows builds on main for this feedstock. Let's see how that goes 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants