Skip to content

Conversation

@NotSqrt
Copy link

@NotSqrt NotSqrt commented Nov 18, 2025

To trigger the actions following #1447 and the new docker netcdf-manylinux images

@CLAassistant
Copy link

CLAassistant commented Nov 18, 2025

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ NotSqrt
❌ jswhit2


jswhit2 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@jswhit
Copy link
Collaborator

jswhit commented Nov 19, 2025

the plugins are working for linux, but not for windows or macos. It seems like the conda libnetcdf package for windows should be installing the plugins, but they are not being found, even if I set NETCDF_PLUGIN_DIR to point to where they should be installed. Looks like brew on macos should install at least zstd and bzip2, but netcdf4-python can't find them there either.

EDIT: looks like homebrew does not install the netcdf plugins on macos.

@ocefpaf
Copy link
Collaborator

ocefpaf commented Nov 20, 2025

the plugins are working for linux, but not for windows or macos. It seems like the conda libnetcdf package for windows should be installing the plugins, but they are not being found, even if I set NETCDF_PLUGIN_DIR to point to where they should be installed. Looks like brew on macos should install at least zstd and bzip2, but netcdf4-python can't find them there either.

EDIT: looks like homebrew does not install the netcdf plugins on macos.

I'll investigate this next week. I thought we had all the plugins in the Windows conda package.

@jswhit
Copy link
Collaborator

jswhit commented Nov 20, 2025

Updated the libnetcdf version to 4.9.3 on windows (this version has the netcdf plugins). The blosc plugin test fails with an HDF5 lib error, so I've skipped that test for now on windows. Switched to using conda for the macos wheel, and now plugins are found there also. However, the delocate step fails on macos since the libs linked by the plugins are not found.

@jswhit
Copy link
Collaborator

jswhit commented Nov 21, 2025

After converting the macos wheels to use conda installed libs, the compression plugin tests are working. The blosc test is still failing on windows though. I noticed that delvewheel was not installing the blosc dll into the wheel, so I many added it (along with zstd and lz4). That didn't help. Interestingly, the blosc test is passing in the "regular" github action tests - the only difference there is HDF5_PLUGIN_PATH is set to point to the location of the conda installed plugins.

@jswhit
Copy link
Collaborator

jswhit commented Nov 21, 2025

This h5py issue seems to be relevant for the windows blosc test failure we are seeing.

"the plugin calls back into the HDF5 API (several plugins do in order to determine chunk size and type information) then it needs to be the same instance of the HDF5 library that h5py is using - so plugins installed system wide that call into the API will not work"

This suggests that perhaps the plugin installed in the wheel is not using the same instance of the HDF5 library that netcdf4-python is using.

@jswhit
Copy link
Collaborator

jswhit commented Nov 21, 2025

@NotSqrt could you test one of the linux wheels generated here in a clean environment (outside the manylinux docker image environment) to make sure the plugins work? I can do the same for macos.

@jswhit
Copy link
Collaborator

jswhit commented Nov 22, 2025

Plugin tests pass on macos in a clean virtual environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants