provide binary wheels for Linux compatible with lxml

[nijel]

I know this error has been added intentionally to catch other breakages, but is there a way to provide binary wheels that will work with current lxml? So that pip install lxml xmlsec will bring working combo.

Presently, it doesn't:

(.venv) $ pip install lxml xmlsec
Collecting lxml
  Using cached lxml-5.2.1-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (3.4 kB)
Collecting xmlsec
  Downloading xmlsec-1.3.14.tar.gz (68 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 68.9/68.9 kB 549.7 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Using cached lxml-5.2.1-cp311-cp311-manylinux_2_28_x86_64.whl (5.0 MB)
Building wheels for collected packages: xmlsec
  Building wheel for xmlsec (pyproject.toml) ... done
  Created wheel for xmlsec: filename=xmlsec-1.3.14-cp311-cp311-linux_x86_64.whl size=160772 sha256=79e2ffc317a1754761d069e1e76adf033ac7be8e4189c816411284fe4eee6d41
  Stored in directory: /home/nijel/.cache/pip/wheels/98/60/8b/bc03288ec33436feed643c9a680f5124f697a735c7cb5f0d23
Successfully built xmlsec
Installing collected packages: lxml, xmlsec
Successfully installed lxml-5.2.1 xmlsec-1.3.14
(.venv) $ python -c 'import xmlsec'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
xmlsec.InternalError: (-1, 'lxml & xmlsec libxml2 library version mismatch')

The problem is that lxml wheels are built with internal libxml while xmlsec is using system one.

Rebuilding lxml against system libxml does address the issue, but working combo out of the box would be better.

[mxamin]

@nijel The workaround solution would be to force lxml not to use the pre-compiled wheel: pip install lxml xmlsec --no-binary=lxml
We already have pre-compile wheels for linux and macos (see release note: https://github.com/xmlsec/python-xmlsec/releases/tag/1.3.14) but I didn't upload them in pypi because of this version checking that was added in the latest release.
I'm gonna note this issue for the next release, meanwhile can you provide the following information:

• OS distribution and version
• xmlsec1 library version
• libxml2 version
• python version

[nijel]

I know the workaround, the problem is that there is no way to tell this inside dependencies. Instead, you need to tell users to not use pip install <package>, but pip install --no-binary=lxml <package>, what half of them won't do and complain that things do not work then :-).

I'm on Debian 12 (Python 3.11, libxml2 2.9.14, xmlsec1 1.2.37), it happens on GitHub CI environment with Ubuntu 22.04 (any Python version installed by actions/setup-python, libxml 2.9.13, xmlsec1 1.2.33).

PS: I'm happy user of https://github.com/pypa/cibuildwheel and https://github.com/pypa/gh-action-pypi-publish to automate all the wheel builds and publishing.

[mxamin]

@nijel You're right, I put it in the todo list for the next release. Hope we can solve it better :)

[jonathangreen]

@mxamin i like the idea of providing Linux wheels for the next release as well. We are already making the wheels with cibuildwheel, so it's just a matter of distributing them.

I can work on a PR to automate attaching the wheels to the release in GitHub and uploading them to PyPI if you would like.

[mxamin]

@mxamin i like the idea of providing Linux wheels for the next release as well. We are already making the wheels with cibuildwheel, so it's just a matter of distributing them.

I can work on a PR to automate attaching the wheels to the release in GitHub and uploading them to PyPI if you would like.

I could have do it manually (simply downloading the artificats and uploading them to pypi) but I decided not to do it because of libxml2 version check. Assume that lxml wheel is build with version 2.9.x and python-xmlsec with 2.11.x, we would end up with the mismatch error which is not good. So the real problem here is to fix the lxml/python-xmlsec compatibility issue for real :)

[jonathangreen]

Making the integration with lxml safer, like you are suggesting, would be a win for sure.

Even if we do that though, I think there is still a strong argument for providing wheels for Linux. It makes things much easier for end users of the package, especially when installing inside a container where you don't necessarily have all the build tooling available.