Version 1.3.14 fails to compile

[corneauf]

While installing xmlsec 1.3.14 we encounter the following error:

#16 21.06       /tmp/pip-install-z5k61bf_/xmlsec/src/keys.c: In function ‘PyXmlSec_KeyFromEngine’:

#16 21.06       /tmp/pip-install-z5k61bf_/xmlsec/src/keys.c:218:65: error: ‘xmlSecKeyDataFormatEngine’ undeclared (first use in this function); did you mean ‘xmlSecKeyDataFormat’?

#16 21.06         218 |         key->handle = xmlSecCryptoAppKeyLoad(engine_and_key_id, xmlSecKeyDataFormatEngine, NULL, xmlSecCryptoAppGetDefaultPwdCallback(), (void*)engine_and_key_id);

#16 21.06             |                                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~

#16 21.06             |                                                                 xmlSecKeyDataFormat

Looking at the project, the xmlSecKeyDataFormatEngine does not seem to be defined anywhere. I've looked the <xmlsec/crypto.h> include and it does not contain the defined struct either.

[corneauf]

From the discussion here #284 it seems some code was copy pasted without the necessary context.

[jplrank]

I'm having the same issue too when installing on a Red Hat 8 system with Python 3.9.12. Version 1.3.13 installs just fine

pip install xmlsec==1.3.14
Collecting xmlsec==1.3.14
  Downloading xmlsec-1.3.14.tar.gz (68 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 68.9/68.9 kB 6.1 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting lxml>=3.8 (from xmlsec==1.3.14)
  Using cached lxml-5.2.1-cp39-cp39-manylinux_2_28_x86_64.whl.metadata (3.4 kB)
Using cached lxml-5.2.1-cp39-cp39-manylinux_2_28_x86_64.whl (5.0 MB)
Building wheels for collected packages: xmlsec
  Building wheel for xmlsec (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for xmlsec (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [39 lines of output]
      WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-39
      creating build/lib.linux-x86_64-cpython-39/xmlsec
      copying src/xmlsec/__init__.pyi -> build/lib.linux-x86_64-cpython-39/xmlsec
      copying src/xmlsec/constants.pyi -> build/lib.linux-x86_64-cpython-39/xmlsec
      copying src/xmlsec/template.pyi -> build/lib.linux-x86_64-cpython-39/xmlsec
      copying src/xmlsec/tree.pyi -> build/lib.linux-x86_64-cpython-39/xmlsec
      copying src/xmlsec/py.typed -> build/lib.linux-x86_64-cpython-39/xmlsec
      running build_ext
      building 'xmlsec' extension
      creating build/temp.linux-x86_64-cpython-39
      creating build/temp.linux-x86_64-cpython-39/tmp
      creating build/temp.linux-x86_64-cpython-39/tmp/pip-install-khokr2w2
      creating build/temp.linux-x86_64-cpython-39/tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b
      creating build/temp.linux-x86_64-cpython-39/tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src
      gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DXMLSEC_CRYPTO_DYNAMIC_LOADING=1 -D__XMLSEC_FUNCTION__=__func__ -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_GOST2012=1 -DXMLSEC_DL_LIBLTDL=1 -DMODULE_NAME=xmlsec -DMODULE_VERSION=1.3.14 -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libexslt -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libxml -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/extlibs -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libxslt -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/__pycache__ -I/venv/include -I/opt/python39/include/python3.9 -c /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/constants.c -o build/temp.linux-x86_64-cpython-39/tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/constants.o -g -std=c99 -fPIC -fno-strict-aliasing -Wno-error=declaration-after-statement -Werror=implicit-function-declaration -Os
      gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DXMLSEC_CRYPTO_DYNAMIC_LOADING=1 -D__XMLSEC_FUNCTION__=__func__ -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_GOST2012=1 -DXMLSEC_DL_LIBLTDL=1 -DMODULE_NAME=xmlsec -DMODULE_VERSION=1.3.14 -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libexslt -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libxml -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/extlibs -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libxslt -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/__pycache__ -I/venv/include -I/opt/python39/include/python3.9 -c /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/ds.c -o build/temp.linux-x86_64-cpython-39/tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/ds.o -g -std=c99 -fPIC -fno-strict-aliasing -Wno-error=declaration-after-statement -Werror=implicit-function-declaration -Os
      gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DXMLSEC_CRYPTO_DYNAMIC_LOADING=1 -D__XMLSEC_FUNCTION__=__func__ -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_GOST2012=1 -DXMLSEC_DL_LIBLTDL=1 -DMODULE_NAME=xmlsec -DMODULE_VERSION=1.3.14 -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libexslt -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libxml -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/extlibs -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libxslt -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/__pycache__ -I/venv/include -I/opt/python39/include/python3.9 -c /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/enc.c -o build/temp.linux-x86_64-cpython-39/tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/enc.o -g -std=c99 -fPIC -fno-strict-aliasing -Wno-error=declaration-after-statement -Werror=implicit-function-declaration -Os
      /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/enc.c: In function ‘PyXmlSec_ClearReplacedNodes’:
      /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/enc.c:207:14: warning: assignment to ‘struct LxmlElement **’ from incompatible pointer type ‘PyXmlSec_LxmlElementPtr’ {aka ‘struct LxmlElement *’} [-Wincompatible-pointer-types]
               elem = PyXmlSec_elementFactory(doc, n);
                    ^
      gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DXMLSEC_CRYPTO_DYNAMIC_LOADING=1 -D__XMLSEC_FUNCTION__=__func__ -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_GOST2012=1 -DXMLSEC_DL_LIBLTDL=1 -DMODULE_NAME=xmlsec -DMODULE_VERSION=1.3.14 -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libexslt -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libxml -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/extlibs -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libxslt -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/__pycache__ -I/venv/include -I/opt/python39/include/python3.9 -c /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/exception.c -o build/temp.linux-x86_64-cpython-39/tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/exception.o -g -std=c99 -fPIC -fno-strict-aliasing -Wno-error=declaration-after-statement -Werror=implicit-function-declaration -Os
      /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/exception.c: In function ‘PyXmlSec_ExchangeLastError’:
      /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/exception.c:76:9: warning: variable ‘r’ set but not used [-Wunused-but-set-variable]
           int r;
               ^
      gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DXMLSEC_CRYPTO_DYNAMIC_LOADING=1 -D__XMLSEC_FUNCTION__=__func__ -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_GOST2012=1 -DXMLSEC_DL_LIBLTDL=1 -DMODULE_NAME=xmlsec -DMODULE_VERSION=1.3.14 -I/usr/include/xmlsec1 -I/usr/include/libxml2 -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libexslt -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libxml -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/extlibs -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/libxslt -I/tmp/pip-build-env-8vp3i0k9/overlay/lib/python3.9/site-packages/lxml/includes/__pycache__ -I/venv/include -I/opt/python39/include/python3.9 -c /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/keys.c -o build/temp.linux-x86_64-cpython-39/tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/keys.o -g -std=c99 -fPIC -fno-strict-aliasing -Wno-error=declaration-after-statement -Werror=implicit-function-declaration -Os
      /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/keys.c: In function ‘PyXmlSec_KeyFromEngine’:
      /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/keys.c:218:65: error: ‘xmlSecKeyDataFormatEngine’ undeclared (first use in this function); did you mean ‘xmlSecKeyDataFormat’?
               key->handle = xmlSecCryptoAppKeyLoad(engine_and_key_id, xmlSecKeyDataFormatEngine, NULL, xmlSecCryptoAppGetDefaultPwdCallback(), (void*)engine_and_key_id);
                                                                       ^~~~~~~~~~~~~~~~~~~~~~~~~
                                                                       xmlSecKeyDataFormat
      /tmp/pip-install-khokr2w2/xmlsec_ba08e5441ebd4063854e30606f4e9b9b/src/keys.c:218:65: note: each undeclared identifier is reported only once for each function it appears in
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for xmlsec
Failed to build xmlsec
ERROR: Could not build wheels for xmlsec, which is required to install pyproject.toml-based projects

[fram-sse]

I can confirm that this broke my build as well. ubuntu 22.04 on GHA.

[shamanthb]

I can confirm my python project build is broken as well.
Using the docker image: python:3.11-slim-bullseye.

[mxamin]

Thanks for the update, xmlsec1 (C library) introduced breaking changes in v1.3.x which made it hard to support old and new version at the same time. I'll note this issue and hopefully we can fix it in the new release.

@corneauf @jplrank @fram-sse @shamanthb Can you provide the following information so we can debug easier:

• OS version and distribution
• xmlsec1 library version
• libxml2 version

[shamanthb]

@mxamin
Here are the details:
OS: Debian GNU/Linux 11
xmlsec1 version: 1.2.31
libxml2 version: libxml2-dev:arm64 (2.9.10+dfsg-6.7+deb11u4)

[jplrank]

@mxamin
Here are my details:

• OS version and distribution: Red Hat Enterprise Linux 8.9 (Ootpa)
• xmlsec1 library version: 1.2.25-4.el8
• libxml2 version: 2.9.7-18.el8_9