Can't install frozenlist on Python 3.12

[atugushev]

Expected behaviour

Successful installation.

Actual behaviour

$ pip install frozenlist
...
Failed to build frozenlist
ERROR: Could not build wheels for frozenlist, which is required to install pyproject.toml-based projects

Details

$ pip install frozenlist
Collecting frozenlist
  Downloading frozenlist-1.3.3.tar.gz (66 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: frozenlist
  Building wheel for frozenlist (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for frozenlist (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [217 lines of output]
      *********************
      * Accelerated build *
      *********************
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-13.0-x86_64-cpython-312
      creating build/lib.macosx-13.0-x86_64-cpython-312/frozenlist
      copying frozenlist/__init__.py -> build/lib.macosx-13.0-x86_64-cpython-312/frozenlist
      running egg_info
      writing frozenlist.egg-info/PKG-INFO
      writing dependency_links to frozenlist.egg-info/dependency_links.txt
      writing top-level names to frozenlist.egg-info/top_level.txt
      reading manifest file 'frozenlist.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no previously-included files matching '*.pyc' found anywhere in distribution
      warning: no previously-included files matching '*.pyd' found anywhere in distribution
      warning: no previously-included files matching '*.so' found anywhere in distribution
      warning: no previously-included files matching '*.lib' found anywhere in distribution
      warning: no previously-included files matching '*.dll' found anywhere in distribution
      warning: no previously-included files matching '*.a' found anywhere in distribution
      warning: no previously-included files matching '*.obj' found anywhere in distribution
      warning: no previously-included files found matching 'frozenlist/*.html'
      no previously-included directories found matching 'docs/_build'
      adding license file 'LICENSE'
      writing manifest file 'frozenlist.egg-info/SOURCES.txt'
      copying frozenlist/__init__.pyi -> build/lib.macosx-13.0-x86_64-cpython-312/frozenlist
      copying frozenlist/_frozenlist.pyx -> build/lib.macosx-13.0-x86_64-cpython-312/frozenlist
      copying frozenlist/py.typed -> build/lib.macosx-13.0-x86_64-cpython-312/frozenlist
      running build_ext
      building 'frozenlist._frozenlist' extension
      creating build/temp.macosx-13.0-x86_64-cpython-312
      creating build/temp.macosx-13.0-x86_64-cpython-312/frozenlist
      clang -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/usr/local/opt/zlib/include -I/private/tmp/foo/.venv/include -I/Users/albert/.pyenv/versions/3.12-dev/include/python3.12 -c frozenlist/_frozenlist.c -o build/temp.macosx-13.0-x86_64-cpython-312/frozenlist/_frozenlist.o
      frozenlist/_frozenlist.c:4136:5: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
          __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_pyx_unpickle_FrozenList); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 13, __pyx_L1_error)
          ^
      frozenlist/_frozenlist.c:1300:43: note: expanded from macro '__Pyx_GetModuleGlobalName'
          (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\
                                                ^
      frozenlist/_frozenlist.c:1272:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
      #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
                                                                      ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here
          Py_DEPRECATED(3.12) uint64_t ma_version_tag;
          ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/pyport.h:318:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      frozenlist/_frozenlist.c:4182:5: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
          __Pyx_GetModuleGlobalName(__pyx_t_5, __pyx_n_s_pyx_unpickle_FrozenList); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 15, __pyx_L1_error)
          ^
      frozenlist/_frozenlist.c:1300:43: note: expanded from macro '__Pyx_GetModuleGlobalName'
          (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\
                                                ^
      frozenlist/_frozenlist.c:1272:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
      #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
                                                                      ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here
          Py_DEPRECATED(3.12) uint64_t ma_version_tag;
          ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/pyport.h:318:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      frozenlist/_frozenlist.c:5463:3: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
        __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_sys); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 8, __pyx_L1_error)
        ^
      frozenlist/_frozenlist.c:1300:43: note: expanded from macro '__Pyx_GetModuleGlobalName'
          (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\
                                                ^
      frozenlist/_frozenlist.c:1272:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
      #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
                                                                      ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here
          Py_DEPRECATED(3.12) uint64_t ma_version_tag;
          ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/pyport.h:318:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      frozenlist/_frozenlist.c:5481:5: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
          __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_types); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 9, __pyx_L1_error)
          ^
      frozenlist/_frozenlist.c:1300:43: note: expanded from macro '__Pyx_GetModuleGlobalName'
          (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\
                                                ^
      frozenlist/_frozenlist.c:1272:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
      #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
                                                                      ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here
          Py_DEPRECATED(3.12) uint64_t ma_version_tag;
          ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/pyport.h:318:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      frozenlist/_frozenlist.c:5535:3: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
        __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_MutableSequence); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 123, __pyx_L1_error)
        ^
      frozenlist/_frozenlist.c:1300:43: note: expanded from macro '__Pyx_GetModuleGlobalName'
          (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\
                                                ^
      frozenlist/_frozenlist.c:1272:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
      #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
                                                                      ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here
          Py_DEPRECATED(3.12) uint64_t ma_version_tag;
          ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/pyport.h:318:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      frozenlist/_frozenlist.c:5967:36: error: no member named 'curexc_traceback' in 'struct _ts'
              PyObject* tmp_tb = tstate->curexc_traceback;
                                 ~~~~~~  ^
      frozenlist/_frozenlist.c:5970:21: error: no member named 'curexc_traceback' in 'struct _ts'
                  tstate->curexc_traceback = tb;
                  ~~~~~~  ^
      frozenlist/_frozenlist.c:6338:53: error: no member named 'ob_digit' in 'struct _longobject'
              const digit* digits = ((PyLongObject*)op1)->ob_digit;
                                    ~~~~~~~~~~~~~~~~~~~~  ^
      frozenlist/_frozenlist.c:6623:27: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
          return likely(dict) ? __PYX_GET_DICT_VERSION(dict) : 0;
                                ^
      frozenlist/_frozenlist.c:1272:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
      #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
                                                                      ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here
          Py_DEPRECATED(3.12) uint64_t ma_version_tag;
          ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/pyport.h:318:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      frozenlist/_frozenlist.c:6635:36: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
          return (dictptr && *dictptr) ? __PYX_GET_DICT_VERSION(*dictptr) : 0;
                                         ^
      frozenlist/_frozenlist.c:1272:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
      #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
                                                                      ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here
          Py_DEPRECATED(3.12) uint64_t ma_version_tag;
          ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/pyport.h:318:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      frozenlist/_frozenlist.c:6639:56: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
          if (unlikely(!dict) || unlikely(tp_dict_version != __PYX_GET_DICT_VERSION(dict)))
                                                             ^
      frozenlist/_frozenlist.c:1272:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
      #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
                                                                      ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here
          Py_DEPRECATED(3.12) uint64_t ma_version_tag;
          ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/pyport.h:318:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      frozenlist/_frozenlist.c:6656:5: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
          __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version)
          ^
      frozenlist/_frozenlist.c:1274:21: note: expanded from macro '__PYX_UPDATE_DICT_CACHE'
          (version_var) = __PYX_GET_DICT_VERSION(dict);\
                          ^
      frozenlist/_frozenlist.c:1272:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
      #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
                                                                      ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here
          Py_DEPRECATED(3.12) uint64_t ma_version_tag;
          ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/pyport.h:318:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      frozenlist/_frozenlist.c:7047:9: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
              __PYX_PY_DICT_LOOKUP_IF_MODIFIED(
              ^
      frozenlist/_frozenlist.c:1279:16: note: expanded from macro '__PYX_PY_DICT_LOOKUP_IF_MODIFIED'
          if (likely(__PYX_GET_DICT_VERSION(DICT) == __pyx_dict_version)) {\
                     ^
      frozenlist/_frozenlist.c:1272:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
      #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
                                                                      ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here
          Py_DEPRECATED(3.12) uint64_t ma_version_tag;
          ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/pyport.h:318:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      frozenlist/_frozenlist.c:7047:9: warning: 'ma_version_tag' is deprecated [-Wdeprecated-declarations]
              __PYX_PY_DICT_LOOKUP_IF_MODIFIED(
              ^
      frozenlist/_frozenlist.c:1283:30: note: expanded from macro '__PYX_PY_DICT_LOOKUP_IF_MODIFIED'
              __pyx_dict_version = __PYX_GET_DICT_VERSION(DICT);\
                                   ^
      frozenlist/_frozenlist.c:1272:65: note: expanded from macro '__PYX_GET_DICT_VERSION'
      #define __PYX_GET_DICT_VERSION(dict)  (((PyDictObject*)(dict))->ma_version_tag)
                                                                      ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here
          Py_DEPRECATED(3.12) uint64_t ma_version_tag;
          ^
      /Users/albert/.pyenv/versions/3.12-dev/include/python3.12/pyport.h:318:54: note: expanded from macro 'Py_DEPRECATED'
      #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                           ^
      frozenlist/_frozenlist.c:7348:55: error: no member named 'ob_digit' in 'struct _longobject'
                  const digit* digits = ((PyLongObject*)x)->ob_digit;
                                        ~~~~~~~~~~~~~~~~~~  ^
      frozenlist/_frozenlist.c:7403:55: error: no member named 'ob_digit' in 'struct _longobject'
                  const digit* digits = ((PyLongObject*)x)->ob_digit;
                                        ~~~~~~~~~~~~~~~~~~  ^
      frozenlist/_frozenlist.c:7582:55: error: no member named 'ob_digit' in 'struct _longobject'
                  const digit* digits = ((PyLongObject*)x)->ob_digit;
                                        ~~~~~~~~~~~~~~~~~~  ^
      frozenlist/_frozenlist.c:7637:55: error: no member named 'ob_digit' in 'struct _longobject'
                  const digit* digits = ((PyLongObject*)x)->ob_digit;
                                        ~~~~~~~~~~~~~~~~~~  ^
      frozenlist/_frozenlist.c:8088:47: error: no member named 'ob_digit' in 'struct _longobject'
          const digit* digits = ((PyLongObject*)b)->ob_digit;
                                ~~~~~~~~~~~~~~~~~~  ^
      11 warnings and 8 errors generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for frozenlist
Failed to build frozenlist
ERROR: Could not build wheels for frozenlist, which is required to install pyproject.toml-based projects

Steps to reproduce

pip install frozenlist

Your environment

$ sw_vers
ProductName:		macOS
ProductVersion:		13.0.1
BuildVersion:		22A400

$ python -VV
Python 3.12.0a5+ (heads/main:22b8d77, Feb 23 2023, 11:25:13) [Clang 14.0.0 (clang-1400.0.29.202)]

[hugovk]

      frozenlist/_frozenlist.c:8088:47: error: no member named 'ob_digit' in 'struct _longobject'
          const digit* digits = ((PyLongObject*)b)->ob_digit;
                                ~~~~~~~~~~~~~~~~~~  ^

Looks like this is the cause: python/cpython#101292 / python/cpython#101291, new in 3.12.0a5.

It works for 3.12.04.

(See also aio-libs/aiohttp#7229, aio-libs/yarl#829 and cython/cython#5238 for similar issues.)

[mjpieters]

In principle, building with Cython 3.0.0b1 should fix this specific issue. However, there could be incompatibilities with 3.0.0 preventing a direct upgrade, and 3.0.0b1 is not yet the final release.

We could try to require 3.0.0b1 for Python 3.12 and up? And hope that Cython 3.0.0 makes it before 3.12 is final. Not a great option, TBH.

[mjpieters]

I can confirm that when using Cython 3.0.0b1 the generated C files don't show the problem, plus everything continues to work for every other Python release. We'd have to use Cython 3.0.0b1 for all Python versions as there is just the one tgz source release.

[mjpieters]

Note that the 3.12-dev build now fails again, 3.12.0a6 introduced more changes that Cython needs to adapt to:

      frozenlist/_frozenlist.c: In function ‘__Pyx_PyErr_ExceptionMatchesInState’:
      frozenlist/_frozenlist.c:7808:32: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_type’
       7808 |     PyObject *exc_type = tstate->curexc_type;
            |                                ^~
      frozenlist/_frozenlist.c: In function ‘__Pyx_ErrRestoreInState’:
      frozenlist/_frozenlist.c:7821:22: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_type’
       7821 |     tmp_type = tstate->curexc_type;
            |                      ^~
      frozenlist/_frozenlist.c:7822:23: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_value’
       7822 |     tmp_value = tstate->curexc_value;
            |                       ^~
      frozenlist/_frozenlist.c:7823:20: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_traceback’
       7823 |     tmp_tb = tstate->curexc_traceback;
            |                    ^~
      frozenlist/_frozenlist.c:7824:11: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_type’
       7824 |     tstate->curexc_type = type;
            |           ^~
      frozenlist/_frozenlist.c:7825:11: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_value’
       7825 |     tstate->curexc_value = value;
            |           ^~
      frozenlist/_frozenlist.c:7826:11: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_traceback’
       7826 |     tstate->curexc_traceback = tb;
            |           ^~
      frozenlist/_frozenlist.c: In function ‘__Pyx_ErrFetchInState’:
      frozenlist/_frozenlist.c:7832:19: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_type’
       7832 |     *type = tstate->curexc_type;
            |                   ^~
      frozenlist/_frozenlist.c:7833:20: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_value’
       7833 |     *value = tstate->curexc_value;
            |                    ^~
      frozenlist/_frozenlist.c:7834:17: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_traceback’
       7834 |     *tb = tstate->curexc_traceback;
            |                 ^~
      frozenlist/_frozenlist.c:7835:11: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_type’
       7835 |     tstate->curexc_type = 0;
            |           ^~
      frozenlist/_frozenlist.c:7836:11: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_value’
       7836 |     tstate->curexc_value = 0;
            |           ^~
      frozenlist/_frozenlist.c:7837:11: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_traceback’
       7837 |     tstate->curexc_traceback = 0;
            |           ^~
      frozenlist/_frozenlist.c: In function ‘__Pyx_Raise’:
      frozenlist/_frozenlist.c:8481:34: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_traceback’
       8481 |         PyObject* tmp_tb = tstate->curexc_traceback;
            |                                  ^~
      frozenlist/_frozenlist.c:8484:19: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘curexc_traceback’
       8484 |             tstate->curexc_traceback = tb;
            |                   ^~
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

This is cython/cython#5286

[raphaelauv]

still failing with 3.12.0b1 , @webknjaz could we reopen , thanks