Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to Build on Python 3.11.0 #81

Open
shubhamshah02 opened this issue Oct 27, 2022 · 32 comments
Open

Failed to Build on Python 3.11.0 #81

shubhamshah02 opened this issue Oct 27, 2022 · 32 comments

Comments

@shubhamshah02
Copy link

OS/Arch

$ python -c 'import platform;print(platform.uname())'

Python version

3.11.0

$ python --version

cChardet version

Latest

$ python -c 'import cchardet;print(cchardet.__version__)'

What is the problem?

Failed to build.

Expected behavior

Supposed to build.

Actual behavior

fatal error: longintrepr.h: No such file or directory

Steps to reproduce the behavior

Just update to Python 3.11 and attempt to build code results in error.

@ToasterUwU
Copy link

Same here.

      src/cchardet\_cchardet.cpp(196): fatal error C1083: Cannot open include file: 'longintrepr.h': No such file or directory
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> cchardet

@ToasterUwU
Copy link

nvm that issue mention, i made a little mistake and assumed that orjson depends on this, which it doesnt.

@lvillis
Copy link

lvillis commented Nov 2, 2022

Same here.

      src/cchardet\_cchardet.cpp(196): fatal error C1083: Cannot open include file: 'longintrepr.h': No such file or directory
      error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.32.31326\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> cchardet

@seperman
Copy link

seperman commented Nov 3, 2022

Same here

@SimplicityGuy
Copy link

There are 2 PRs #78 and #80 that will address this. @PyYoshi, can you merge and release a new build, please?

@woctezuma
Copy link

woctezuma commented Nov 19, 2022

I encounter the same issue with the cchardet package.

I have noticed it as I wanted to install ASF_IPC, which depends on two other packages:

aiohttp
cchardet

For the aiohttp package, the issue was solved in September:

@ToasterUwU
Copy link

ToasterUwU commented Nov 19, 2022

@PyYoshi Doesnt seem to come back to take care of this. Tried to contact them on all platforms I could find them on, never got any response.

Its also well known that they don't maintain this, it has been like this for literal years.

So chances for an update are very slim. I recommend everyone who uses this, to find an alternative.

@woctezuma
Copy link

woctezuma commented Nov 19, 2022

It is definitely a bit weird that the requirements mention chardet v3.0.4 when the current version is v5.0.0, which was updated for Python v3.11.

However, I think one would have to also change other requirements, notably the version of Cython.

@seperman
Copy link

seperman commented Nov 19, 2022 via email

@ToasterUwU
Copy link

@seperman There is also a process that allows to transfer Pypi projects to a new user when the old owner is inactive.

Don't know exactly how it works, but I know it exists

@seperman
Copy link

seperman commented Nov 19, 2022 via email

@ToasterUwU
Copy link

Even if it takes long, it should still be done.

Throwing it away and making a new package is not a good idea in my opinion. Since it forces everyone who depends on it to figure out there is a new one and than switch.

@seperman
Copy link

seperman commented Nov 20, 2022 via email

@ToasterUwU
Copy link

@seperman Im neither attached enough to take this project under my wings, or enough of a C nerd to even be able to maintain this.

Besides that, Pypi will not take years to do this. Definetly not.
I heard a few stories about people needing to use this option, and from what I heard, it was only a matter of months.

Multiple months is still quiet some time. But just slapping a 2 behind the name and than having the normal one still exist and look like the better option on first glance is not a better option than waiting. I agree there should be a new package for the meantime. But its crucial that the package with the most trustworthy name and most downloads and all that, is taken over as soon as its possible.

@wbarnha
Copy link

wbarnha commented Nov 25, 2022

We use this project for https://github.com/faust-streaming/faust and we're trying to migrate everything to support Python 3.11. I am willing to take a try at upgrading things in this project, in my own fork, for Python 3.11.

@seperman
Copy link

@wbarnha Cool. Please let us know once you fork it.

@seperman
Copy link

@ToasterUwU The last time @PyYoshi RIP pushed any code to this code base seems to be 2 years ago.

@wbarnha
Copy link

wbarnha commented Nov 28, 2022

I've made a fork at https://github.com/faust-streaming/cChardet and I'm going to merge the pending PRs into it.

@wbarnha
Copy link

wbarnha commented Nov 28, 2022

@seperman wheels for cchardet on Python 3.11 are now available!

pip install faust-cchardet

Sadly, wheels for Windows are not available at the moment until I figure out what's going on with https://github.com/faust-streaming/cChardet/actions/runs/3568007197.

Edit: I made a mistake and forgot to build Python 3.11 wheels. Should be fixed momentarily.

@seperman
Copy link

Awesome! Thanks.

@MathieuDuponchelle
Copy link

@wbarnha thanks for taking this on, can you open issues on your fork? I'm encountering various problems, such as when installing via a setup.py install_requires:

Searching for faust-cchardet
Reading https://pypi.org/simple/faust-cchardet/
Downloading https://files.pythonhosted.org/packages/68/18/923a6257a9bbc40d1fbbf35740f823472ca73d7ff13939be9e5c7cbdfbf5/faust-cchardet-2.1.10.tar.gz#sha256=b5f50e2fa4256a3a237f2e527e212d85130117d58e4bd1a0e729bc7920634ecb
Best match: faust-cchardet 2.1.10
Processing faust-cchardet-2.1.10.tar.gz
Writing /tmp/easy_install-xphxe8vb/faust-cchardet-2.1.10/setup.cfg
Running faust-cchardet-2.1.10/setup.py -q bdist_egg --dist-dir /tmp/easy_install-xphxe8vb/faust-cchardet-2.1.10/egg-dist-tmp-f_de48lz
warning: no files found matching '*.pxd' under directory 'src'
warning: no files found matching '*.pxi' under directory 'src'
warning: no files found matching '*.cpp' under directory 'src'
warning: no files found matching '*' under directory 'src/ext'
cc1plus: fatal error: src/ext/uchardet/src/CharDistribution.cpp: No such file or directory

And when installing manually:

$ python3 -c "import cchardet"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/meh/.local/lib/python3.10/site-packages/cchardet/__init__.py", line 1, in <module>
    from cchardet import _cchardet
ImportError: dynamic module does not define module export function (PyInit__cchardet)

OS is Linux (Fedora 36), python version 3.10.8

@wbarnha
Copy link

wbarnha commented Dec 5, 2022

I created an issues tab in my fork, sorry it wasn't already there. I'll take a look when I get the opportunity.

@MathieuDuponchelle
Copy link

Thanks a bunch @wbarnha , and yeah I think by default github doesn't create an issues tab on new repos, no reason to be sorry :)

I'll file the two issues later today.

rockstar added a commit to Watchfulio/watchful-py that referenced this issue Apr 20, 2023
`cchardet` hasn't had a commit in almost two years (8 days shy as
of this commit). [This issue](PyYoshi/cChardet#81)
is a six month old bug showing that `cchardet` doesn't support
python 3.11, which is the most recent python.

`cchardet` is fast, certainly, though the usage here is not as
performance sensitive as would warrant its use over `chardet`,
e.g. a difference of 800ms is probably not something that is worth
keeping problematic dependencies around.
Cheaterman added a commit to Cheaterman/samp-query that referenced this issue May 6, 2023
andersk added a commit to andersk/talon that referenced this issue May 11, 2023
cchardet doesn’t build on Python 3.11.0, and its maintainer seems to
be absent.  Switch to the faust-cchardet fork where this is fixed.

PyYoshi/cChardet#81

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
andersk added a commit to zulip/talon that referenced this issue May 11, 2023
cchardet doesn’t build on Python 3.11.0, and its maintainer seems to
be absent.  Switch to the faust-cchardet fork where this is fixed.

PyYoshi/cChardet#81

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
hf-kklein added a commit to Hochfrequenz/edi_energy_scraper that referenced this issue Jul 24, 2023
@ayratzay
Copy link

This update (from 2.0.4) helped me with the issue.
python -m pip install charset-normalizer==2.1.0

hf-kklein added a commit to Hochfrequenz/edi_energy_scraper that referenced this issue Sep 24, 2023
…65)

* Revert "(❌) *Temporary* drop support for Python 3.11 (#41)"

This reverts commit c2b568a.

* Add Cython >= 3.0.0 as explicit dependency

PyYoshi/cChardet#81 (comment)

* Revert "Add Cython >= 3.0.0 as explicit dependency"

This reverts commit f5727ef.

* set min version
andersk added a commit to andersk/talon that referenced this issue Oct 17, 2023
These are unused as of commit 14f106e
(mailgun#227).  cchardet doesn’t build on Python 3.11.0, and its maintainer
seems to be absent (PyYoshi/cChardet#81).

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
@lahwaacz
Copy link

One year later and this is still a problem... 😞

andersk added a commit to andersk/talon that referenced this issue Nov 13, 2023
These are unused as of commit 14f106e
(mailgun#227).  cchardet doesn’t build on Python 3.11.0, and its maintainer
seems to be absent (PyYoshi/cChardet#81).

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
AVMusorin added a commit to AVMusorin/aiochclient that referenced this issue Dec 6, 2023
Exception caused by abandoned cchardet library:

`src/cchardet/_cchardet.cpp:196:12: fatal error: longintrepr.h: No such file or directory`

See also:
* PyYoshi/cChardet#81
* PyYoshi/cChardet#81 (comment)
@Mohsen-Amiri
Copy link

cchardet for Python 3.11 requires recent version of Cython at build phase to eliminate longintrepr.h issue

I installed Cython but cannot install twint yet. Do you have any idea helps me?

@xloc
Copy link

xloc commented Dec 16, 2023

python 3.11.6 with Cython 3.0.6 works for me. Please double check your environment

@vgavro
Copy link

vgavro commented Dec 25, 2023

@xloc For same versions, still same error. python3.11.6, Cython 3.0.6
on installing from pypi!
UPD: BUT from git everything ok for some reason!

vgavro:~/w/extractnet(venv)(master+%=)> python3 -m venv ./venv
vgavro:~/w/extractnet(venv)(master+%=)> ./venv/bin/python --version
Python 3.11.6
vgavro:~/w/extractnet(venv)(master+%=)> cat /etc/*release
DISTRIB_ID="Arch"
DISTRIB_RELEASE="rolling"
DISTRIB_DESCRIPTION="Arch Linux"
NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://bugs.archlinux.org/"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo
vgavro:~/w/extractnet(venv)(master+%=)> ./venv/bin/pip install cython==3.0.6
Collecting cython==3.0.6
  Obtaining dependency information for cython==3.0.6 from https://files.pythonhosted.org/packages/4b/5b/d51fa7c329e278ab53d24ee0837a77993244d48f5c6d7c159844ca475f5b/Cython-3.0.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading Cython-3.0.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB)
Downloading Cython-3.0.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 6.3 MB/s eta 0:00:00
Installing collected packages: cython
Successfully installed cython-3.0.6
vgavro:~/w/extractnet(venv)(master+%=)> ./venv/bin/pip install cchardet
Collecting cchardet
  Using cached cchardet-2.1.7.tar.gz (653 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: cchardet
  Building wheel for cchardet (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for cchardet (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [23 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/cchardet
      copying src/cchardet/version.py -> build/lib.linux-x86_64-cpython-311/cchardet
      copying src/cchardet/__init__.py -> build/lib.linux-x86_64-cpython-311/cchardet
      running build_ext
      building 'cchardet._cchardet' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/src
      creating build/temp.linux-x86_64-cpython-311/src/cchardet
      creating build/temp.linux-x86_64-cpython-311/src/ext
      creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet
      creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet/src
      creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet/src/LangModels
      gcc -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -Isrc/ext/uchardet/src -I/home/vgavro/work/extractnet/venv/include -I/usr/include/python3.11 -c src/cchardet/_cchardet.cpp -o build/temp.linux-x86_64-cpython-311/src/cchardet/_cchardet.o
      src/cchardet/_cchardet.cpp:196:12: fatal error: longintrepr.h: No such file or directory
        196 |   #include "longintrepr.h"
            |            ^~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for cchardet
Failed to build cchardet
ERROR: Could not build wheels for cchardet, which is required to install pyproject.toml-based projects
 ./venv/bin/pip install git+https://github.com/PyYoshi/cChardet
Collecting git+https://github.com/PyYoshi/cChardet
  Cloning https://github.com/PyYoshi/cChardet to /tmp/pip-req-build-y6txjczj
  Running command git clone --filter=blob:none --quiet https://github.com/PyYoshi/cChardet /tmp/pip-req-build-y6txjczj
  Resolved https://github.com/PyYoshi/cChardet to commit bd1cc39950f60fea2773cbfc587c9395e85fe6f3
  Running command git submodule update --init --recursive -q
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: cchardet
  Building wheel for cchardet (pyproject.toml) ... done
  Created wheel for cchardet: filename=cchardet-2.1.7-cp311-cp311-linux_x86_64.whl size=196658 sha256=4944e02fb0d6d11caba507519a658aec3e56b1fe42a816bdd0699fd5d415b5d9
  Stored in directory: /tmp/pip-ephem-wheel-cache-bws5po5i/wheels/7c/70/41/0a677f278fd9d41235ddbde99ac020e0488fc3b2db384d26a5
Successfully built cchardet
Installing collected packages: cchardet
Successfully installed cchardet-2.1.7

@liquidaty
Copy link

Seems to have worked here. Yet to be fully tested:

pip3 install cchardet==2.2.0a2

@slycordinator
Copy link

I've made a fork at https://github.com/faust-streaming/cChardet and I'm going to merge the pending PRs into it.

Incidentally, since you're already running a fork of this:
#94

PyYoshi writes:
"あと、 メンテナー募集について考えたほうが良さそう"

which seems to be Japanese for "Also, I think we should think about recruiting maintainers".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests