Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"No matching distribution" due to wheel tags not matching in pip 20.0 #7626

Closed
IRDonch opened this issue Jan 21, 2020 · 38 comments · Fixed by #7643
Closed

"No matching distribution" due to wheel tags not matching in pip 20.0 #7626

IRDonch opened this issue Jan 21, 2020 · 38 comments · Fixed by #7643
Labels
auto-locked Outdated issues that have been locked by automation project: vendored dependency Related to a vendored dependency type: bug A confirmed bug or unintended behavior
Milestone

Comments

@IRDonch
Copy link

IRDonch commented Jan 21, 2020

Environment

  • pip version: 20.0.1
  • Python version: 3.6.9
  • OS: Ubuntu 18.04

Description
Pip can seemingly no longer install any version of mxnet newer than 0.9.5.

Expected behavior
It should be able to. :-) This worked before pip 20.

How to Reproduce
Try to install mxnet==1.3.1 in a virtual environment.

Output

$ virtualenv -ppython3 /tmp/venv
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /tmp/venv/bin/python3
Also creating executable in /tmp/venv/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
$ /tmp/venv/bin/pip install mxnet==1.3.1
ERROR: Could not find a version that satisfies the requirement mxnet==1.3.1 (from versions: 0.9.5)
ERROR: No matching distribution found for mxnet==1.3.1

Running pip install with --verbose produces a huge log, in which this seems relevant:

  Skipping link: none of the wheel's tags match: py2-none-manylinux1_x86_64, py3-none-manylinux1_x86_64: https://files.pythonhosted.org/packages/f0/2e/b26eb7273aed1945f59993b3b306442eb41684f931b5380821c39cf50a31/mxnet-1.3.1-py2.py3-none-manylinux1_x86_64.whl#sha256=939575fddd45e8ba39177dd3d53ccce64dea312bc08f493392b1ecace9e1b117 (from https://pypi.org/simple/mxnet/)
@triage-new-issues triage-new-issues bot added the S: needs triage Issues/PRs that need to be triaged label Jan 21, 2020
@otrejoso
Copy link

otrejoso commented Jan 21, 2020

We are also having this error using version 20.0.1 with our in-house wheel

(venv) C:\depot\bitbucket\mytests\tests_pti>pip -vvv install C:\Users\otrejoso\Downloads\pti-2.0.510-py3-none-win_amd64.whl
Non-user install because user site-packages disabled
Created temporary directory: C:\Users\otrejoso\AppData\Local\Temp\pip-ephem-wheel-cache-wquw3si6
Created temporary directory: C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Initialized build tracking at C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Created build tracker: C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Entered build tracker: C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Created temporary directory: C:\Users\otrejoso\AppData\Local\Temp\pip-install-vb0u5yy4
Cleaning up...
Removed build tracker: 'C:\\Users\\otrejoso\\AppData\\Local\\Temp\\pip-req-tracker-ik56de2r'
ERROR: pti-2.0.510-py3-none-win_amd64.whl is not a supported wheel on this platform.
Exception information:
....
pip._internal.exceptions.InstallationError: pti-2.0.510-py3-none-win_amd64.whl is not a supported wheel on this platform.

Using pip install pip==19.3.1 works fine.

@jeroendecroos
Copy link

jeroendecroos commented Jan 21, 2020

Same here with an in-house wheel.

Doesn't work:
pip install -U pip==20.0.1; pip install <wheel>
ERROR: is not a supported wheel on this platform.

Works:
pip install -U pip==19.3.1; pip install <wheel>

Seems that the platform tags are the problem here: tags 'any' are working, but this specify wheel has 'linux_x86_64'.

Note that I have:

uname -a
Linux <propretiery> 3.10.0-957.5.1.el7.x86_64 #1 SMP Fri Feb 1 14:54:57 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

 python -c "import wheel.pep425tags as w; print(w.get_supported())"
[('cp27', 'cp27mu', 'linux_x86_64'), ('cp27', 'none', 'linux_x86_64'), ('cp27', 'none', 'any'), ('cp2', 'none', 'any'), ('cp26', 'none', 'any'), ('cp25', 'none', 'any'), ('cp24', 'none', 'any'), ('cp23', 'none', 'any'), ('cp22', 'none', 'any'), ('cp21', 'none', 'any'), ('cp20', 'none', 'any'), ('py2', 'none', 'linux_x86_64'), ('py27', 'none', 'any'), ('py2', 'none', 'any'), ('py26', 'none', 'any'), ('py25', 'none', 'any'), ('py24', 'none', 'any'), ('py23', 'none', 'any'), ('py22', 'none', 'any'), ('py21', 'none', 'any'), ('py20', 'none', 'any')]

@chrisoro
Copy link

Same here.

19.3.1 works but 20.0.1 gives:
pip._internal.exceptions.InstallationError: pyenchant-2.0.0-py2.py3.cp27.cp32.cp33.cp34.cp35.cp36.pp27.pp33.pp35-none-win32.whl is not a supported wheel on this platform.

Tags for my pc: [('cp37', 'cp37m', 'win32'), ('cp37', 'none', 'win32'), ('cp37', 'none', 'any'), ('cp3', 'none', 'any'), ('cp36', 'none', 'any'), ('cp35', 'none', 'any'), ('cp34', 'none', 'any'), ('cp33', 'none', 'any'), ('cp32', 'none', 'any'), ('cp31', 'none', 'any'), ('cp30', 'none', 'any'), ('py3', 'none', 'win32'), ('py37', 'none', 'any'), ('py3', 'none', 'any'), ('py36', 'none', 'any'), ('py35', 'none', 'any'), ('py34', 'none', 'any'), ('py33', 'none', 'any'), ('py32', 'none', 'any'), ('py31', 'none', 'any'), ('py30', 'none', 'any')]

Tags for file can be seen in filename.

@xavfernandez
Copy link
Member

Could you print the differences between pip debug -v in pip 20.0.1 & pip 19.3.1 ?

@IRDonch
Copy link
Author

IRDonch commented Jan 21, 2020

--- /tmp/old.txt	2020-01-21 17:22:10.221211433 +0300
+++ /tmp/new.txt	2020-01-21 17:22:30.725552363 +0300
@@ -1,4 +1,4 @@
-pip version: pip 19.3.1 from /tmp/venv/lib/python3.6/site-packages/pip (python 3.6)
+pip version: pip 20.0.1 from /tmp/venv/lib/python3.6/site-packages/pip (python 3.6)
 sys.version: 3.6.9 (default, Nov  7 2019, 10:44:02) 
 [GCC 8.3.0]
 sys.executable: /tmp/venv/bin/python3
@@ -8,7 +8,11 @@
 sys.platform: linux
 sys.implementation:
   name: cpython
-Compatible tags: 42
+'cert' config value: global
+REQUESTS_CA_BUNDLE: None
+CURL_CA_BUNDLE: None
+pip._vendor.certifi.where(): /tmp/venv/lib/python3.6/site-packages/pip/_vendor/certifi/cacert.pem
+Compatible tags: 41
   cp36-cp36m-manylinux2014_x86_64
   cp36-cp36m-manylinux2010_x86_64
   cp36-cp36m-manylinux1_x86_64
@@ -37,12 +41,11 @@
   cp32-abi3-manylinux2010_x86_64
   cp32-abi3-manylinux1_x86_64
   cp32-abi3-linux_x86_64
-  py3-none-manylinux2014_x86_64
-  py3-none-manylinux2010_x86_64
-  py3-none-manylinux1_x86_64
-  py3-none-linux_x86_64
+  py36-none-manylinux2014_x86_64
+  py36-none-manylinux2010_x86_64
+  py36-none-manylinux1_x86_64
+  py36-none-linux_x86_64
   cp36-none-any
-  cp3-none-any
   py36-none-any
   py3-none-any
   py35-none-any

@chrisoro
Copy link

chrisoro commented Jan 21, 2020

-pip version: pip 19.3.1 from c:\sdks\python37-32\lib\site-packages\pip (python 3.7)
+pip version: pip 20.0.1 from c:\sdks\python37-32\lib\site-packages\pip (python 3.7)
 sys.version: 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 18 2019, 23:46:00) [MSC v.1916 32 bit (Intel)]
 sys.executable: c:\sdks\python37-32\python.exe
 sys.getdefaultencoding: utf-8
@@ -8,14 +8,21 @@ locale.getpreferredencoding: cp1252
 sys.platform: win32
 sys.implementation:
   name: cpython
-Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
-Config variable 'WITH_PYMALLOC' is unset, Python ABI tag may be incorrect
-Compatible tags: 14
+'cert' config value: global
+REQUESTS_CA_BUNDLE: None
+CURL_CA_BUNDLE: None
+pip._vendor.certifi.where(): c:\sdks\python37-32\lib\site-packages\pip\_vendor\certifi\cacert.pem
+Compatible tags: 19
   cp37-cp37m-win32
+  cp37-abi3-win32
   cp37-none-win32
-  py3-none-win32
+  cp36-abi3-win32
+  cp35-abi3-win32
+  cp34-abi3-win32
+  cp33-abi3-win32
+  cp32-abi3-win32
+  py37-none-win32
   cp37-none-any
-  cp3-none-any
   py37-none-any
   py3-none-any
   py36-none-any

@pfmoore
Copy link
Member

pfmoore commented Jan 21, 2020

Similar on Windows - the tag section of the output:

--- ".\\pip19.txt"      2020-01-21 14:30:16 +0000
+++ ".\\pip20.txt"      2020-01-21 14:26:54 +0000
@@ -1,9 +1,15 @@
-Compatible tags: 15
+Compatible tags: 21
   cp38-cp38-win_amd64
+  cp38-abi3-win_amd64
   cp38-none-win_amd64
-  py3-none-win_amd64
+  cp37-abi3-win_amd64
+  cp36-abi3-win_amd64
+  cp35-abi3-win_amd64
+  cp34-abi3-win_amd64
+  cp33-abi3-win_amd64
+  cp32-abi3-win_amd64
+  py38-none-win_amd64
   cp38-none-any
-  cp3-none-any
   py38-none-any
   py3-none-any
   py37-none-any

Looks like packaging.tags has different values than the version pip used internally in pip 19. The main difference is the lack of {py3,cp3}-none-win_amd64. Which isn't a standard tag generated by bdist_wheel AFAIK, so at least the impact will be limited to people who set custom tags.

The specs don't say much about what custom tags like this are valid, so this is arguably in the area of "undefined behaviour". While that doesn't help the people affected by this, it does suggest that being more specific in the standards would be good.

BTW, I'm not entirely sure what mxnet-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl even means - the MacOS releases of mxnet have a specific ABI set, why does the manylinux build not? Numpy's manylinux builds have an ABI, so it doesn't appear to be a general issue with the manylinux toolchain. The tags for pyenchant also look a bit bizarre...

@IRDonch
Copy link
Author

IRDonch commented Jan 21, 2020

the MacOS releases of mxnet have a specific ABI set, why does the manylinux build not?

I briefly checked the Linux package, and it appears that none of the native libraries there refer to Python symbols. It looks like MXNet uses ctypes for interop with native code, so not having an ABI makes sense.

@CeadeS
Copy link

CeadeS commented Jan 21, 2020

Have the same issue installing icc-rt (from intel-numpy) (2020.0.133) using pip==20.0.1

@pfmoore
Copy link
Member

pfmoore commented Jan 21, 2020

I briefly checked the Linux package, and it appears that none of the native libraries there refer to Python symbols. It looks like MXNet uses ctypes for interop with native code, so not having an ABI makes sense.

OK. Why does it need a "manylinux" tag in that case, if it's using ctypes for everything? Actually, don't spend any time on that question, I'm not a Linux expert so I probably wouldn't follow the answer anyway.

At a minimum, this sounds like it should be raised as an issue against the packaging library. Regardless of what pip does, if these are valid tags they should be supported in packaging.tags, and a general discussion of what tags should be supported is probably better had there than here.

@IRDonch
Copy link
Author

IRDonch commented Jan 21, 2020

OK. Why does it need a "manylinux" tag in that case, if it's using ctypes for everything? Actually, don't spend any time on that question, I'm not a Linux expert so I probably wouldn't follow the answer anyway.

I will answer anyway: the wheel has native Linux libraries in it, so the manylinux1 tag makes sense.

@jamadden
Copy link
Member

In #7620 (comment) @tomasaschan reported what I think is this same issue for xgboost, which ships as xgboost-0.90-py2.py3-none-manylinux1_x86_64.whl. It also appears to contain native libs, perhaps for the JVM.

@pfmoore
Copy link
Member

pfmoore commented Jan 21, 2020

@IRDonch Thanks. I did actually follow that explanation 🙂 Makes sense.

@pfmoore
Copy link
Member

pfmoore commented Jan 21, 2020

@jamadden Agreed, that looks like the same issue.

@tomasaschan
Copy link

@jamadden What can I do locally to help you determine whether this is the same?

@pfmoore
Copy link
Member

pfmoore commented Jan 21, 2020

@tomasaschan Can you paste here the output of pip debug -v?

@tomasaschan
Copy link

tomasaschan commented Jan 21, 2020

 λ diff pip19.log pip20.log 
1c1
- pip version: pip 19.3.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)
---
+ pip version: pip 20.0.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)
11c11,15
- Compatible tags: 42
---
+ 'cert' config value: global
+ REQUESTS_CA_BUNDLE: None
+ CURL_CA_BUNDLE: None
+ pip._vendor.certifi.where(): /usr/local/lib/python3.6/dist-packages/pip/_vendor/certifi/cacert.pem
+ Compatible tags: 41
40,43c44,47
-   py3-none-manylinux2014_x86_64
-   py3-none-manylinux2010_x86_64
-   py3-none-manylinux1_x86_64
-   py3-none-linux_x86_64
---
+   py36-none-manylinux2014_x86_64
+   py36-none-manylinux2010_x86_64
+   py36-none-manylinux1_x86_64
+   py36-none-linux_x86_64
45d48
-   cp3-none-any
 λ cat pip19.log 
pip version: pip 19.3.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)
sys.version: 3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0]
sys.executable: /usr/bin/python
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: UTF-8
sys.platform: linux
sys.implementation:
  name: cpython
Compatible tags: 42
  cp36-cp36m-manylinux2014_x86_64
  cp36-cp36m-manylinux2010_x86_64
  cp36-cp36m-manylinux1_x86_64
  cp36-cp36m-linux_x86_64
  cp36-abi3-manylinux2014_x86_64
  cp36-abi3-manylinux2010_x86_64
  cp36-abi3-manylinux1_x86_64
  cp36-abi3-linux_x86_64
  cp36-none-manylinux2014_x86_64
  cp36-none-manylinux2010_x86_64
  cp36-none-manylinux1_x86_64
  cp36-none-linux_x86_64
  cp35-abi3-manylinux2014_x86_64
  cp35-abi3-manylinux2010_x86_64
  cp35-abi3-manylinux1_x86_64
  cp35-abi3-linux_x86_64
  cp34-abi3-manylinux2014_x86_64
  cp34-abi3-manylinux2010_x86_64
  cp34-abi3-manylinux1_x86_64
  cp34-abi3-linux_x86_64
  cp33-abi3-manylinux2014_x86_64
  cp33-abi3-manylinux2010_x86_64
  cp33-abi3-manylinux1_x86_64
  cp33-abi3-linux_x86_64
  cp32-abi3-manylinux2014_x86_64
  cp32-abi3-manylinux2010_x86_64
  cp32-abi3-manylinux1_x86_64
  cp32-abi3-linux_x86_64
  py3-none-manylinux2014_x86_64
  py3-none-manylinux2010_x86_64
  py3-none-manylinux1_x86_64
  py3-none-linux_x86_64
  cp36-none-any
  cp3-none-any
  py36-none-any
  py3-none-any
  py35-none-any
  py34-none-any
  py33-none-any
  py32-none-any
  py31-none-any
  py30-none-any
 λ cat pip20.log 
pip version: pip 20.0.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)
sys.version: 3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0]
sys.executable: /usr/bin/python
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: UTF-8
sys.platform: linux
sys.implementation:
  name: cpython
'cert' config value: global
REQUESTS_CA_BUNDLE: None
CURL_CA_BUNDLE: None
pip._vendor.certifi.where(): /usr/local/lib/python3.6/dist-packages/pip/_vendor/certifi/cacert.pem
Compatible tags: 41
  cp36-cp36m-manylinux2014_x86_64
  cp36-cp36m-manylinux2010_x86_64
  cp36-cp36m-manylinux1_x86_64
  cp36-cp36m-linux_x86_64
  cp36-abi3-manylinux2014_x86_64
  cp36-abi3-manylinux2010_x86_64
  cp36-abi3-manylinux1_x86_64
  cp36-abi3-linux_x86_64
  cp36-none-manylinux2014_x86_64
  cp36-none-manylinux2010_x86_64
  cp36-none-manylinux1_x86_64
  cp36-none-linux_x86_64
  cp35-abi3-manylinux2014_x86_64
  cp35-abi3-manylinux2010_x86_64
  cp35-abi3-manylinux1_x86_64
  cp35-abi3-linux_x86_64
  cp34-abi3-manylinux2014_x86_64
  cp34-abi3-manylinux2010_x86_64
  cp34-abi3-manylinux1_x86_64
  cp34-abi3-linux_x86_64
  cp33-abi3-manylinux2014_x86_64
  cp33-abi3-manylinux2010_x86_64
  cp33-abi3-manylinux1_x86_64
  cp33-abi3-linux_x86_64
  cp32-abi3-manylinux2014_x86_64
  cp32-abi3-manylinux2010_x86_64
  cp32-abi3-manylinux1_x86_64
  cp32-abi3-linux_x86_64
  py36-none-manylinux2014_x86_64
  py36-none-manylinux2010_x86_64
  py36-none-manylinux1_x86_64
  py36-none-linux_x86_64
  cp36-none-any
  py36-none-any
  py3-none-any
  py35-none-any
  py34-none-any
  py33-none-any
  py32-none-any
  py31-none-any
  py30-none-any

@jeroendecroos
Copy link

jeroendecroos commented Jan 21, 2020

pip/_vendor/packaging/tags.py
332c332
-         platforms = _platform_tags
---
+         platforms = _platform_tags()
334c334
-         for platform_ in platforms():
---
+         for platform_ in platforms:

seems to solve the problem

@tomasaschan
Copy link

Here's a Dockerfile that reproduces our error message:

FROM ubuntu:bionic-20190912.1

RUN set -ex \
  && apt-get update \
  && apt-get install -y --no-install-recommends \
  ca-certificates \
  python3 python3-dev python3-pip

RUN pip3 install --upgrade pip==20.0.1 setuptools

RUN echo "xgboost==0.81" >> requirements.txt

RUN pip3 install -r requirements.txt

@pfmoore
Copy link
Member

pfmoore commented Jan 21, 2020

@jeroendecroos Good catch - that looks like it may be a straight bug in packaging.tags (re-using an iterator rather than re-creating it each time). Could you open an issue against https://github.com/pypa/packaging for this - and if you could make your fix into a PR that would be even better!

@pradyunsg
Copy link
Member

Okie, so the fix is now in master. I'll make the release in a bit -- please follow #7531.

@pradyunsg
Copy link
Member

Released 20.0.2 containing the fix for this.

If you're still seeing something similar, please take a look at #7629 (if you're on PyPy) or file a new issue. :)

@anthrotype
Copy link

This now works again with pip 20.0.2 released a few minutes ago. Thanks all for the timely patch!

@Korijn
Copy link

Korijn commented Jan 24, 2020

Thanks, we're up and running again!

@tomasaschan
Copy link

@pradyunsg I can confirm that my Docker repro above is fixed in 20.0.2.

Great work on this, huge thanks (from all of us)! ❤️

@afabiani
Copy link

There's a regression

ModuleNotFoundError: No module named 'pip._internal.download'

@pfmoore
Copy link
Member

pfmoore commented Jan 24, 2020

@afabiani can you provide a full traceback and instructions on how to reproduce, please? In a new issue, as this seems like it's unrelated to the subject of this issue.

Oh, I see you did at #7645

@afabiani
Copy link

@pfmoore #7645

@pradyunsg
Copy link
Member

Thanks! That's an unrelated problem caused by an unsupported use of pip, and not a bug/regression introduced in pip 20.0.2. I see that @pfmoore has responded in more detail there, so let's have further discussion in that issue.

@hodgestar
Copy link
Contributor

Ran into this late on Friday and arrived at work this morning to find it already fixed and released -- thank you to everyone involved in making the fix happen so quickly! :D

@eibrunorodrigues
Copy link

eibrunorodrigues commented Jan 27, 2020

Hey! This fix (20.0.2) actually didnt fix my problem. Anyone has a clue about what is causing this problem?

pip install artifacts-keyring
Looking in indexes: https://pypi.org/simple, PRIVATE_PACKAGE_REFERENCE
Collecting artifacts-keyring
Downloading artifacts_keyring-0.2.9-py2.py3-none-any.whl (4.8 MB)
|████████████████████████████████| 4.8 MB 2.5 MB/s
Requirement already satisfied: keyring>=16.0 in /usr/local/lib/python3.7/site-packages (from artifacts-keyring) (21.1.0)
Requirement already satisfied: requests>=2.20.0 in /usr/local/lib/python3.7/site-packages (from artifacts-keyring) (2.22.0)
ERROR: Could not find a version that satisfies the requirement dotnetcore2; sys_platform != "win32" and python_version >= "3.0" (from artifacts-keyring) (from ver
sions: none)
ERROR: No matching distribution found for dotnetcore2; sys_platform != "win32" and python_version >= "3.0" (from artifacts-keyring)

@pradyunsg
Copy link
Member

If you're still seeing something similar, please take a look at #7629 (if you're on PyPy) or file a new issue. :)

Please file a new issue.

@pradyunsg pradyunsg unpinned this issue Jan 28, 2020
jjfiv added a commit to jjfiv/example_rust_py that referenced this issue Feb 1, 2020
jjfiv added a commit to toybox-rs/toybox-rs that referenced this issue Feb 1, 2020
rosbo added a commit to Kaggle/docker-python that referenced this issue Feb 6, 2020
We added the pining because pip `20.0.1` introduced an issue with wheels tag matching

pip `20.0.2` has since been released and fixes the issue: 
pypa/pip#7626
@lock lock bot added the auto-locked Outdated issues that have been locked by automation label Feb 28, 2020
@lock lock bot locked as resolved and limited conversation to collaborators Feb 28, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
auto-locked Outdated issues that have been locked by automation project: vendored dependency Related to a vendored dependency type: bug A confirmed bug or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.