[RPi] pip for Python2 fulls wheels from Python3-only piwheels.org repo

[MichaIng]

Environment

• pip version: multiple current versions (19 + 20)
• Python version: 2.7.16
• OS: Raspbian Buster

Description
We faced two cases where pip for Python2 tries to pull wheels from https://piwheels.org/ repo, which is clearly Python3-only. This of course errors out then. This was the case with setuptools some time ago, was fixed some time later, now it is the case with Markdown.

Expected behavior
Python2 pip must never pull any wheels from https://piwheels.org/.

How to Reproduce
On RPi running Raspbian Buster:

apt install python-pip python-dev
python <(curl -s https://bootstrap.pypa.io/get-pip.py)
pip install Markdown

Output

DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
...
Collecting Markdown>=2.0.1
  Downloading https://www.piwheels.org/simple/markdown/Markdown-3.2.1-py2.py3-none-any.whl (88 kB)
     |████████████████████████████████| 88 kB 800 kB/s
ERROR: Package 'Markdown' requires a different Python: 2.7.16 not in '>=3.5'

That https://www.piwheels.org/simple is inside the indexes is the core of the issue. Not sure if there is a way to add/remove indexes based on Python version?

For reference (the old and the new appearance have the same above mentioned root of failure): MichaIng/DietPi#3346

[pfmoore]

It looks like https://www.piwheels.org/simple/markdown doesn't include the data-requires-python flag, so pip doesn't know to ignore the Python 3 wheels until later, when it checks the metadata inside the wheel.

You should probably flag this to the piwheels maintainers.

[pfmoore]

For comparison:

PS C:\Work\Support> curl https://pypi.org/simple/markdown/ | grep 3.2.1
    <a href="https://files.pythonhosted.org/packages/2a/05/5d7d994904f8ea5643a69c8b37312a9e49517a261ada880dbbea70de2e3e/Markdown-2.2.1.zip#sha256=4a402dadb4053f3a0c57e3400553872a1b4ad64f90a85102392417a1bece2e3b">Markdown-2.2.1.zip</a><br/>
    <a href="https://files.pythonhosted.org/packages/ab/c4/ba46d44855e6eb1770a12edace5a165a0c6de13349f592b9036257f3c3d3/Markdown-3.2.1-py2.py3-none-any.whl#sha256=e4795399163109457d4c5af2183fbe6b60326c17cfdf25ce6e7474c6624f725d" data-requires-python="&gt;=3.5">Markdown-3.2.1-py2.py3-none-any.whl</a><br/>
    <a href="https://files.pythonhosted.org/packages/98/79/ce6984767cb9478e6818bd0994283db55c423d733cc62a88a3ffb8581e11/Markdown-3.2.1.tar.gz#sha256=90fee683eeabe1a92e149f7ba74e5ccdc81cd397bd6c516d93a8da0ef90b6902" data-requires-python="&gt;=3.5">Markdown-3.2.1.tar.gz</a><br/>
PS C:\Work\Support> curl https://www.piwheels.org/simple/markdown/ | grep 3.2.1
<a href="Markdown-3.2.1-py2.py3-none-any.whl#sha256=e4795399163109457d4c5af2183fbe6b60326c17cfdf25ce6e7474c6624f725d">Markdown-3.2.1-py2.py3-none-any.whl</a><br>

[MichaIng]

@pfmoore
Great, thanks for this info and good to know there is such a flag system to exclude incompatible wheels automatically. I'll forward this accordingly (not required, already there): piwheels/piwheels#208

Shall I close this issue as this is not a pip bug in particular, or shell it stay open as reference?

[MichaIng]

I mark this issue as closed as it is not a pip bug. The issue is known to piwheels.org maintainers and the long-term fix is planned (see linked issue). The particular Markdown wheel has been taken out from the repo.