Installation not aborted when Requires-Python does not match the running version

[zzh1996]

Environment

• pip version: 18.1
• Python version: 2.7
• OS: ubuntu 18.04 (docker)

Description
After merging #3846, package installation should be aborted if Requires-Python do not match the running Python version. But this feature is not working.

Expected behavior
Package installation should stop before setup.py is run.

How to Reproduce
On a fresh installed ubuntu 18.04 (Here I use docker run -it ubuntu:18.04 bash)
Run the following commands

apt update && apt install python-pip
python -m pip install -U pip
python -m pip install https://files.pythonhosted.org/packages/6a/f5/d8f543ce16d9748a2cff663f23b5c1a6d7b844cb341e3343e4bbe5a9c620/angr-8.18.10.25.tar.gz

Why do this?

Yes, python -m pip install angr works without any error. This is because data-requires-python attribute is provided on the official PyPI website according to PEP503. Python 2.7 will ignore the versions that doesn't satisfy the requirement and finally install angr-7.8.9.26 (the correct version for Python 2.7).

Unfortunately, according to PEP503,

A repository MAY include a data-requires-python attribute on a file link.

There are many mirrors of PyPI which do not support data-requires-python attribute, such as https://mirrors.ustc.edu.cn/pypi/web/. When you install PyPI packages from these mirrors, you will download package versions which are in conflict with the running Python version (eg. the URL I provided above). Some packages will detect it when installing and stop immediately, but others will succeed in installing and just throw something like SyntaxError: invalid syntax only when you import it. pip should ensure that a package should not start installing if Requires-Python do not match the running Python version.

Output

root@267436d93d0c:/# python -V
Python 2.7.15rc1

root@267436d93d0c:/# python -m pip -V
pip 18.1 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)

root@267436d93d0c:/# python -m pip install https://files.pythonhosted.org/packages/6a/f5/d8f543ce16d9748a2cff663f23b5c1a6d7b844cb341e3343e4bbe5a9c620/angr-8.18.10.25.tar.gz
Collecting https://files.pythonhosted.org/packages/6a/f5/d8f543ce16d9748a2cff663f23b5c1a6d7b844cb341e3343e4bbe5a9c620/angr-8.18.10.25.tar.gz
  Using cached https://files.pythonhosted.org/packages/6a/f5/d8f543ce16d9748a2cff663f23b5c1a6d7b844cb341e3343e4bbe5a9c620/angr-8.18.10.25.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-req-build-4N0yQX/setup.py", line 27, in <module>
        """)
    Exception:
    
    =-=-=-=-=-=-=-=-=-=-=-=-=  WELCOME TO THE FUTURE!  =-=-=-=-=-=-=-=-=-=-=-=-=-=
    
    angr has transitioned to python 3. Due to the small size of the team behind it,
    we can't reasonably maintain compatibility between both python 2 and python 3.
    If you want to continue using the most recent version of angr (you definitely
    want that, trust us) you should upgrade to python 3. It's like getting your
    vaccinations. It hurts a little bit initially but in the end it's worth it.
    
    If you are staying on python 2 and would like to make sure you don't get
    incompatible versions, make sure your pip is at least version 9.0, and it will
    use our metadata to implicitly avoid them.
    
    For more information, see here: https://docs.angr.io/MIGRATION.html
    
    Good luck!
    
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-4N0yQX/

root@267436d93d0c:/# wget https://files.pythonhosted.org/packages/6a/f5/d8f543ce16d9748a2cff663f23b5c1a6d7b844cb341e3343e4bbe5a9c620/angr-8.18.10.25.tar.gz && tar xf angr-8.18.10.25.tar.gz && cat angr-8.18.10.25/PKG-INFO 
Metadata-Version: 1.2
Name: angr
Version: 8.18.10.25
Summary: A multi-architecture binary analysis toolkit, with the ability to perform dynamic symbolic execution and various static analyses on binaries
Home-page: https://github.com/angr/angr
License: UNKNOWN
Description: UNKNOWN
Platform: UNKNOWN
Requires-Python: >=3.5

[benoit-pierre]

Note that pip needs to "prepare" a source distribution to read its metadata (e.g. run setup.py egg_info), meaning it cannot check requires-python before that.

[benoit-pierre]

To elaborate: PKG-INFO and/or existing egg-info metadata is not used, because it might not be present (pip support installing from VCS snapshots, not just proper source distributions).

[RonnyPfannschmidt]

main question is should pip consider it if available (as pretty much all setuptools based sdists supply it in a reasonable fashion)

[pradyunsg]

python -m pip install https://files.pythonhosted.org/packages/6a/f5/d8f543ce16d9748a2cff663f23b5c1a6d7b844cb341e3343e4bbe5a9c620/angr-8.18.10.25.tar.gz

This isn't installing from a PEP 503 repository -- this is an install from a source tarball URL. As noted above, pip has no way to know the metadata of that distribution, without running setup.py egg_info.

Attempting pip install angr==8.18.0.25 --no-binary angr correctly deems this version as not-compatible, as indicated by the data-requires-python=">=3.5" on the relevant anchor tag for this distribution.

<a href="https://files.pythonhosted.org/packages/6f/9a/e7dc9923c63c993c6a6e7ceae6f3f82d93e68fc8e0f2cc2fb442c1e68006/angr-8.18.10.5.tar.gz#sha256=5a614728f4e5d2c74101c43a7a78ce0ad4a125b2e722d209ec1774fb9448235c" data-requires-python=">=3.5">angr-8.18.10.5.tar.gz</a>

[no-response]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.

[uranusjr]

Is it the intention to close this? I do feel pip should abort the installation by default instead of breaking the environment silently.

[webknjaz]

@uranusjr it does abort the installation. In fact, it explodes during the build from sdists. The problem is that it needs to build sdist in order to figure out python-requires if there's no wheel.

[uranusjr]

Got it, thanks for the clarification.

[pfmoore]

So there are a number of points here:

1. data-requires-python works fine, if available. The OP was using a direct URL, which doesn't have access to that data.
2. Requires-Python works if available, although current pip will typically install and then report the problem. The new resolver will skip the candidate, giving an experience more like data-requires-python.
3. Pip doesn't use metadata from a sdist file. This is an issue under active discussion - see here. If we could trust (this part of) sdist metadata, then using it is an option, although it would be an enhancement.
4. For projects whose sdist can't even build under the current Python version, we have no way of getting metadata. The best we could possibly do would be to improve the error message.

Keeping this issue open (particularly as the OP hasn't responded) doesn't seem to add much value here.