pip-10.0.0 misinterprets version restrictions in pyproject.toml

[TimoRoth]

• Pip version: 10.0.0
• Python version: 3.6.4
• Operating system: Linux x86_64

Description:

Building a wheel for pandas results in:

Could not find a version that satisfies the requirement numpy==1.9.3 (from versions: 1.11.3, 1.12.0rc2, 1.12.0, 1.12.1rc1, 1.12.1, 1.13.0rc1, 1.13.0rc2, 1.13.0, 1.13.1, 1.13.3, 1.14.0rc1, 1.14.0, 1.14.1, 1.14.2)
No matching distribution found for numpy==1.9.3

What I've run:

virtualenv work_venv
MYPIP="$PWD/work_venv/bin/pip"
#$MYPIP install "pip<10"
$MYPIP wheel --no-binary :all: -w wheel_out pandas

Commenting in the install "pip<10" fixes the issue. So this is a regression from pip-10.0.0.

Looking at the pandas repository, the numpy==1.9.3 is coming from https://github.com/pandas-dev/pandas/blob/v0.22.0/pyproject.toml
There is a version restriction to python-3.5 though, which pip seems to ignore.

[pfmoore]

Correct - this is a limitation of the current PEP 518 support in pip. See the docs for details. Pandas appears to have already started using PEP 518 features in its sources - which is questionable, as no existing installer provides full support for PEP 518 yet. But you'd have to take it up with them as to why they chose to do that.

[benoit-pierre]

@TimoRoth: could you check with #5286 (to install: pip install https://github.com/benoit-pierre/pip/archive/improve_pep518_build_requirements_handling.zip)?

[benoit-pierre]

Also, note that PEP 518 build dependencies will always be installed from wheels.

@pfmoore: maybe we should add a warning to the current implementation if the user asked for sources only and pip needs to handle a PEP 518 project.

[pfmoore]

That may be a good idea - it depends on whether source support makes it into 10.1 TBH. Not much point in putting a warning in 10.1 if we're fixing the issue in that release :-)

[TimoRoth]

@benoit-pierre

Using base prefix '/home/local/sw/python/3.6.4'
New python executable in /home/users/timo/work_venv/bin/python3.6
Also creating executable in /home/users/timo/work_venv/bin/python
Installing setuptools, pip, wheel...done.
Collecting https://github.com/benoit-pierre/pip/archive/improve_pep518_build_requirements_handling.zip
  Downloading https://github.com/benoit-pierre/pip/archive/improve_pep518_build_requirements_handling.zip (6.5MB)
    100% |████████████████████████████████| 6.5MB 1.3MB/s 
  Installing build dependencies ... done
Building wheels for collected packages: pip
  Running setup.py bdist_wheel for pip ... done
  Stored in directory: /tmp/pip-ephem-wheel-cache-0s630szj/wheels/15/9f/08/1aec7b0d64b80b8566e902b9df3915563ecbdbfa343ba61fa6
Successfully built pip
Installing collected packages: pip
  Found existing installation: pip 10.0.0
    Uninstalling pip-10.0.0:
      Successfully uninstalled pip-10.0.0
Successfully installed pip-10.0.0
/home/users/timo/work_venv/lib/python3.6/site-packages/pip/_internal/commands/wheel.py:105: UserWarning: Disabling all use of wheels due to the use of --build-options / --global-options / --install-options.
  cmdoptions.check_install_build_global(options)
Collecting pandas
  Using cached https://files.pythonhosted.org/packages/08/01/803834bc8a4e708aedebb133095a88a4dad9f45bbaf5ad777d2bea543c7e/pandas-0.22.0.tar.gz
  Installing build dependencies ... done
Collecting python-dateutil>=2 (from pandas)
  Using cached https://files.pythonhosted.org/packages/c5/39/4da7c2dbc4f023fba5fb2325febcadf0d0ce0efdc8bd12083a0f65d20653/python-dateutil-2.7.2.tar.gz
  Installing build dependencies ... done
Collecting pytz>=2011k (from pandas)
  Using cached https://files.pythonhosted.org/packages/10/76/52efda4ef98e7544321fd8d5d512e11739c1df18b0649551aeccfb1c8376/pytz-2018.4.tar.gz
Collecting numpy>=1.9.0 (from pandas)
  Using cached https://files.pythonhosted.org/packages/0b/66/86185402ee2d55865c675c06a5cfef742e39f4635a4ce1b1aefd20711c13/numpy-1.14.2.zip
Collecting six>=1.5 (from python-dateutil>=2->pandas)
  Using cached https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz
Building wheels for collected packages: pandas, python-dateutil, pytz, numpy, six
  Running setup.py bdist_wheel for pandas ... done
  Stored in directory: /home/users/timo/oggm_wheels
  Running setup.py bdist_wheel for python-dateutil ... done
  Stored in directory: /home/users/timo/oggm_wheels
  Running setup.py bdist_wheel for pytz ... done
  Stored in directory: /home/users/timo/oggm_wheels
  Running setup.py bdist_wheel for numpy ... done
  Stored in directory: /home/users/timo/oggm_wheels
  Running setup.py bdist_wheel for six ... done
  Stored in directory: /home/users/timo/oggm_wheels
Successfully built pandas python-dateutil pytz numpy six

Looking good I'd say.

Seems like pandas has already realized there is an issue and removed the file from their master:
pandas-dev/pandas@7ed1f53

[jorisvandenbossche]

Pandas appears to have already started using PEP 518 features in its sources - which is questionable, as no existing installer provides full support for PEP 518 yet. But you'd have to take it up with them as to why they chose to do that.

It's removed in the meantime (but we can't change the released versions that already have it of course).

Another suggestion would be: disable PEP 518 support by default in pip until it is better supported. That would also fix all the installation problems with the latest pip we are seeing.

[pradyunsg]

That would also fix all the installation problems with the latest pip we are seeing.

I'd rather that we improve support as we can, in pip, in the near future and make a release.

[jorisvandenbossche]

Yes, but the discussion on other issues gave me the impression that that would take some time (eg supporting not only wheels in build dependencies). And if that is a case, a pip with the support disabled by default would give less user problems until then.

[pradyunsg]

They'll be supported in the next release of pip, which should be ~start of July.

[voglster]

For those of you running into this issue and cannot find the work around until next pip/pandas release in a docker file for Alpine3.6/7 add the line:
RUN pip install cython && pyp install --no-build-isolation pandas
just before you try and install anything that requires pandas. You may also want to add --no-cache-dir to each pip call keep the image small

[benoit-pierre]

This can probably be closed as another duplicate of #5229.