pip 10 wheel fails when install.user is set in the config

[anntzer]

• Pip version: 10.0.0.dev0 (94d186b)
• Python version: 3.6
• Operating system: Arch Linux

Description:

Tried to build a wheel using pyproject.toml while the install.user key is set in ~/.config/pip/pip.conf. Failed with the traceback below.

What I've run:

$ cat setup.py
from setuptools import setup
setup()

$ cat pyproject.toml
[build-system]
requires = ["setuptools", "wheel"]

$ cat ~/.config/pip/pip.conf
[install]
user = true

$ pip wheel -wdist .
Processing /tmp/testdir
  Installing build dependencies ... error
  Complete output from command /usr/bin/python -m pip install --ignore-installed --prefix /tmp/pip-build-env-x1otknjf https://pypi.python.org/packages/20/d7/04a0b689d3035143e2ff288f4b9ee4bf6ed80585cc121c90bfd85a1a8c2e/setuptools-39.0.1-py2.py3-none-any.whl#md5=ca299c7acd13a72e1171a3697f2b99bc https://pypi.python.org/packages/0c/80/16a85b47702a1f47a63c104c91abdd0a6704ee8ae3b4ce4afc49bc39f9d9/wheel-0.30.0-py2.py3-none-any.whl#md5=1d61793f816d6b60513364fe2de9c1b3:
  ERROR: Can not combine '--user' and '--prefix' as they imply different installation locations
  
  ----------------------------------------
Command "/usr/bin/python -m pip install --ignore-installed --prefix /tmp/pip-build-env-x1otknjf https://pypi.python.org/packages/20/d7/04a0b689d3035143e2ff288f4b9ee4bf6ed80585cc121c90bfd85a1a8c2e/setuptools-39.0.1-py2.py3-none-any.whl#md5=ca299c7acd13a72e1171a3697f2b99bc https://pypi.python.org/packages/0c/80/16a85b47702a1f47a63c104c91abdd0a6704ee8ae3b4ce4afc49bc39f9d9/wheel-0.30.0-py2.py3-none-any.whl#md5=1d61793f816d6b60513364fe2de9c1b3" failed with error code 1 in None

[pradyunsg]

@pypa/pip-committers I think just setting the appropriate environment variable to "" be fine as a fix for this?

[pfmoore]

I think this may be because the PEP 518 code for creating the build environment uses --prefix to decide where to install the build dependencies. So we need to force --user not to be set in that case. I'm not sure if setting the PIP_USER env var to "" has that effect, but if it does, then yes that would be a way of doing it. Or does --no-user work to force not installing in user-site regardless of the config file?

[pfmoore]

I'm looking at writing a test for this (as the first step towards putting together a fix) and I'm not sure I understand the PEP 518 build isolation process well enough. @pradyunsg can you help here?

In tests/functional/test_install.py the first test (test_without_setuptools) uninstalls setuptools, then runs an install of INITools. But my understanding of PEP 518 is that if a project doesn't have a pyproject.toml, it would be assumed to have build dependencies of (setuptools, wheel), and those would then be installed in the build environment. Clearly that's not what is going on here, as the test is succeeding (i.e., pip is failing to install because setuptools is not present).

I can probably just ignore this and make the OP's description of how to demonstrate the issue into a test (BTW, thanks @anntzer for a clear description!) but I'd still like to understand why the existing test works.

[pradyunsg]

I'm pretty sure that test would be using a source distribution. PEP 518 support and isolation is for installations that happen through wheels for now.

…

On Sun, 25 Mar 2018, 21:41 Paul Moore, ***@***.***> wrote: I'm looking at writing a test for this (as the first step towards putting together a fix) and I'm not sure I understand the PEP 518 build isolation process well enough. @pradyunsg <https://github.com/pradyunsg> can you help here? In tests/functional/test_install.py the first test ( test_without_setuptools) uninstalls setuptools, then runs an install of INITools. But my understanding of PEP 518 is that if a project doesn't have a pyproject.toml, it would be assumed to have build dependencies of (setuptools, wheel), and those would then be installed in the build environment. Clearly that's *not* what is going on here, as the test is succeeding (i.e., pip is failing to install because setuptools is not present). I can probably just ignore this and make the OP's description of how to demonstrate the issue into a test (BTW, thanks @anntzer <https://github.com/anntzer> for a clear description!) but I'd still like to understand why the existing test works. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5085 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADH7SZ9Z_ToTYhGG6DC-qJUT2aDyQyiuks5th8GygaJpZM4SvqA-> .

[pfmoore]

Oh, I thought it worked on any top-level build of wheels, and as an install from sdist builds a wheel then installs it, I thought that would trigger the build isolation stuff. Thanks for the clarification.

(This might be something we should document in more detail at some point. Not a high priority though...)

[pradyunsg]

If we build a wheel, that installation would go through the PEP 518 support code. Without looking at the test (doing an assignment), I'd guess there's something that is causing it to not build a wheel. Maybe --no-binary?

…

On Sun, 25 Mar 2018 at 22:08 Paul Moore ***@***.***> wrote: Oh, I thought it worked on any top-level build of wheels, and as an install from sdist builds a wheel then installs it, I thought that would trigger the build isolation stuff. Thanks for the clarification. (This might be something we should document in more detail at some point. Not a high priority though...) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5085 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADH7SVk7QdSJBqDbZy5-sTaAbyTK8i55ks5th8gEgaJpZM4SvqA-> .

[pfmoore]

Yeah, that's in the test code. I didn't realise that changed the build path like that. My mistake. Thanks.

[pradyunsg]

No issues. Cheers! :)

…

On Sun, 25 Mar 2018 at 22:15 Paul Moore ***@***.***> wrote: Yeah, that's in the test code. I didn't realise that changed the build path like that. My mistake. Thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5085 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADH7SfZ4ORbL1O154OStFZtygXKy5CYJks5th8m_gaJpZM4SvqA-> .

[pfmoore]

BTW, it would be much nicer if we had a --no-user option, as that would explicitly override whatever was in the config file. Not sure why that doesn't work :-( For the same reason, I don't think it's possible to set the environment variable PIP_USER to anything that would work.

I've managed to get a working patch for this by adding an explicit --no-user option that stores False in the use_user_site option value, but (a) that's ugly as sin, and (b) I dread to think what interactions might exist (--user --no-user vs --no-user --user).

I don't have time to do more on this just now, so I've created #5116 with my work so far. If anyone wants to review and/or suggest better approaches, I'd be most grateful.

[pfmoore]

Fixed via #5116