Tests failing to run with BackendUnavailable since pip 19

[jaraco]

It seems that since the release of pip 19, Setuptools' own tests fail to run thus:

setuptools fix_889_and_non-ascii_in_setup.cfg_take_2 $ tox -e py27                                                                                                                       
py27 create: /Users/jaraco/code/main/setuptools/.tox/py27
py27 installdeps: -rtests/requirements.txt
py27 develop-inst: /Users/jaraco/code/main/setuptools
ERROR: invocation failed (exit code 2), logfile: /Users/jaraco/code/main/setuptools/.tox/py27/log/py27-2.log
ERROR: actionid: py27
msg: developpkg
cmdargs: '/Users/jaraco/code/main/setuptools/.tox/py27/bin/pip install --exists-action w -e /Users/jaraco/code/main/setuptools'

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Obtaining file:///Users/jaraco/code/main/setuptools
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
Exception:
Traceback (most recent call last):
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 315, in run
    resolver.resolve(requirement_set)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/resolve.py", line 131, in resolve
    self._resolve_one(requirement_set, req)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/resolve.py", line 294, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/resolve.py", line 226, in _get_abstract_dist_for
    req, self.require_hashes, self.use_user_site, self.finder,
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 382, in prepare_editable_requirement
    abstract_dist.prep_for_dist(finder, self.build_isolation)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 149, in prep_for_dist
    reqs = self.req.pep517_backend.get_requires_for_build_wheel()
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_vendor/pep517/wrappers.py", line 71, in get_requires_for_build_wheel
    'config_settings': config_settings
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_vendor/pep517/wrappers.py", line 162, in _call_hook
    raise BackendUnavailable
BackendUnavailable

py27 installed: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.,apipkg==1.5,atomicwrites==1.2.1,attrs==18.2.0,configparser==3.7.1,contextlib2==0.5.5,coverage==4.5.2,enum34==1.1.6,execnet==1.5.0,flake8==3.6.0,funcsigs==1.0.2,futures==3.2.0,importlib-metadata==0.8,mccabe==0.6.1,mock==2.0.0,more-itertools==5.0.0,path.py==11.5.0,pathlib2==2.3.3,pbr==5.1.1,pluggy==0.8.1,py==1.7.0,pycodestyle==2.4.0,pyflakes==2.0.0,pytest==3.10.1,pytest-cov==2.6.1,pytest-fixture-config==1.4.0,pytest-flake8==1.0.3,pytest-shutil==1.4.0,pytest-virtualenv==1.4.0,scandir==1.9.0,six==1.12.0,termcolor==1.1.0,virtualenv==16.3.0,zipp==0.3.3
________________________________________________________________________________________ summary ________________________________________________________________________________________
ERROR:   py27: InvocationError for command /Users/jaraco/code/main/setuptools/.tox/py27/bin/pip install --exists-action w -e /Users/jaraco/code/main/setuptools (see /Users/jaraco/code/main/setuptools/.tox/py27/log/py27-2.log) (exited with code 2)

I suspect this is related to #1642 if not just another manifestation of the same issue.

[jaraco]

In order to unblock this project, I was going to configure tox to downgrade pip after creating the environment, but tox only allows one install command, and this project has already used up that configuration option to work around another issue, and it's currently not possible to specify a pip version.

[jaraco]

After getting a test failure from tox, I'm able to replicate the failure outside of tox by invoking the same command tox does (or similar):

setuptools master $ .tox/py27/bin/pip install -e .                                                                                                                                       
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Obtaining file:///Users/jaraco/code/main/setuptools
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
Exception:
Traceback (most recent call last):
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 315, in run
    resolver.resolve(requirement_set)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/resolve.py", line 131, in resolve
    self._resolve_one(requirement_set, req)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/resolve.py", line 294, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/resolve.py", line 226, in _get_abstract_dist_for
    req, self.require_hashes, self.use_user_site, self.finder,
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 382, in prepare_editable_requirement
    abstract_dist.prep_for_dist(finder, self.build_isolation)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 149, in prep_for_dist
    reqs = self.req.pep517_backend.get_requires_for_build_wheel()
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_vendor/pep517/wrappers.py", line 71, in get_requires_for_build_wheel
    'config_settings': config_settings
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_vendor/pep517/wrappers.py", line 162, in _call_hook
    raise BackendUnavailable
BackendUnavailable

The problem, of course, is that the virtualenv doesn't have setuptools installed. Setuptools historically has relied on itself to build/install itself... but pep517 is breaking from the convention/expectation that '' is on sys.path.

[jaraco]

the virtualenv doesn't have setuptools installed

I'm wrong about that. The virtualenv does have setuptools installed... it just doesn't show up with pip freeze (probably due to special handling).

[jaraco]

If I update pep517._in_process to inject '' to sys.path in main(), the command no longer fails. That might also explain why python -m pep517.build . succeeds also - because in that mode of invocation, '' will be on sys.path.

[jaraco]

This is particularly difficult to debug because of the subprocess invocation. If I put a breakpoint in the _in_process script to troubleshoot, it exits immediately because there's no pipes for stdin/stdout.

[jaraco]

With lots of edits in the pip codebase, I managed to get a PDB prompt inside the _in_process, and here's what I see:

setuptools master $ .tox/py27/bin/pip install -e .                                                                                                                                       
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Obtaining file:///Users/jaraco/code/main/setuptools
  Installing build dependencies ... done
> /Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py(186)main()
-> if len(sys.argv) < 3:
(Pdb) import sys
(Pdb) sys.path
['/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_vendor/pep517', '/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-env-KYbcw6/site', '/Users/jaraco/code/main/setuptools/.tox/py27/lib/python27.zip', '/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7', '/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/plat-darwin', '/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/plat-mac', '/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/plat-mac/lib-scriptpackages', '/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/lib-tk', '/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/lib-old', '/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/lib-dynload', '/usr/local/Cellar/python@2/2.7.15_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/usr/local/Cellar/python@2/2.7.15_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/usr/local/Cellar/python@2/2.7.15_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/usr/local/Cellar/python@2/2.7.15_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/usr/local/Cellar/python@2/2.7.15_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-env-KYbcw6/overlay/lib/python2.7/site-packages', '/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-env-KYbcw6/normal/lib/python2.7/site-packages']
(Pdb) import os
(Pdb) os.getcwd()
'/Users/jaraco/code/main/setuptools'
(Pdb) import setuptools
*** ImportError: No module named setuptools

Why is setuptools not available? I suspect it has something to do with the 'overlay' and the fact that the site-packages of the virtualenv isn't in sys.path.

[jaraco]

~ $ ls /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-env-KYbcw6/normal/lib/python2.7/site-packages                                                                  
ls: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-env-KYbcw6/normal/lib/python2.7/site-packages: No such file or directory
~ $ ls /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-env-KYbcw6/overlay/lib/python2.7/site-packages                                                                 
wheel                  wheel-0.32.3.dist-info

So it seems something is creating a clean environment of dependencies in which the build can happen, containing only the dependencies declared in build-system.requires. That makes sense. As I was tracing the code, I didn't see where that happens.

[benoit-pierre]

I've used pdb-clone for debugging such cases.

The build environment code is in pip._internal.build_env.

So setuptools is a particular case, since it provides its own backend, which mean that for supporting such a use-case, the package source directory must be added to sys path, and this must be done in pip.

[benoit-pierre]

I can only reproduce the issue with Python 3 when tox-venv is not installed, which is also weird.

[jaraco]

I can only reproduce the issue with Python 3 when tox-venv is not installed, which is also weird.

That's probably a pip bug in the build isolation code on a venv.