Installing poetry project results in get pip._vendor.pep517.wrappers.BackendUnavailable

[cs01]

Environment

• pip version: pip 19.0.dev0
• Python version: Python 3.7.0
• OS: mac

Description

I have a pyproject.toml with

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

I expected the pep 517 implementation to be able to handle pip installs of the source code.

Expected behavior

Successfully install package using pyproject.toml to define a build system other than setup.py.

How to Reproduce

1. Clone latest pip, install to venv with pip install -e .
2. Then run pip install -e . or pip install . on a project without a setup.py
3. An error occurs.

Output

>> pip install -e .
Directory '.' is not installable. File 'setup.py' not found.

>> pip install .
Processing /Users/user/git/project
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
Exception:
Traceback (most recent call last):
  File "/Users/user/git/pip/src/pip/_internal/cli/base_command.py", line 154, in main
    status = self.run(options, args)
  File "/Users/user/git/pip/src/pip/_internal/commands/install.py", line 315, in run
    resolver.resolve(requirement_set)
  File "/Users/user/git/pip/src/pip/_internal/resolve.py", line 131, in resolve
    self._resolve_one(requirement_set, req)
  File "/Users/user/git/pip/src/pip/_internal/resolve.py", line 294, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/user/git/pip/src/pip/_internal/resolve.py", line 242, in _get_abstract_dist_for
    self.require_hashes
  File "/Users/user/git/pip/src/pip/_internal/operations/prepare.py", line 356, in prepare_linked_requirement
    abstract_dist.prep_for_dist(finder, self.build_isolation)
  File "/Users/user/git/pip/src/pip/_internal/operations/prepare.py", line 156, in prep_for_dist
    reqs = self.req.pep517_backend.get_requires_for_build_wheel()
  File "/Users/user/git/pip/src/pip/_vendor/pep517/wrappers.py", line 66, in get_requires_for_build_wheel
    'config_settings': config_settings
  File "/Users/user/git/pip/src/pip/_vendor/pep517/wrappers.py", line 155, in _call_hook
    raise BackendUnavailable
pip._vendor.pep517.wrappers.BackendUnavailable

poetry>=0.12 is installed:

>> pip freeze
attrs==18.2.0
CacheControl==0.12.5
cachy==0.2.0
certifi==2018.11.29
chardet==3.0.4
cleo==0.6.8
html5lib==1.0.1
idna==2.8
jsonschema==3.0.0a3
lockfile==0.12.2
msgpack==0.6.0
pastel==0.1.0
pkginfo==1.4.2
poetry==0.12.10
pylev==1.3.0
pyparsing==2.3.0
pyrsistent==0.14.8
requests==2.21.0
requests-toolbelt==0.8.0
shellingham==1.2.8
six==1.12.0
tomlkit==0.5.3
urllib3==1.24.1
webencodings==0.5.1

[uSpike]

I am having this same problem, python 3.6.4 on Ubuntu 16.04, pip @ 2a902dd

[pfmoore]

pip install -e is only supported for setuptools (PEP 517 does not support editable installs) so it's not surprising that the pip install -e case fails.

However, the pip install . case should work. It looks like the poetry backend is failing to import for some reason, to get some more detailed debugging information, could you please edit the _build_backend function in pip\_vendor\pep417\_in_process.py to add the line noted in the except ImportError clause?

def _build_backend():
    """Find and load the build backend"""
    ep = os.environ['PEP517_BUILD_BACKEND']
    mod_path, _, obj_path = ep.partition(':')
    try:
        obj = import_module(mod_path)
    except ImportError:
        import traceback; traceback.print_exc()    #### ADD THIS LINE
        raise BackendUnavailable
    if obj_path:
        for path_part in obj_path.split('.'):
            obj = getattr(obj, path_part)
    return obj

Hopefully, the traceback for the import error will give a better indication of why the backend isn't importing.

[uSpike]

@pfmoore it looks like jsonschema is naughty and uses pkg_resources without declaring it as a dependency:

  Successfully installed attrs-18.2.0 cachecontrol-0.12.5 cachy-0.2.0 certifi-2018.11.29 chardet-3.0.4 cleo-0.6.8 html5lib-1.0.1 idna-2.8 jsonschema-3.0.0a3 lockfile-0.12.2 msgpack-0.6.0 pastel-0.1.0 pkginfo-1.4.2 poetry-0.12.10 pylev-1.3.0 pyparsing-2.3.0 pyrsistent-0.14.8 requests-2.21.0 requests-toolbelt-0.8.0 shellingham-1.2.8 six-1.12.0 tomlkit-0.5.3 urllib3-1.24.1 webencodings-0.5.1
  Cleaning up...
  Cleaned build tracker '/tmp/pip-req-tracker-azk6hts9'
done
  Getting requirements to build wheel ...   Running command...
  Traceback (most recent call last):
    File "...", line 32, in _build_backend
      obj = import_module(mod_path)
    File "/...", line 126, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 994, in _gcd_import
    File "<frozen importlib._bootstrap>", line 971, in _find_and_load
    File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "<frozen importlib._bootstrap>", line 994, in _gcd_import
    File "<frozen importlib._bootstrap>", line 971, in _find_and_load
    File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 678, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "/tmp/pip-build-env-hqs1u1p6/overlay/lib/python3.6/site-packages/poetry/masonry/__init__.py", line 10, in <module>
      from .builder import Builder
    File "/tmp/pip-build-env-hqs1u1p6/overlay/lib/python3.6/site-packages/poetry/masonry/builder.py", line 1, in <module>
      from .builders import CompleteBuilder
    File "/tmp/pip-build-env-hqs1u1p6/overlay/lib/python3.6/site-packages/poetry/masonry/builders/__init__.py", line 1, in <module>
      from .complete import CompleteBuilder
    File "/tmp/pip-build-env-hqs1u1p6/overlay/lib/python3.6/site-packages/poetry/masonry/builders/complete.py", line 4, in <module>
      import poetry.poetry
    File "/tmp/pip-build-env-hqs1u1p6/overlay/lib/python3.6/site-packages/poetry/poetry.py", line 11, in <module>
      from .json import validate_object
    File "/tmp/pip-build-env-hqs1u1p6/overlay/lib/python3.6/site-packages/poetry/json/__init__.py", line 4, in <module>
      import jsonschema
    File "/tmp/pip-build-env-hqs1u1p6/overlay/lib/python3.6/site-packages/jsonschema/__init__.py", line 32, in <module>
      from pkg_resources import get_distribution
  ModuleNotFoundError: No module named 'pkg_resources'
done

If I change my pyproject.toml to include "setuptools" as a requirement, it works fine:

[build-system]
requires = ["setuptools", "poetry>=0.12"]
build-backend = "poetry.masonry.api"

It seems to me that hiding the traceback may lead to more confusion like this in the future.

[pfmoore]

See pypa/pyproject-hooks#10 - agreed, hiding the traceback detail is not ideal.

[chrahunt]

It looks like @cjerdonek has captured the need to show traceback details for build backends in #6634. Is there anything else actionable here?

[pradyunsg]

Yea, not to my knowledge.

[cjerdonek]

Just to clarify, I don't think #6634 currently covers the error mentioned in this issue. #6634 is more about errors happening in the subprocess.

[pradyunsg]

Okai. I'll reopen since if #6634 doesn't cover this, this isn't addressed.

[nm17]

Same here, I can't install aiohttp on Python 3.9.