backend-path parameter in pyproject.toml is ignored when installing from sdist

[st-pasha]

Environment

• pip version: 19.3.1
• Python version: 3.6
• OS: MacOS

Description
I'm using a custom build backend for my project, and created the following pyproject.toml file:

[build-system]
requires = ["pip >= 19.0"]
backend-path = ["."]
build-backend = "ext"

However, after building an sdist and trying to install it, the following error emerges:

$ pip install github/datatable/dist/datatable-0.10.1.tar.gz -vvv
...
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 65, in _build_backend
    obj = import_module(mod_path)
  File "/Users/pasha/py36/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'ext'

The module in question is just a single file ext.py located in the root directory of my source distribution (and I checked, the file is really there in the sdist tarball). It appears that pip fails to find it because the option backend-path is not properly handled.

Looking at the file _in_process.py that produces the error, I can see that the backend path is supposed to arrive via the PEP517_BACKEND_PATH environment variable, and that variable is set in Pep517HookCaller._call_hook() ("_vendor/pep517/wrappers.py"). After manually adding a stacktrace printout to the constructor of Pep517HookCaller, I observe that it is invoked via

  File "/Users/pasha/py36/bin/pip", line 10, in <module>
    sys.exit(main())
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_internal/main.py", line 47, in main
    return command.main(cmd_args)
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 103, in main
    return self._main(args)
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 153, in _main
    status = self.run(options, args)
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 382, in run
    resolver.resolve(requirement_set)
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 201, in resolve
    self._resolve_one(requirement_set, req)
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 365, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 313, in _get_abstract_dist_for
    req, self.session, self.finder, self.require_hashes
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 224, in prepare_linked_requirement
    req, self.req_tracker, finder, self.build_isolation,
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 49, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_internal/distributions/source/legacy.py", line 34, in prepare_distribution_metadata
    self.req.load_pyproject_toml()
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 549, in load_pyproject_toml
    self.unpacked_source_directory, backend
  File "/Users/pasha/py36/lib/python3.6/site-packages/pip/_vendor/pep517/wrappers.py", line 121, in __init__
    traceback.print_stack(file=sys.stdout)

Thus, the problem can be traced to file "_internal/req/req_install.py" lines 548-550:

        self.pep517_backend = Pep517HookCaller(
            self.unpacked_source_directory, backend
        )

as you can see, this invocation of the constructor passes only the build_backend variable, but not the backend_path, which causes the build process to fail later on.

Expected behavior
Pip should be able to instal from sdists PEP-517 compliant packages that use in-source-tree build backends.

[uranusjr]

#6599

[xavfernandez]

Hello @st-pasha,

this should have been fixed in #7394: it should work on pip master and will be included in pip 20 planned to be released in January 2020.

Could you check ? (pip install https://github.com/pypa/pip/archive/master.zip)

[st-pasha]

Thanks @xavfernandez the installation worked after pip install git+https://github.com/pypa/pip.