Specifying --use-pep517 to a non-pyproject.toml project should populate build requirements

[jaraco]

Environment

• pip version: 21
• Python version: 3.8
• OS: macOS 11

Description

Attempting to download a project that depends on termcolor~=1.1.0 fails to build the metadata for termcolor if setuptools isn't implicitly installed, even with pep517 enabled.

In order to wean myself from implicit setuptools, I've left setuptools uninstalled from my environment, and instead rely on passing --use-pep517 to invoke the "fallback to setuptools" behavior for packages not supplying pyproject.toml. Downloading just termcolor by itself works without error, but attempting to download a project that depends on termcolor (such as tensorflow) will fail to build the metadata for termcolor. It seems that the pep517 fallback to setuptools behavior doesn't take effect in the download command.

Expected behavior

Passing --use-pep517 to pip should enable the fallback to setuptools behavior (and make it available for a project if the build dependencies aren't specified in pyproject.toml).

How to Reproduce

$ python3.8 -m pip download --use-pep517 tensorflow
Collecting tensorflow
  Using cached tensorflow-2.4.1-cp38-cp38-macosx_10_11_x86_64.whl (173.9 MB)
Collecting termcolor~=1.1.0
  Using cached termcolor-1.1.0.tar.gz (3.9 kB)
    ERROR: Command errored out with exit status 1:
     command: /Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-78dnfhfc/termcolor_3986fcda2fa34e69b903950f54c7bcc5/setup.py'"'"'; __file__='"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-78dnfhfc/termcolor_3986fcda2fa34e69b903950f54c7bcc5/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-pip-egg-info-38q33ums
         cwd: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-78dnfhfc/termcolor_3986fcda2fa34e69b903950f54c7bcc5/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'
Collecting tensorflow
...
<proceeds to download every version of tensorflow>

[jaraco]

I can replicate the issue in isolation using this local package:

draft $ tree pkg
pkg
├── pyproject.toml
└── setup.cfg

0 directories, 2 files
draft $ cat pkg/*
[build-system]
requires=['setuptools', 'wheel']
[metadata]
name=pkg
version=1.0

[options]
install_requires=termcolor~=1.1
draft $ python3.8 -m pip download --use-pep517 ./pkg
Processing ./pkg
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting termcolor~=1.1
  Using cached termcolor-1.1.0.tar.gz (3.9 kB)
    ERROR: Command errored out with exit status 1:
     command: /Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-29mgi_lq/termcolor_7d20508cc80849e78090a2c413349ce7/setup.py'"'"'; __file__='"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-29mgi_lq/termcolor_7d20508cc80849e78090a2c413349ce7/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-pip-egg-info-oeb1bqb9
         cwd: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-29mgi_lq/termcolor_7d20508cc80849e78090a2c413349ce7/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz#sha256=1d6d69ce66211143803fbc56652b41d73b4a400a2891d7bf7a1cdf4c02de613b (from https://pypi.org/simple/termcolor/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement termcolor~=1.1 (from foo)
ERROR: No matching distribution found for termcolor~=1.1

[benoit-pierre]

Yes, and wheel too, or it will fails when trying to build the wheel...

[uranusjr]

I edited the title to reflect my understanding to the root cause.

[SpecLad]

FWIW, this is where the --use-pep517 flag gets lost: https://github.com/pypa/pip/blob/21.2.4/src/pip/_internal/resolution/resolvelib/factory.py#L493.

I tried adding use_pep517=comes_from.use_pep517 to the call and that fixes the issue, although I'm not well enough versed in the codebase to know whether that's the right solution.

[uranusjr]

@SpecLad I think that’s on the right track. Feel free to submit a PR and a test case and we can go from there.