Latest poetry-core with poetry 1.0 caused pip error during tox building

[fredrikaverpil]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Linux
• Poetry version: 1.0.10
• Link of a Gist with the contents of your pyproject.toml file:

Issue

We use tox (with isolated_build=true) with poetry 1.0.10 and the build system in pyproject.toml looked like this:

[build-system]
requires = ["poetry-core>=1.0.0a9"]
build-backend = "poetry.core.masonry.api"

Jenkins log, from executing the tox testenv:

+ tox -e mytestenv
.package create: REDACTED/.tox/.package
.package installdeps: poetry-core>=1.0.0a9
mytestenv create: REDACTED/.tox/mytestenv
mytestenv inst: REDACTED/.tox/.tmp/package/1/pkg1-0.0.0.tar.gz
ERROR: invocation failed (exit code 2), logfile: REDACTED/.tox/mytestenv/log/mytestenv-1.log
================================== log start ===================================
Looking in indexes: https://REDACTED/api/pypi/pypi-remote/simple, https://REDACTED/api/pypi/custom/simple
Processing ./.tox/.tmp/package/1/pkg1-0.0.0.tar.gz
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
ERROR: Exception:
Traceback (most recent call last):
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
    status = self.run(options, args)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
    return func(self, options, args)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 325, in run
    reqs, check_supported_wheels=not options.target_dir
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 340, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(req)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 483, in prepare_linked_requirement
    req, self.req_tracker, self.finder, self.build_isolation,
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 91, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata
    self._setup_isolation(finder)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation
    reqs = backend.get_requires_for_build_wheel()
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 161, in get_requires_for_build_wheel
    'config_settings': config_settings
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 265, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 86, in _build_backend
    obj = import_module(mod_path)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/__init__.py", line 10, in <module>
    from .builder import Builder
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/builder.py", line 6, in <module>
    from .builders.sdist import SdistBuilder
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/builders/__init__.py", line 2, in <module>
    from .wheel import WheelBuilder
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/builders/wheel.py", line 18, in <module>
    from packaging.tags import sys_tags
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/_vendor/packaging/tags.py", line 7, in <module>
    import distutils.util
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/_distutils_hack/__init__.py", line 83, in create_module
    return importlib.import_module('setuptools._distutils')
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'setuptools'

Changing our build system to the following solved the problem:

[build-system]
requires = ["poetry==1.0.10"]
build-backend = "poetry.masonry.api"

[abn]

@fredrikaverpil I believe the issue is the system default pip version. There was a bug in pip (circa 19.1) that meant that the PEP 517 build environment was not correctly created with the right versions of setuptools etc. With a poetry managed virtual environment this should not be an issue anymore as we will enforce the version of pip used via virtualenv (#2666).

Maybe #2826 will also remove this issue for externally (tox) managed virtual environments as well. In the interim, I would suggest that you get tox to upgrade pip used int he environment. Note that by default (if not using virtualenv's seed wheels), the system Python version's default pip version will be used, not the version you have in your active environment (ie. if you did pip install --upgrade --user pip (user is redundant in new versions of pip). The system default version ususaly is older on

[abn]

Also, this should not work unless your environment somehow already has poetry along with poetry-core

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.masonry.api"

[fredrikaverpil]

Oops, I meant to write:

Changing our build system to the following solved the problem:

[build-system]
requires = ["poetry==1.0.10"]
build-backend = "poetry.masonry.api"

[fredrikaverpil]

The funny thing is this happened over night without a change in tox.ini or pyproject.toml.

Ok, I will see which version of pip is in the system-Python.

[finswimmer]

The funny thing is this happened over night without a change in tox.ini or pyproject.toml.

The funny thing about that, is that a colleague of mine showed me this today with exact the same words 😄

We simply update poetry to the freshly released 1.1 and the problem vanished.

[Shadowsong27]

I am seeing this as well (not in tox, but standard poetry install), less than 1 day since the last success built, strange thing is: we have always fixed the poetry version to 1.0.5, but once updated to the latest 1.1.0, it works fine.

[nbraem]

seeing this as well, workaround is to add setuptools to requires:

[build-system]
requires = ["poetry", "setuptools"]
build-backend = "poetry.masonry.api"

This way, you can still use poetry 1.1.0

[abn]

@fredrikaverpil I am unable to reproduce this in a clean environment,

$ podman run --rm -i --entrypoint bash python:3.7 <<EOF
> pip install --quiet --upgrade pip
> pip install --quiet poetry tox
> pushd /opt
> poetry new foobar
> pushd foobar
> cat > tox.ini <<TOXEOF
> [tox]
> minversion = 3.3.0
> isolated_build = True
> envlist = py37
> 
> [testenv]
> commands =
>     pip list
> TOXEOF
> cat pyproject.toml
> tox -e py37
> EOF
/opt /
Created package foobar in foobar
/opt/foobar /opt /
[tool.poetry]
name = "foobar"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]

[tool.poetry.dependencies]
python = "^3.7"

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
.package create: /opt/foobar/.tox/.package
.package installdeps: poetry-core>=1.0.0
py37 create: /opt/foobar/.tox/py37
py37 inst: /opt/foobar/.tox/.tmp/package/1/foobar-0.1.0.tar.gz
py37 installed: foobar @ file:///opt/foobar/.tox/.tmp/package/1/foobar-0.1.0.tar.gz
py37 run-test-pre: PYTHONHASHSEED='2706476223'
py37 run-test: commands[0] | pip list
Package    Version
---------- -------
foobar     0.1.0
pip        20.2.3
setuptools 50.3.0
wheel      0.35.1
___________________________________ summary ____________________________________
  py37: commands succeeded
  congratulations :)

[abn]

@fredrikaverpil these issues may be of relevance for you pypa/setuptools#2353 pypa/setuptools#2376 pypa/pip#6264.

[fredrikaverpil]

Hm, a lot of people complaining about setuptools 50.0, but it all worked itself out fine on my end by using the following:

[build-system]
requires = ["poetry==1.0.10"]
build-backend = "poetry.masonry.api"

Does the above not make use of setuptools?

[mtkennerly]

@abn Those tickets are from a month ago, but I saw this issue start happening specifically within the last week in a GitHub Actions workflow, without any code changes in between.

• No issue (2020-09-27): https://github.com/mtkennerly/poetry-dynamic-versioning/actions/runs/275347882
• Issue began (2020-10-03): https://github.com/mtkennerly/poetry-dynamic-versioning/actions/runs/287057357

It occurs only in certain combinations. Python (3.7 | 3.8) + Poetry (1.0.2 | 1.0.5) fail, whereas the same Poetry versions work with Python (3.5 | 3.6), and Poetry (1.0.10 | 1.1.0rc1) work with all four versions of Python tested. It doesn't matter whether Poetry is installed with pip or with get-poetry.py (called "script" in the workflow matrix).

[jhrmnn]

I'm seeing the missing Setuptools error in environments created with Conda.