ModuleNotFoundError: No module named 'setuptools._distutils'

[dHannasch]

pip install . suddenly started failing for many packages. Since setuptools just got a new version and pip didn't, and setuptools appears in the error, I'm guessing it's related to setuptools 50. Apologies if this turns out to be wrong.

This can be seen in any number of repositories, such as https://github.com/dHannasch/tox-sitepackages-example.

This doesn't appear to be quite the same as #2352.
I don't fully understand all the implications of https://github.com/pypa/setuptools/issues/2350, but SETUPTOOLS_USE_DISTUTILS=stdlib has no effect on this (ditto SETUPTOOLS_USE_DISTUTILS=1), so I think this is a separate issue.

Travis log:

$ python -m pip install .

Processing /home/travis/build/dHannasch/tox-sitepackages-example

  Installing build dependencies ... \/done

  Getting requirements to build wheel ... done

ERROR: Exception:

Traceback (most recent call last):

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main

    status = self.run(options, args)

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper

    return func(self, options, args)

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 325, in run

    reqs, check_supported_wheels=not options.target_dir

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve

    discovered_reqs.extend(self._resolve_one(requirement_set, req))

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one

    abstract_dist = self._get_abstract_dist_for(req_to_install)

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 340, in _get_abstract_dist_for

    abstract_dist = self.preparer.prepare_linked_requirement(req)

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 483, in prepare_linked_requirement

    req, self.req_tracker, self.finder, self.build_isolation,

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 91, in _get_prepared_distribution

    abstract_dist.prepare_distribution_metadata(finder, build_isolation)

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata

    self._setup_isolation(finder)

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation

    reqs = backend.get_requires_for_build_wheel()

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 161, in get_requires_for_build_wheel

    'config_settings': config_settings

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 265, in _call_hook

    raise BackendUnavailable(data.get('traceback', ''))

pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):

  File "/home/travis/build/dHannasch/tox-sitepackages-example/py38/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 86, in _build_backend

    obj = import_module(mod_path)

  File "/opt/python/3.7.1/lib/python3.7/importlib/__init__.py", line 127, in import_module

    return _bootstrap._gcd_import(name[level:], package, level)

  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import

  File "<frozen importlib._bootstrap>", line 983, in _find_and_load

  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import

  File "<frozen importlib._bootstrap>", line 983, in _find_and_load

  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked

  File "<frozen importlib._bootstrap_external>", line 728, in exec_module

  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

  File "/opt/python/3.7.1/lib/python3.7/site-packages/setuptools/__init__.py", line 5, in <module>

    import distutils.core

  File "/tmp/pip-build-env-co0toouh/overlay/lib/python3.7/site-packages/_distutils_hack/__init__.py", line 82, in create_module

    return importlib.import_module('._distutils', 'setuptools')

  File "/opt/python/3.7.1/lib/python3.7/importlib/__init__.py", line 127, in import_module

    return _bootstrap._gcd_import(name[level:], package, level)

ModuleNotFoundError: No module named 'setuptools._distutils'

[dHannasch]

This one doesn't happen for every package, I haven't quite narrowed it down but I think it's related to having a pyproject.toml or setup_requires, but that might only be because having a build environment is what causes setuptools 50 to be installed in the first place.

[vkrot-exos]

I had same issue. Removing pyproject.toml from the library which failed to install pass through it

[dHannasch]

That's true, and a useful workaround, but I think that's just because with no pyproject.toml, setuptools 50 doesn't get installed in the first place.

So anyone can avoid this problem for now by specifying setuptools<50 in their pyproject.toml, but the fact that that works probably isn't relevant for debugging setuptools, sadly. Oh well.

E.g.

[build-system]
requires = [
    "setuptools>=30.3.0,<50",
    "wheel",
    "pytest-runner",
    "setuptools_scm>=3.3.1",
]

[leycec]

So, setuptools 50.0 is fundamentally broken and breaks the entire Python ecosystem – both open-source and not. Well, isn't that special. Heads need rolling (especially those currently attached to the still-functioning torsos of managerial project leads). Until the community tastes sweet vengeance, the following is a slightly saner solution than @dHannasch's excellent starting point:

[build-system]
requires = [
    "setuptools!=50.0",
    "wheel",
]

That is to say, downstream projects should probably only blacklist the specific version of setuptools known to catastrophically fail under the fairly safe assumption that the next stable release will either hopefully revert or perhaps even correctly fix the breakage.

I can confirm the above circumvention behaves as expected in a project just maliciously blind-sided by this packaging horror show. Passing tox-based tests or it didn't happen, of course.

Die, setuptools 50.0! Die! 🩸

[shakthifuture]

How to resolve this issue? please give us at least a temporary solution.

[simmel]

@shakthifuture Setting environment variable SETUPTOOLS_USE_DISTUTILS=stdlib is a workaround, e.g.:

$ export SETUPTOOLS_USE_DISTUTILS=stdlib
$ pip3 install […]

[shakthifuture]

@simmel thank you, it working.