AttributeError and ModuleNotFoundError after release of 72.x

[jaraco]

After removing the test command in #4458, the integration tests started failing.

Several of the tests passed, but one failed:

[gw0] [ 11%] PASSED setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[pip-v.LATEST] 
setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[pytest-v.LATEST] 
[gw1] [ 22%] PASSED setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[mypy-v.LATEST] 
setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[botocore-v.LATEST] 
[gw0] [ 33%] PASSED setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[pytest-v.LATEST] 
[gw3] [ 44%] PASSED setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[pyyaml-v.LATEST] 
setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[protobuf-v.LATEST] 
setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[charset-normalizer-v.LATEST] 
[gw2] [ 55%] PASSED setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[kiwisolver-v.LATEST] 
setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[brotli-v.LATEST] 
[gw1] [ 66%] PASSED setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[botocore-v.LATEST] 
[gw2] [ 77%] FAILED setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[brotli-v.LATEST] 
[gw3] [ 88%] PASSED setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[charset-normalizer-v.LATEST] 
[gw0] [100%] PASSED setuptools/tests/integration/test_pip_install_sdist.py::test_install_sdist[protobuf-v.LATEST] 

The traceback, as reported by pip, seems to indicate some issue with an entry point expecting setuptools.dist.check_test_suite:

        File "/tmp/pytest-of-runner/pytest-0/popen-gw2/test_install_sdist_brotli_v_LA0/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 286, in __init__
          self.finalize_options()
        File "/tmp/pytest-of-runner/pytest-0/popen-gw2/test_install_sdist_brotli_v_LA0/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 630, in finalize_options
          ep(self)
        File "/tmp/pytest-of-runner/pytest-0/popen-gw2/test_install_sdist_brotli_v_LA0/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 650, in _finalize_setup_keywords
          ep.load()(self, ep.name, value)
        File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/importlib/metadata/__init__.py", line 173, in load
          return functools.reduce(getattr, attrs, module)
      AttributeError: module 'setuptools.dist' has no attribute 'check_test_suite'

[jaraco]

And indeed, that attribute was removed. But what is it that's leaving the lingering reference to it? Is it metadata left over from an older Setuptools version? Is it something specific to Brotli?

That Brotli package does still declare the test_suite, so perhaps that's the reason only it is affected.

I had expected the removal of the finalize_distribution_options:test_suite to have the effect of silently ignoring that parameter, so it strengthens my suspicion that there's another latent copy of setuptools metadata that's causing that entry point reference to linger.

[jaraco]

It's not simply a problem with brotli, as I'm able to install it locally using the latest setuptools:

 draft 🐚 py -m venv .venv
 draft 🐚 py -m pip install git+https://github.com/pypa/setuptools
Collecting git+https://github.com/pypa/setuptools
  Cloning https://github.com/pypa/setuptools to /private/var/folders/f2/2plv6q2n7l932m2x004jlw340000gn/T/pip-req-build-vl7j12d3
  Running command git clone --filter=blob:none --quiet https://github.com/pypa/setuptools /private/var/folders/f2/2plv6q2n7l932m2x004jlw340000gn/T/pip-req-build-vl7j12d3
  Resolved https://github.com/pypa/setuptools to commit 5e1b3c414779317bc3e105d9bae82ce70c22dbf9
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: setuptools
  Building wheel for setuptools (pyproject.toml) ... done
  Created wheel for setuptools: filename=setuptools-72.0.0.post20240728-py3-none-any.whl size=2337441 sha256=0b6397e08cd8e4dfabd7312d2a570f210263b9bab5234a42db06c21dc27270ef
  Stored in directory: /private/var/folders/f2/2plv6q2n7l932m2x004jlw340000gn/T/pip-ephem-wheel-cache-g748brzi/wheels/c3/65/21/c2a5b2034fd0d358d3c4ccf631d8e46540876cb99e34699224
Successfully built setuptools
Installing collected packages: setuptools
Successfully installed setuptools-72.0.0.post20240728
 draft 🐚 py -m pip install --no-build-isolation --no-deps --no-binary brotli brotli
Collecting brotli
  Downloading Brotli-1.1.0.tar.gz (7.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.4/7.4 MB 20.0 MB/s eta 0:00:00
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: brotli
  Building wheel for brotli (pyproject.toml) ... done
  Created wheel for brotli: filename=Brotli-1.1.0-cp313-cp313t-macosx_14_0_arm64.whl size=399312 sha256=e8da1903b55d7390454f9a31f9520485d70d97675273a31e191aea2becc45203
  Stored in directory: /Users/jaraco/Library/Caches/pip/wheels/c9/c5/78/f2d3f736e7ebfd06954dba411d3ebcaef8f8c8008279e2532e
Successfully built brotli
Installing collected packages: brotli
Successfully installed brotli-1.1.0

[jaraco]

Indeed, even if I just run tox -e integration, the tests all pass locally.

[jaraco]

So I tried re-running the integration tests, and this time, they passed, allowing the release to complete.

Looking at the integration tests, I'm struggling to conceive of how such a failure could occur non-deterministically.

[itemir]

Not sure what happened here but this removal also seems to have broken a bunch of things. I found this issue while debugging an error that came out of nowhere earlier today: ModuleNotFoundError: No module named 'setuptools.command.test'

Our builds started failing with obscure errors like this while installing our dependencies:

% pip install cssbeautifier
Collecting cssbeautifier
  Downloading cssbeautifier-1.15.1.tar.gz (25 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      Traceback (most recent call last):
        File "/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "r/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/tz/c3k9jy_579b6k3rjmx09rc6w0000gn/T/pip-build-env-anm7x4qw/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/tz/c3k9jy_579b6k3rjmx09rc6w0000gn/T/pip-build-env-anm7x4qw/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/tz/c3k9jy_579b6k3rjmx09rc6w0000gn/T/pip-build-env-anm7x4qw/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 497, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/private/var/folders/tz/c3k9jy_579b6k3rjmx09rc6w0000gn/T/pip-build-env-anm7x4qw/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 9, in <module>
      ModuleNotFoundError: No module named 'setuptools.command.test'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[meh9]

For others who have landed here wondering what's going on and how to fix it...

Following @jaraco's comment above you might already have a version of setuptools pinned and trying to figure out why it's not being used, try passing the --no-build-isolation to pip so it can use your pinned version, e.g.:

pip install --no-build-isolation -r requirements.txt

It's temporarily solved it for me.

[jaraco]

Good news is I can replicate the issue with pip install cssbeautifier (and presumably any other source-only distribution). I find that surprising because there should be no references to setuptools.command.test.

I've yanked 72.0.0 for now while I investigate.

[jaraco]

I'm going to move this discussion to #4519.