Upgrade to setuptools 67.6.1 causes `DeprecationWarning` about `pkg_resources` #11975

robsdedude · 2023-04-17T16:04:01Z

Description

d7e0248 bumped setuptools introducing a deprecation warning "pkg_resources is deprecated as an API".

I don't see a way of making pip not run into it. With env variable _PIP_USE_IMPORTLIB_METADATA set, trying to install something with pip, I get

Traceback (most recent call last):
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 169, in exc_logging_wrapper
    status = run_func(*args)
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
    return func(self, options, args)
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 324, in run
    session = self.get_default_session(options)
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 98, in get_default_session
    self._session = self.enter_context(self._build_session(options))
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 125, in _build_session
    session = PipSession(
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/network/session.py", line 342, in __init__
    self.headers["User-Agent"] = user_agent()
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/network/session.py", line 175, in user_agent
    setuptools_dist = get_default_environment().get_distribution("setuptools")
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/metadata/importlib/_envs.py", line 188, in get_distribution
    return next(matches, None)
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/metadata/importlib/_envs.py", line 183, in <genexpr>
    matches = (
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/metadata/base.py", line 612, in iter_all_distributions
    for dist in self._iter_distributions():
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/metadata/importlib/_envs.py", line 176, in _iter_distributions
    for dist in finder.find_eggs(location):
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/metadata/importlib/_envs.py", line 144, in find_eggs
    yield from self._find_eggs_in_dir(location)
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/metadata/importlib/_envs.py", line 111, in _find_eggs_in_dir
    from pip._vendor.pkg_resources import find_distributions
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_vendor/pkg_resources/__init__.py", line 121, in <module>
    warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
DeprecationWarning: pkg_resources is deprecated as an API

without, I get

Traceback (most recent call last):
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 169, in exc_logging_wrapper
    status = run_func(*args)
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
    return func(self, options, args)
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 324, in run
    session = self.get_default_session(options)
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 98, in get_default_session
    self._session = self.enter_context(self._build_session(options))
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 125, in _build_session
    session = PipSession(
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/network/session.py", line 342, in __init__
    self.headers["User-Agent"] = user_agent()
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/network/session.py", line 175, in user_agent
    setuptools_dist = get_default_environment().get_distribution("setuptools")
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/metadata/__init__.py", line 75, in get_default_environment
    return select_backend().Environment.default()
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/metadata/__init__.py", line 63, in select_backend
    from . import pkg_resources
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_internal/metadata/pkg_resources.py", line 8, in <module>
    from pip._vendor import pkg_resources
  File "/.pyenv/versions/3.10.10/lib/python3.10/site-packages/pip/_vendor/pkg_resources/__init__.py", line 121, in <module>
    warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
DeprecationWarning: pkg_resources is deprecated as an API

Expected behavior

I'd like there to be a way to invoke pip such that it does not emit any warnings (unless of course some package relies on legacy setup methods or other valid warning cases).

That way, I can run pip with warnings as errors in my CI and actually notice when I run into deprecations.

pip version

23.1

Python version

3.10.10

OS

Linux

How to Reproduce

Install pip 23.1
python -m install pip==23.1
Run it with warnings as error
python -W error -m pip freeze
💥

Output

No response

Code of Conduct

I agree to follow the PSF Code of Conduct.

The text was updated successfully, but these errors were encountered:

pfmoore · 2023-04-17T16:32:19Z

_PIP_USE_IMPORTLIB_METADATA avoids most use of pkg_resources, but it looks like we still use it to find .egg distributions, even in the importlib.metadata code path.

Maybe we should just stop supporting eggs? IMO, it's about time.

In the meantime, I don't know what the best course of action is - we could patch out the warning from our vendored copy of setuptools, or disable the warning using the warnings machinery. Or we could just revert the setuptools upgrade, as being more trouble than it's worth... @pypa/pip-committers any thoughts?

I should point out, this might block the backport of 23.1 to ensurepip in CPython 3.10 and earlier¹, as I believe the CPython test suite runs with warnings as errors.

I've created the PR for CPython, but not merged it yet as I wanted to wait in case we needed a 23.1.1... ↩

pradyunsg · 2023-04-17T16:37:29Z

Or we could just revert the setuptools upgrade, as being more trouble than it's worth... @pypa/pip-committers any thoughts?

Let's do that, for now.

pfmoore · 2023-04-17T16:51:40Z

Let's do that, for now.

#11976

I won't merge just yet, I'll wait in case any other committers have different views.

I don't think this means that we should remove the setuptools upgrade from the ensurepip PR for CPython, as it's only used to pre-install in venv.

sbidoul · 2023-04-17T16:56:22Z

Should we enable the egg deprecation at the same time?

pip/src/pip/_internal/metadata/importlib/_envs.py

Lines 176 to 178 in 3b8791f

    
           for dist in finder.find_eggs(location): 
        
               # _emit_egg_deprecation(dist.location)  # TODO: Enable this. 
        
               yield dist

pfmoore · 2023-04-17T17:01:33Z

I'm -1 on doing that in a bugfix release. Maybe we should have done it in 23.1, but we didn't...

pradyunsg · 2023-04-17T18:09:37Z

I don't think this means that we should remove the setuptools upgrade from the ensurepip PR for CPython, as it's only used to pre-install in venv.

Oh, I think we should -- python/cpython#101039 ;)

pfmoore · 2023-04-17T19:06:16Z

If that goes in, then yes I'll drop setuptools from the upgrade PR. The current iteration of that PR is on the basis that the removal is stalled while the test suite issues get fixed (and so I'd rather we had the latest setuptools version in the meantime).

pradyunsg · 2023-04-17T19:35:39Z

Oh, the test suite was fixed in a separate PR. :)

pfmoore · 2023-04-17T19:39:25Z

Cool, I'll update the PR when python/cpython#101039 gets merged (I'd have to anyway as my PR would get merge conflicts 🙂)

uranusjr · 2023-04-18T00:58:30Z

So what should we move forward with this? It seems like a bad idea for us to suddenly deprecate eggs in a point release, so maybe it’d be better to roll back the pkg_resources update in 23.1, and do the upgrade + deprecate eggs in 23.2?

pfmoore · 2023-04-18T07:26:04Z

I will revert and issue a 23.1.1 bug fix release in a few days (in case there are any other bug fixes to include). For 23.2, we can have a separate discussion. The unconditional deprecation of pkg_resources still hits our users who are on 3.10 and earlier, so we may still prefer to pin to the old pkg_resources even if we do deprecate eggs (which I’m in favour of).

robsdedude · 2023-04-26T08:25:27Z

Seems like the downgrade of pkg_resources has been released in 23.1.1.

Thanks for the quick resolution 🙌

robsdedude added S: needs triage Issues/PRs that need to be triaged type: bug A confirmed bug or unintended behavior labels Apr 17, 2023

pfmoore mentioned this issue Apr 17, 2023

Mention setup.py develop change in 23.1 changelog #11973

Merged

pradyunsg changed the title ~~Upgrade to setup_tools 67.6.1 causes DeprecationWarning no matter what~~ Upgrade to setuptools 67.6.1 causes DeprecationWarning about pkg_resources Apr 17, 2023

pfmoore mentioned this issue Apr 18, 2023

Release 23.1 #11902

Closed

dalthviz mentioned this issue Apr 18, 2023

PR: Add QEnum macro for PyQt bindings spyder-ide/qtpy#424

Merged

pfmoore mentioned this issue Apr 20, 2023

Unexpected --user behavior since version 23.1 #11982

Closed

1 task

fcharras mentioned this issue Apr 20, 2023

pkg_ressources is deprecated skrub-data/skrub#550

Closed

robsdedude closed this as completed Apr 26, 2023

ClementPinard mentioned this issue Apr 26, 2023

DeprecationWarning: pkg_resources is deprecated as an API voxel51/eta#593

Closed

github-actions bot locked as resolved and limited conversation to collaborators May 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to setuptools 67.6.1 causes `DeprecationWarning` about `pkg_resources` #11975

Upgrade to setuptools 67.6.1 causes `DeprecationWarning` about `pkg_resources` #11975

robsdedude commented Apr 17, 2023 •

edited

Loading

pfmoore commented Apr 17, 2023

pradyunsg commented Apr 17, 2023

pfmoore commented Apr 17, 2023

sbidoul commented Apr 17, 2023

pfmoore commented Apr 17, 2023

pradyunsg commented Apr 17, 2023

pfmoore commented Apr 17, 2023

pradyunsg commented Apr 17, 2023

pfmoore commented Apr 17, 2023

uranusjr commented Apr 18, 2023

pfmoore commented Apr 18, 2023

robsdedude commented Apr 26, 2023 •

edited

Loading

Upgrade to setuptools 67.6.1 causes DeprecationWarning about pkg_resources #11975

Upgrade to setuptools 67.6.1 causes DeprecationWarning about pkg_resources #11975

Comments

robsdedude commented Apr 17, 2023 • edited Loading

Description

Expected behavior

pip version

Python version

OS

How to Reproduce

Output

Code of Conduct

pfmoore commented Apr 17, 2023

Footnotes

pradyunsg commented Apr 17, 2023

pfmoore commented Apr 17, 2023

sbidoul commented Apr 17, 2023

pfmoore commented Apr 17, 2023

pradyunsg commented Apr 17, 2023

pfmoore commented Apr 17, 2023

pradyunsg commented Apr 17, 2023

pfmoore commented Apr 17, 2023

uranusjr commented Apr 18, 2023

pfmoore commented Apr 18, 2023

robsdedude commented Apr 26, 2023 • edited Loading

Upgrade to setuptools 67.6.1 causes `DeprecationWarning` about `pkg_resources` #11975

Upgrade to setuptools 67.6.1 causes `DeprecationWarning` about `pkg_resources` #11975

robsdedude commented Apr 17, 2023 •

edited

Loading

robsdedude commented Apr 26, 2023 •

edited

Loading