import pkg_resources triggers a DeprecationWarning on Python 3.9

[alexhenrie]

I'm pretty sure this is a bug because I don't think pkg_resources is deprecated in its entirety.

$ PYTHONWARNINGS=error python3 -c 'import pkg_resources'
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/packaging/version.py", line 57, in parse
    return Version(version)
  File "/usr/lib/python3.9/site-packages/packaging/version.py", line 298, in __init__
    raise InvalidVersion("Invalid version: '{0}'".format(version))
packaging.version.InvalidVersion: Invalid version: '/usr/lib/python3.9/site'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3239, in <module>
    def _initialize_master_working_set():
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3222, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3251, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 558, in _build_master
    ws = cls()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 551, in __init__
    self.add_entry(entry)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 607, in add_entry
    for dist in find_distributions(entry, True):
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2056, in find_on_path
    path_item_entries = _by_version_descending(filtered)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2026, in _by_version_descending
    return sorted(names, key=_by_version, reverse=True)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2024, in _by_version
    return [packaging.version.parse(part) for part in parts]
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2024, in <listcomp>
    return [packaging.version.parse(part) for part in parts]
  File "/usr/lib/python3.9/site-packages/packaging/version.py", line 59, in parse
    return LegacyVersion(version)
  File "/usr/lib/python3.9/site-packages/packaging/version.py", line 127, in __init__
    warnings.warn(
DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release

[alexhenrie]

Cross-posted to pypa/packaging#368 because I'm not sure whether the bug is in packaging or in setuptools.

[yan12125]

The issue seems to be that _by_version_descending() in setuptools attempts to pass invalid version strings to packaging. With this patch

diff --git a/pkg_resources/__init__.py b/pkg_resources/__init__.py
index 737f4d5f..6ca1c7b4 100644
--- a/pkg_resources/__init__.py
+++ b/pkg_resources/__init__.py
@@ -2020,7 +2020,8 @@ def _by_version_descending(names):
         Parse each component of the filename
         """
         name, ext = os.path.splitext(name)
-        parts = itertools.chain(name.split('-'), [ext])
+        parts = list(itertools.chain(name.split('-'), [ext]))
+        print(parts)
         return [packaging.version.parse(part) for part in parts]
 
     return sorted(names, key=_by_version, reverse=True)

I got

['/usr/lib/python3.9/site', 'packages/MarkupSafe', '1.1.1', 'py3.9', '.egg-info']

With that observation, I created a fix(?)

diff --git a/pkg_resources/__init__.py b/pkg_resources/__init__.py
index 737f4d5f..84df31dd 100644
--- a/pkg_resources/__init__.py
+++ b/pkg_resources/__init__.py
@@ -2020,8 +2020,15 @@ def _by_version_descending(names):
         Parse each component of the filename
         """
         name, ext = os.path.splitext(name)
-        parts = itertools.chain(name.split('-'), [ext])
-        return [packaging.version.parse(part) for part in parts]
+        parts = list(itertools.chain(name.split('-'), [ext]))
+
+        def parse_version_ignore_invalid(version):
+            try:
+                return packaging.version.Version(version)
+            except packaging.version.InvalidVersion:
+                return version
+
+        return [parse_version_ignore_invalid(part) for part in parts]
 
     return sorted(names, key=_by_version, reverse=True)
 

UPDATE:

I believe the issue also appears on older Python version if packaging is updated to 20.5 or newer [1]. Here is my testing environment:

• Arch Linux
• setuptools git (5cf3865)
• Changing packaging==20.4 to packaging==20.4 in pkg_resources/_vendor/vendored.txt and run paver update_vendored

[1] pypa/packaging@5abbd38

[bnavigator]

This is also hit in dask/fastparquet#558. @yan12125's investigation makes it clear: passing an invalid version (the first parts of the name here) throws the DeprecationWarning of LegacyVersion.

setuptools/pkg_resources/__init__.py

Lines 2025 to 2027 in c121d28

	name, ext = os.path.splitext(name)
	parts = itertools.chain(name.split('-'), [ext])
	return [packaging.version.parse(part) for part in parts]

setuptools/setuptools/_vendor/packaging/version.py

Lines 48 to 58 in c121d28

	def parse(version):
	# type: (str) -> Union[LegacyVersion, Version]
	"""
	Parse the given version string and return either a :class:`Version` object
	or a :class:`LegacyVersion` object depending on if the given version is
	a valid PEP 440 version or a legacy version.
	"""
	try:
	return Version(version)
	except InvalidVersion:
	return LegacyVersion(version)

[jaraco]

I'm closing this in favor #2497.