possible bug in 4.3.0: stevedore.exception.NoMatches: No extensions found

[maorfr]

Hello,

importlib-metadata is a dependency for several packages we use and we have encountered a problem with the installation of 4.3.0 together with the jenkins-job-builder package (version 2.10.1 fwiw).

Just to illustrate the issue:

$ docker run -it quay.io/app-sre/qontract-reconcile:013d67a /bin/sh
sh-4.4# jenkins-jobs --version
Traceback (most recent call last):
  File "/usr/local/bin/jenkins-jobs", line 33, in <module>
    sys.exit(load_entry_point('jenkins-job-builder==2.10.1', 'console_scripts', 'jenkins-jobs')())
  File "/usr/local/lib/python3.6/site-packages/jenkins_job_builder-2.10.1-py3.6.egg/jenkins_jobs/cli/entry.py", line 164, in main
    jjb = JenkinsJobs(argv)
  File "/usr/local/lib/python3.6/site-packages/jenkins_job_builder-2.10.1-py3.6.egg/jenkins_jobs/cli/entry.py", line 58, in __init__
    self.parser = create_parser()
  File "/usr/local/lib/python3.6/site-packages/jenkins_job_builder-2.10.1-py3.6.egg/jenkins_jobs/cli/parser.py", line 100, in create_parser
    extension_manager.map(parse_subcommand_args, subparser)
  File "/usr/local/lib/python3.6/site-packages/stevedore-3.3.0-py3.6.egg/stevedore/extension.py", line 291, in map
    raise NoMatches('No %s extensions found' % self.namespace)
stevedore.exception.NoMatches: No jjb.cli.subcommands extensions found

after pinning importlib-metadata to 4.2.0 in app-sre/qontract-reconcile#1589, the issue goes away:

$ docker run -it quay.io/app-sre/qontract-reconcile:4e86cce /bin/sh
sh-4.4# jenkins-jobs --version
Jenkins Job Builder version: 2.10.1

hope this helps!

[jaraco]

Indeed, it sounds like you've found a regression. #317 was the change introduced for this release. I'll want to inspect what it is about that environment that violates the expectations of the change.

[jaraco]

The problem seems to be that the normalized_name for eggs isn't calculated properly:

>>> dists = list(md.distributions())
>>> [dist.name for dist in dists]
['setuptools', 'pip', 'reconcile', 'dnspython', 'sendgrid', 'parse', 'tzlocal', 'jenkins-job-builder', 'sentry-sdk', 'prometheus-client', 'UnleashClient', 'tabulate', 'terrascript', 'ruamel.yaml', 'pyOpenSSL', 'jira', 'Jinja2', 'pypd', 'slackclient', 'urllib3', 'botocore', 'boto3', 'python-terraform', 'semver', 'python-gitlab', 'anymarkup', 'ldap3', 'hvac', 'PyGithub', 'jsonpath-rw', 'toml', 'graphqlclient', 'click', 'sretoolbox', 'starkbank-ecdsa', 'python-http-client', 'pytz', 'stevedore', 'python-jenkins', 'pbr', 'fasteners', 'PyYAML', 'certifi', 'requests', 'mmh3', 'fcache', 'APScheduler', 'ruamel.yaml.clib', 'cryptography', 'requests-toolbelt', 'requests-oauthlib', 'oauthlib', 'defusedxml', 'MarkupSafe', 'websocket-client', 'jmespath', 's3transfer', 'xmltodict', 'json5', 'configobj', 'anymarkup-core', 'pyasn1', 'PyNaCl', 'PyJWT', 'Deprecated', 'ply', 'importlib-metadata', 'multi-key-dict', 'idna', 'chardet', 'appdirs', 'cffi', 'wrapt', 'zipp', 'typing-extensions', 'pycparser', 'dbus-python', 'subscription-manager', 'ethtool', 'gpg', 'PyGObject', 'python-dmidecode', 'rpm', 'pyinotify', 'setuptools', 'python-dateutil', 'six', 'syspurpose', 'iniparse', 'decorator', 'pip']
>>> [dist._normalized_name for dist in dists]
['setuptools', 'pip', 'EGG', 'EGG', 'EGG', '', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', '', 'EGG', 'EGG', 'EGG', '', 'EGG', 'EGG', '', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', '', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', '', '', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', '', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'EGG', 'dbus_python', 'subscription_manager', 'ethtool', 'gpg', 'pygobject', 'python_dmidecode', 'rpm', 'pyinotify', 'setuptools', 'python_dateutil', 'six', 'syspurpose', 'iniparse', 'decorator', 'pip']

[jaraco]

>>> dists[0]._path
PosixPath('/usr/local/lib/python3.6/site-packages/setuptools-56.2.0.dist-info')
>>> dists[2]._path
PosixPath('/usr/local/lib/python3.6/site-packages/reconcile-0.2.2-py3.6.egg/EGG-INFO')

The assumption that the dist name is in the stem of the _path was a mistake.

[maorfr]

thanks a lot for the quick action @jaraco! much appreciated.