Setuptools as dependency is problematic w/ pip-tools

[akx]

908da75 (#322) added setuptools as a dependency in this package. However, the pip-tools package that's commonly used for pinning dependencies considers setuptools an unsafe dependency to have in a project at all (as discussed in #492), and as such doesn't save it in the pinned requirements file at all.

Since google-auth depends on Setuptools but a version couldn't have been pinned in the requirements, we're seeing

Collecting setuptools>=40.3.0 (from google-auth==1.19.1->our-proprietary-package==0.31.1)
  Downloading https://files.pythonhosted.org/packages/b0/8b/379494d7dbd3854aa7b85b216cb0af54edcb7fce7d086ba3e35522a713cf/setuptools-50.0.0-py3-none-any.whl (783kB)

which wreaks havoc on Ubuntu 16.04 + Python 3.5 machines due to pypa/setuptools#2352 / https://github.com/pypa/setuptools/issues/2350 / pypa/setuptools#2356 ...

The workaround is to add --allow-unsafe or manually pin setuptools, but is the requirement actually necessary in this package? No other package in the 48-line requirements.txt for this particular project would have required a version of setuptools.

Environment details

• OS: Ubuntu 16.04
• Python version: 3.5
• pip version: irrelevant
• google-auth version: 1.19.1

Steps to reproduce

1. Install google-auth on an Ubuntu 16.04 machine
2. It installs setuptools==50.0.0
3. SystemError: Parent module 'setuptools' not loaded, cannot perform relative import with setuptools 50 pypa/setuptools#2352 and friends

[busunkim96]

Hi @akx,

It looks like the pin was added to avoid an issue related to namespaces not being handled correctly in older versions of setuptools.

#322

Gotcha.

Grep'ing https://setuptools.readthedocs.io/en/latest/history.html for "namespace" shows 40.3.0 solved a problem with pkg_resource-style namespaces (pypa/setuptools#1321).

That said, 40.3.0 was released Sept 16, 2018 which is fairly recent...

If that's too new, 38.2.2 from Nov 27, 2017 fixed another bug (pypa/setuptools#1214 solved by pypa/setuptools#1215).

Thoughts?

Discussion in https://stackoverflow.com/questions/58843905/what-is-the-proper-way-to-decide-whether-to-allow-unsafe-package-versions-in-pip/58864335#58864335 suggests that this restriction will eventually be removed, but it doesn't look like there is any active discussion at the moment.

I think it might be alright to remove the pin - it has been nearly two years 40.3.0 was released at this point so folks are more likely to have it.

Paging @crwilcox, @tseaver, @plamut, @software-dov for thoughts.

[akx]

Hey @busunkim96, thanks for the response. :)

Considering the issue in #322 was only a warning, and even so only manifested when using https://github.com/pantsbuild/pex (which, as far as I know and have used it, is used for application packaging, not library packaging), I think (well, with hindsight being 20:20 and all) the real fix would be for the pex-built project to require a newer setuptools, not this library.

That said, though, I wonder if pkg_resources (from setuptools) is required at all anymore here:

~/b/google-auth-library-python (master) $ git grep pkg_res
docs/conf.py:import pkg_resources
docs/conf.py:version = pkg_resources.get_distribution("google-auth").version
google/__init__.py:    import pkg_resources
google/__init__.py:    pkg_resources.declare_namespace(__name__)
google/auth/crypt/_cryptography_rsa.py:import pkg_resources
google/auth/crypt/_cryptography_rsa.py:    release = pkg_resources.get_distribution("cryptography").parsed_version
google/auth/crypt/_cryptography_rsa.py:    if release < pkg_resources.parse_version("1.4.0"):
google/auth/crypt/_cryptography_rsa.py:except pkg_resources.DistributionNotFound:  # pragma: NO COVER
google/auth/crypt/es256.py:import pkg_resources
google/auth/crypt/es256.py:    release = pkg_resources.get_distribution("cryptography").parsed_version
google/auth/crypt/es256.py:    if release < pkg_resources.parse_version("1.4.0"):
google/auth/crypt/es256.py:except pkg_resources.DistributionNotFound:  # pragma: NO COVER

It seems to be used for two things in library code:

• declaring google as a namespace package; considering the hitch-hiker's guide to packaging prefers native namespace packages that have been a thing since 3.3 and this package doesn't support versions <3.3 anyway (

  google-auth-library-python/setup.py

  Line 49 in 6269643

  python_requires=">=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*",

  ), that could go away.
  EDIT: It looks like different Google API packages have different approaches to using or not using pkg_resources here; e.g. https://github.com/googleapis/python-cloudbuild uses native NS packages and https://github.com/googleapis/python-bigquery uses pkg_resources with a pkgutil fallback.

• figuring out whether the cryptography package is >= 1.4. Cryptography has exposed cryptography.__version__ for at least 7 years so I think just checking that would be enough. Or, you know, just require cryptography >= 1.4 – it's been out since 2014.
  EDIT: I just gave installing cryptography<1.4 in a python:3.5 container a shot and oh boy does it break with modern versions of OpenSSL (1.1.1d in this container). Version 2.0.3 (from 2017) is the first one that successfully manages to build, and we're on 3.1 now.
  EDIT 2: Figured I could be proactive and make a PR: fix: remove checks for ancient versions of Cryptography #596