Drop Support for Python 2 in pip 21.0

[dstufft]

Placeholder issue for dropping support for Python 2. Discussion is largely occurring at https://discuss.python.org/t/packaging-and-python-2/662.

Related: #4350.

[pradyunsg]

We've documented how we're gonna handle Python 2 support going forward, at https://pip.pypa.io/en/latest/development/release-process/#python-2-support.

Python 2 support

pip will continue to ensure that it runs on Python 2.7 after the CPython 2.7 EOL date. Support for Python 2.7 will be dropped, if bugs in Python 2.7 itself make this necessary (which is unlikely) or Python 2 usage reduces to a level where pip maintainers feel it is OK to drop support. The same approach is used to determine when to drop support for other Python versions.

However, bugs reported with pip which only occur on Python 2.7 would likely not be addressed directly by pip’s maintainers. Pull Requests to fix Python 2.7 only bugs will be considered, and merged (subject to normal review processes). Note that there may be delays due to the lack of developer resources for reviewing such pull requests.

[pfmoore]

It might be worth considering what we'd do if one of our vendored dependencies dropped support for Python 2. We would basically have two options - freeze the version of the dependency that we vendored, or drop support for Python 2 ourselves. Neither seems ideal, but I assume we'd probably have to freeze the version unless/until the changes in the vendored library became too significant enough for that to be viable (e.g., a security issue).

I'd be a strong -1 on vendoring two versions, one that supports Python 2 and one that doesn't - that would be unreasonable bloat as well as a maintenance overhead. But I guess another option might be to start shipping separate Python 2 and Python 3 versions of pip in that case (but again, that's probably more of a maintenance overhead than I'd be OK with).

I've no idea what our dependncies' plans are, but having a documented plan of action on how we'll react is probably good.

My preferred option would be to say that we'd stick on the last version that supported Python 2 for minor dependencies but if a major dependency dropped Python 2 support, we'd expect to follow suit. What's "minor" would be a judgement call, but I'd probably class things like requests, html5lib and CacheControl as major (mainly because of the risk of not getting security fixes).

[pradyunsg]

I recently started a pip-maintainers email thread (basically, an email with To: containing everyone with write access to this repo) to talk about pip's Python 2 support plans, with 2 specific questions:

1. What metric do we want to use?
2. What %-age of that metric do we stop supporting Python 2 support?

Our answers ended up along the lines of:

1. PyPI file downloads/simple downloads w/ Python 2.7 using the latest pip (that's been out for more than a month), are the best metrics we have.
2. ~20%, with 3 releases that print "okay, Python 2.7 support in pip is gonna go away in release YY.N".

---

I looked at our metrics for the last few weeks and... we've hit the "~20%" target in April 2020! So, pip 20.1 is going to be the first version to start printing a warning containing the exact version of pip that will drop Python 2 support. pip 20.3 will be the last version of pip that supports Python 2.7.

---

Picking out some of the statements from that thread (without attribution):

I don’t care about the specific numbers at play, 20%, 15%, 10%, whatever they’re all roughly the same to me. I would even be OK with calling the number 20%, and starting the process now at this point. My only real caveat to that is I think that the higher the target number is, the more releases we should give a warning in before we drop it.

I think that the main thing we should do is give a very clear indication of the timeline, and what people need to do. [snip] we should start sooner rather than later emphasising the point that this does not mean that "Python 2 users will no longer be able to use pip" but rather that "Python 2 users will have to pin to a specific pip version".

I was recently thinking about suggesting dropping Python 2 support in 21.0 (1 year after CPython) [snip] So +1 from me also. A deprecation could be added in 20.1

Another reason I like this target, is that Python 3.5 goes EoL in 2020-09, which means pip 20.3 should be the last version to support Python 2.7 and/or Python 3.5. We'll be able to use f-strings in pip's codebase starting October this year!