Deprecate pip, pipX, and pipX.Y

[dstufft]

Currently, people are regularly running into problems over confusion about which particular Python a particular invocation of pip is going to manage. There are many cases where what Python a particular invocation of pip* is going to invoke is unclear:

• pip3 install --upgrade pip will overwrite pip and possibly switch it from pointing to 2.7 to 3.5.
• pip3 isn't specific enough, you might have 3.4 and 3.5 installed.
• pip3.4 isn't specific enough, you might have 3.4.0 and 3.4.1 installed.
• pip3.4.0 isn't specific enough, you might have multiple copies of 3.4.0 installed in various locations.
• We don't have a good answer for what the pip binary should be called on alternative Python interpreters like PyPy (pip-pypy? What if we have two versions of PyPy? pip-pypy2.6? What if we have PyPy and PyPy3? pip-pypy-2.6 and pip-pypy3-2.6?).

Overall, it's become increasingly clear to me that this is super confusing to people. Python has a built in mechanism for executing a module via python -m. I think we should switch to using this as our preferred method of invocation. This should completely eliminate the confusion that is caused when python and pip don't point to the same version of Python, as well as solve the problem of what do you call the binary on alternative implementations.

In addition to the confusion, we also have the fact that pip install --upgrade pip doesn't actually work on Windows because of problems with the .exe file being open. However python -m pip install --upgrade pip does work there.

I see only three real downsides:

• There is a lot of documentation, examples, code or other instances of inertia using just pip and this will be churn for those.
• It's 10 more letters to type.
• It doesn't work on Python 2.6.

For the first of these, I think the answer is to just have a very long deprecation cycle, in the order of years. I wouldn't even put a specific date on it's removal, I'd just add the warnings and re-evaluate in the future. Luckily we've shipped support for python -m pip for quite some time, so it won't be something that people need to deal with version differences (mostly).

The second of these I don't really have a great answer for, I think that 10 extra letters probably isn't that big of a cost to pay for the reduced confusion and the default answer working on Windows. We could possibly offer a recipe in the docs to restore pip, pipX, and pipX.Y via shell aliases.

This last item is the biggest sticking point for me. As far as I know, Python 2.6 still has far too many users for us to drop it since, as of 6 months ago, it was still ~10% of the traffic on PyPI (source). The problem with Python 2.6 is that it only supports -m when the target is a module not a package. I see four possible solutions to this:

• Don't deprecate pip* on Python 2.6.
• Add a module like pipcli.py that people can invoke like python -m pipcli instead of python -m pip.
• Move pip/ to _pip/ and make pip.py.
• Document that due to limitations of Python 2.6, it will need to be invoked as python -m pip.__main__.

I don't really like the pipcli idea, the other three all have pros and cons, but I think I personally live with either not deprecating pip* on Python 2.6 and/or documenting that it needs to be invoked as ``python -m pip.main on Python 2.6).

What do the @pypa/pip-developers think?

[RonnyPfannschmidt]

i like the move from pip to _pip,
that way pip's implementation goes to a more "private" namespace
the expense is breaking tools that go into pips internals

[dstufft]

The other expense is that if we ever want to make a public API we're either limited to having a single namespace (whatever is in pip.py) or we need to change it back to a package (and possibly break Python 2.6 again, unless we've deprecated it by then).

Of course, we may never make a public API in which case, the point is moot.

[dstufft]

I can't help but think that @warsaw and @ncoghlan probably have some opinions on this too.

[dstufft]

Maybe @bkabrda too! and @tdsmith

[Lukasa]

Don't underestimate the power of that first point. The intertia is high: lots of tools will assume pip, and lots of documentation will be wrong. Having a long deprecation cycle is basically mandatory here. Otherwise, I think this is a good idea: +1.

[dholth]

-1 on removing pip I would have to change all of my deployment scripts.
+1 on removing pipX and pipX.Y

On Tue, Oct 6, 2015, at 08:16 AM, Cory Benfield wrote:

Don't underestimate the power of that first point. The intertia is
high: lots of tools will assume pip, and lots of documentation
will be wrong. Having a long deprecation cycle is basically mandatory
here. Otherwise, I think this is a good idea: +1.

— Reply to this email directly or view it on GitHub[1].

Links:

1. Deprecate pip, pipX, and pipX.Y #3164 (comment)

[dstufft]

Don't underestimate the power of that first point. The intertia is high: lots of tools will assume pip, and lots of documentation will be wrong. Having a long deprecation cycle is basically mandatory here.

Yea, completely agree. I essentially assume that we should not have a defined removal date (and possibly never) and just have it log a message to stderr.

[Lukasa]

I essentially assume that we should not have a defined removal date (and possibly never) and just have it log a message to stderr.

I don't think that will work. Just printing annoying warnings with no defined "your shit will break no later than X" date doesn't really help: people will just ignore the warnings. I think if you want to do this you should decide a date (possibly one far away, but still). One possible date to start the discussion: when the last RHEL LTS release with Python 2.6 stops being supported (that's very far away still, but worth discussing).

[dstufft]

-1 on removing pip I would have to change all of my deployment scripts.
+1 on removing pipX and pipX.Y

I don't think it makes sense to deprecate (not talking about removal any time soon) pipX and pipX.Y without also doing it for pip since that is arguably the worse offender of them all.

[piotr-dobrogost]

One possible date to start the discussion: when the last RHEL LTS release with Python 2.6 stops being supported (that's very far away still, but worth discussing).

This is good idea.

[dstufft]

One possible date to start the discussion: when the last RHEL LTS release with Python 2.6 stops being supported (that's very far away still, but worth discussing).

That's November 30, 2020 for the end of RHEL 6 Production 3 phase. They have a super special extended life cycle beyond that, but perhaps we could just target 2020 and if we roll around to 2020 and Python 2.6 is still somehow in wide support, we push it back further.

[dholth]

Another alternative would be to stop requiring that pip execute in the same Python environment that it is installing things into...

[dstufft]

To be honest, I'm not sure how pip install -p python2.7 is any better than python2.7 -m pip install. We have to inspect the Python we're installing into to get information from it, so either we're going to subshell into that Python to shuffle data back and forth (like my half done virtualenv rewrite does) or we'll need to continue to be executed by the same Python environment. Feels like shuffling deck chairs more than anything else.

[dholth]

For that particular idea the main benefit would be that you would only have to upgrade pip once.

[dstufft]

That much is true, the flip side of that is it makes it (somewhat) harder to support versions of pip older than what pip itself supports, since the installs are no longer independent (or you'll need to keep around an older copy installed somewhere else). On the other hand, pip could more easily drop support for running the main pip binary command in a particular Python, while keeping compatibility for installing into that version of Python. It would (continue) to enable bundling all of pip into a single zip file that can be executed somewhat independently of actually having it installed.

It doesn't address the fact that pip install --upgrade pip on Windows blows up because the OS has a handle open to pip.exe though, which I don't think can be solved without using python -m, at least if I understand @pfmoore correctly.

[bkabrda]

I think @sYnfo wants to comment on this more than I do, since I no longer maintain Python in Fedora/RHEL.

[dstufft]

Ah, that's right, I forgot. Sorry!

[bkabrda]

(But I personally think that even if these are removed, we'll still provide them on distribution level in form that is best for the set of Python interpreters that we ship; at least that was my first idea when I read the proposal... We have a general policy for Python executables that mandates this in Fedora.)

[pfmoore]

• +1 for deprecation - but I'd be happy enough simply to deprecate in documentation, not make the scripts themselves moan at the user.
• It's only 6 more letters on Windows - py -m pip :-)
• For 2.6, I'm OK with either letting people continue to use pip or advising python2.6 -m pip.__main__. I don't think it's worth making changes to pip to give them any other solution.

The inertia issue is huge, and I don't think we should fight it directly. Rather, we should switch the documentation to use the "python -m pip" form, and make that form official PyPA policy (by which I mean we take pains to use that form consistently in whatever we post, etc). Maybe offer PRs for the install documentation of well-known projects to switch them to the new form. We can worry about formally deprecating and/or removing the scripts once the python -m pip form starts to gain a bit of traction in common usage.

[tdsmith]

I think my success criterion is making this the incantation that people get from StackOverflow. Shoot for the moon, etc etc.

I think publishing a linkable intent-to-deprecate message with a rationale may help convince third-party maintainers to accept documentation PRs.

The messaging here is tricky, because deprecating pip doesn't mean deprecating pip...

[dstufft]

I think the only real alternative is Daniel's suggestion. I think the current situation sucks and I can't really think of a way to save the attempt to manage pip versions using a suffix or prefix or anything that doesn't end up actually specifying which interpreter you want to run under.

[sYnfo]

I'm +1 for deprecation—it seems to make a whole lot of sense from upstream point of view and I don't see any issue this could cause downstream.

In Fedora this would mean shipping all the binaries during the deprecation period, as we do now; and not shipping them at all afterward. Perfect sync with upstream, hopefully. :) I'll make sure all the Fedora docs get updated, when this goes official.

( @bkabrda If I understand the guidelines correctly, they only mandate shipping all the MAJOR.MINOR executables iff there are any executables in the first place, this issue seems to be about removing the pip executables entirely, right? )

Also +1 to either keeping pip or using python -m pip.main with python 2.6.

[erikrose]

A couple of points that haven't come up:

• Doesn't this just move the problem to python? Why is python (of various versions and in various virtualenvs) any easier to keep track of than pip?
• If you're using a virtualenv (which most people should be), pip unambiguously refers to the active one. An alternative solution would be to encourage people to always use venvs, which solves other problems as well, e.g. dependency conflicts from dropping everything into a global environment.

[rschoon]

I'm not big on the idea, but I don't think I've ever run pip outside of virtualenv, which is usually activated into my current shell, so the extra typing doesn't gain me anything (I'm selfish!). Of course, if it's not activated, I probably have to type a lot more than that anyway...

How would people feel about making an exception for virtualenvs? There's no mystery for which python it is using, but then again it might be too confusing to have it work differently between for virtualenv and the system install, so I'm feeling pretty humble about its quality as a suggestion. I'm also not sure there's a good solution for checking that.

Also,

pip3.4 isn't specific enough, you might have 3.4.0 and 3.4.1 installed.

Does this really come up often? I'm curious how you're distinguishing between them if so, since I didn't think python would normally install itself any more specific than by the pythonX.Y name (and unless they have different prefixes, sounds like it would also try to share site-packages anyway).

[rschoon]

Oh, I guess that does apply having multiple 3.4.0 installs too, but then it seems like you'd definitely be using full paths to distinguish whether you're using "pip" or "python -m pip", but since you're using full paths anyway, the argument against having extra wordiness goes away.

[mattrobenolt]

I may be a little disconnected here, but why not replace pip with an alias that was effectively something like:

alias pip=/usr/bin/env python -m pip

Or a script that was akin to this to be installed into /usr/local/bin/pip.

That way we don't lose pip, and it universally works errywhere.

Also not sure if that solves the problems you're facing, just my $0.02.

[dstufft]

@erikrose
It does sort of shift the problem to python, but typically the confusion that I've seen arise stems from when python and pip disagree. Like you might have /usr/bin/python and /usr/local/bin/python and if python points to /usr/local/bin/python and pip points to /usr/bin/python it's a recipe for confusion. So we completely eliminate the confusion cased by having the two disagree, but removing any ability for it to disagree. You're still left with confusion about what python means but I don't think there's anything at all that can be done about that, and particularly not by us.

A virtual environment makes the issue harder to hit, but I think that it's still possible. My gut tells me that new users (the ones most likely to get tripped up by this) are probably not going to be religiously using virtual environments (if they're using them at all).

@rschoon
We could possibly do that, it'd require a permanent special case inside of pip when pip installs itself (because there's no setup.py in a virtual environment) but it's certainly a possibility. My main fear with that is it feels like it'd just always be better to use python -m pip anyways because it works inside and outside of a virtual environment instead of having to remember to switch commands based on whether you're in one or not.

I'm not sure exactly how often the 3.4.0 and 3.4.3 thing comes up. I know that it's not super unusual on OSX since you have system provided Python sitting in /usr/bin/python and you might also install the Python.org installer or Homebrew or Macports or pyenv (or some combination of the above).

[dstufft]

@mattrobenolt

Basically, because it's super confusing if you're doing something like running myvenv/bin/pip without activating the virtual environment first or if you have a copy of Python installed to a non standard location /opt/my-app and you run /opt/my-app/bin/pip and expect to manage /opt/my-app/bin/python.

[qwcode]

it feels like a loss for virtual environments to lose a basic pip script when they don't suffer from this problem.

as for real pythons I guess I wouldn't mind seeing a more exact pip-<python binary path> type console script, and let the distro packagers tack on simpler scripts for system-managed pips.

[Ivoz]

-1 on deprecating pip, +1 on the others.
I want to expand on some framing thoughts I have, that might help for discussing what is such a large issue. Skip to the break if you just want to read my thoughts on solutions.

For a start, you want to look at why we have this problem, and is it analogous to anyone else? It comes from having and allowing multiple pythons on the one running system. So system package managers do not have this problem, because for instance, you don't (can't) run debian jessie and debian wheezy live at the same time on one system; so it doesn't need to manage a libreoffice3.5 and libreoffice4.3 for example. However many other language package managers start having to deal with the same problem as pip's when they too have multiple versions installed. As @erikrose mentions, even python itself already runs into this issue on deciding on what python now is when more than one is installed.

I also want to look at the issue from the POV of majority python users. Note that this has become a real pain point mostly (or more-so) for people with more than 2 pythons installed. Otherwise pip would work, or simply pip2 and pip3 (and there probably wouldn't be enough inertia for everyone to start discussing). Beyond that it starts getting really complicated. But I'd believe most python users are happy using just one python. Even if their system somehow gives them 2 at some point, or they manage to install multiple, instead of upgrading / (removing previous and installing a new one) - if they got things right, they'd be fine with only one python going, and by extension, that python's pip. For all of these users suddenly taking away pip makes absolutely no sense and is just painful.

The other big source of pain is when someone merely tries to install a new python over a previous existing one, but the story for the entire environment being migrated to the new one (or "taking over the old one") isn't there. In that case I want to make the distinction that this is the install story's fault, not the existence of multiple pythons. For instance someone hoping to "install a new python!" but not getting it on their path. Even uninstalling the old python, might do nothing about giving them the environment they want (the new python on their path).

Also note that while the number of users collecting problems with managing pips may be small, their complaints are the only ones heard. I'd venture that their opinions are probably the majority on this discussion as well (because they're the ones with the issue). The silent majority doesn't care until things get changed for them, but we should still look to represent their use case fairly. Not of course, that those complaints therefore can't be valid.

---

Now with that in mind, here's solutions I like:

• clearly pip<versionstuff> doesn't scale well at all as a solution. So I'm in agreement for removing it. Most of the time its better to wait for a decent solution (if it exists) than try and keep going with one that creates as many issues as it solves.

• Stop replacing pip (or other pip2s or pip3s, even) without asking. Even system package managers do this. They ask beforehand. So should we. This way at the very least, the user sees straight away that there's an issue and perhaps can make a decision for themselves what they want pip to be. One can do a lot with this - look at who's python installation the existing pip comes from - its the same one as the current pip trying to install itself, then this could be fine. Otherwise make the user say --yes or --replace or answer Y to a prompt. Note that this could help with the same problem with other userland-programs-that-come-from-pypi. Make sure the user wants the executable script replaced. Even if this means we have to wait a long time to tell people that they might have to interact after calling pip install by default (for scripted uses of pip), and give them time to add --replace-scripts (or w/e) to their callouts, so be it.

• Start outputting some information about where things come from in an install! This could solve a lot of issues straight away. If I install a package with pip, pip doesn't speak about

  • what python it's on/from
  • what version it is, where its installed
  • where it is installing the new package(s)

  This would all be super useful information to know. It will immediately show me if the pip I'm calling on the command line is not the actual pip that I want, which is what tricks a LOT of users. It will also show me I'm installing for the right python and into the right site-packages.

I especially believe implementing the last two points above, would remove a lot of average user-problems in relation to this issue. In many cases they would be empowered to know the problem and solution themselves.

[pfmoore]

I still think removing pipX and pipX.Y would be worthwhile. I'm ambivalent about the pip command itself. It is convenient, when it does the right thing. And I've never had any problems with it myself. But it is also the cause of a lot of "why didn't pip do the right thing" questions.

I think my position is that I don't have the guts to do it myself, but I'd be fine if someone else wants to do it. As far as abstract "churn budget" questions go, I simply don't think we have the information to make that call objectively - neither the impacts nor the benefits seem sufficiently measurable to me.

Is there anywhere in the docs where we still mention these commands? Maybe all we should do is add a note to the docs saying:

When installed, pip creates executables "pip", "pipX" and "pipX.Y" (where X.Y is the python version). These are no longer recommended or supported, and are only present for backward compatibility purposes. Users should always use the python -m pip command, as documented in this guide. Projects should never recommend the legacy commands.

While we're at it, we should definitely get PyPI changed:

[dstufft]

FWIW, I still think that python -m pip is the wrong way to go about things, and we should instead add -p to pip and allow it to target other environments. We could then in theory enable people not to have 12742345 different copies of pip installed all different versions.

[pradyunsg]

I mean, if one of our primary concerns is that users get confused when plain pip does not match python / pipX does not match pythonX, we could just... start checking for that and warning about it -- eventually changing that to a failure?

@dstufft Where would the "central" pip installation live then? I mean, we could totally make pip into a zipapp -- I'd be fine with that -- but it's not clear to me how we'd migrate to something like that; such that they don't have 12 million pip copies on disk. :)

[pradyunsg]

This problem interfaces deeply with Linux distributions FWIW, so the soonest we'd actually see genuine improvements from any of this would be... like... 5 years from now. :)

[merwok]

There could be instructions to run python -m ensurepip, then pip install -U --user pip, with one-time step of adding $USER_BASE/bin to PATH.

[pradyunsg]

Users should always use the python -m pip command, as documented in this guide. Projects should never recommend the legacy commands.

While we're at it, we should definitely get PyPI changed:

Wait wait wait -- don't read OP -- see #3164 (comment) for the current proposal for this.

[pfmoore]

Wait wait wait -- don't read OP -- see #3164 (comment) for the current proposal for this.

Um, I thought I did...? In the comment you quoted I was trying to say shouldn't the PyPI banner suggest python -m pip install XXX rather than pip install XXX, if for no other reason than to match the recommendation in our docs?

[dstufft]

Where would the "central" pip installation live then? I mean, we could totally make pip into a zipapp -- I'd be fine with that -- but it's not clear to me how we'd migrate to something like that; such that they don't have 12 million pip copies on disk. :)

@pradyunsg I'll be honest in that I never had a 100% fully formed plan, but the way I have envisioned the transition going is that, at least at the start, we don't try to change the fact that there's a billion copies of pip laying around. We probably drop pipX and pipX.Y, leaving pip behind and the python -m pip support.

Add in the -p command to target a specific interpreter, which defaults such that it defaults to whatever the current active interpreter is. This will keep the current behavior working inside of virtual environments and outside, where python -m pip will continue to select the python that pip is running under unless -p was also used.

At that point we can just start socializing the use of -p. I'm not entirely sure of a good way to progrmatically push people towards it. We can probably do some heuristics inside of __main__ to try and determine when it's being used and differs from what pip would default to, to direct people to use -p instead.

Eventually at some point we could possibly drop python -m support, and support only pip, but that would likely be a long deprecation process.

That more or less fully handles every case where someone currently uses python -m to select what version of Python they're installing into. The other way that people select what version of Python they're installing into is explicitly using the full path to a particular pip command. This is most often used to target a virtual environment without activating it.

One solution is to just say that we don't support that anymore, deprecate it and push people to use the -p mechanism exclusively.

Another option is to have a small shim script that is basically just (untested):

import sys
import os

os.execvp("pip", sys.argv + ["-p", sys.executable])

Another thing to consider is the behavior of the default -p when a virtual environment is active, but pip insn't installed into that virtual environment. Ideally pip would just do the right thing in those cases, which could be handled by the above shim, but we could also test if python is a virtual environment and if so, default to that.

Another option would be to have it always default to python (or python3 maybe?) and have __main__ override the default to use the current Python.

There are a number of ways we could go about selecting what the default is when no -p is picked, and I think that's probably going to be the hardest, subtle thing to get correct with a change like this.

At some point yearsinto the future, we could even start shipping pip as a zip app (or a PyOxidizer app, which then wouldn't even require Python to be installed), and stop installing it by default into virtual environments. That would drastically reduce the number of pip's installed on a user's machine, at a minimum it would reduce it to just the ones installed globally (so on Windows, probably 1 per Python, on macOS maybe the same, on Linux probably just 1 total). We could maybe even be intelligent and try to reduce that down to 1 total.

I think this ultimately ends up way better off for pip's users. A big problem with python -m pip is the same problem with python -m venv, you end up getting different behavior depending on what Python you're targeting because it's typical to have different versions of pip in different environments. This causes end user confusion.

It also means things like creating virtual environments can be drastically sped up, because we no longer need to install pip into every single one of them, since pip can just sit outside of it and install into one.

It also reduces user annoyance on things like the upgrade warning where people swap around between virtual environemnts and end up getting the upgrade nag randomly.

It also means that, in theory, pip itself can narrow it's supported versions of Python easier. This would be because you end up with two "parts" of pip, the part that does most of the logic, then the part that exists just to shell into the target environment and get information out. These don't have to support the same set of Pythons, so you could say pip itself only targets the latest Python while it can target a wider array of Pythons. This probably wouldn't be practical without using PyOxidier or so to bundle Python itself with pip so not really a big thing, but just an example of something this would possibly unlock.

[pfmoore]

If we can make a -p argument work, then I'd be 100% in favour of that. I'd see that as a much better long-term solution.

In fact, why not leave this issue parked for now, and in the meantime work on getting -p working? We can revisit this question when we have -p as a working replacement/alternative.

[pradyunsg]

Coolieo then. That's a plan.

[warsaw]

A big problem with python -m pip is the same problem with python -m venv, you end up getting different behavior depending on what Python you're targeting because it's typical to have different versions of pip in different environments. This causes end user confusion.

I really don't want to remove -m. For $work essentially the only safe invocations of pip are python -m pip outside of a venv and pip inside a venv. The former may be both path and Python version qualified. Personally, I don't even think pip install --user is safe. AFAICT, homebrew only installs pip3 and pipX.Y but I won't use them.

[warsaw]

Just to clarify, removing python -m pip would break tons of code.

[uranusjr]

If I understand the discussion correctly I don’t think the intention is to remove -m, but more like we should develop a better solution than it since it is not that perfect.

[dstufft]

A big problem with python -m pip is the same problem with python -m venv, you end up getting different behavior depending on what Python you're targeting because it's typical to have different versions of pip in different environments. This causes end user confusion.

I really don't want to remove -m. For $work essentially the only safe invocations of pip are python -m pip outside of a venv and pip inside a venv. The former may be both path and Python version qualified. Personally, I don't even think pip install --user is safe. AFAICT, homebrew only installs pip3 and pipX.Y but I won't use them.

I don't think that -m is going to be removed any time soon, if at all, though I'm curious why you think -m would be safe and -p wouldn't be?

[warsaw]

Oh, just to be clear, I think -p would be safe. I'm just saying that today we view -m as the only safe way to do it (as opposed to calling the pip* CLIs). I'm definitely +1 on adding -p and -1 on removing -m. The latter would just cause unnecessary breakage in the ecosystem IMHO.

[dstufft]

Oh yea, gotcha.

I don't think we're looking to remove -m at the moment. If we successfully move enough people over to using -p I could see a possibility of doing a big shake up at some point where we start turning pip into a self contained application that bundles everything together, at which point -m may not work.. or maybe it would! Who knows! Right now i'd say removing -m is a hypothetical idea like Python 4, maybe? maybe not? who knows, not worth thinking about too hard as it's just a placeholder for some big shake up down the road.

I'm also not super active here so take whatever I say with a grain of salt :) I just happened to notice this thread and figured I'd mention what I thought should be done, and let folks still working on things regularly decide what they felt about it.

[pradyunsg]

FWIW, another major advantage of adding a -p option would be that we could use it for setting up build environments, speeding up how we set up build environments (it'll just be a clean venv, instead of a venv with pip) and as a consequence of that, no longer expose an import pip from the build environment.

[aisven]

I just had a 5 hour fight with problems mentioned in this Issue.

I am using:

• Miniconda with 1 virtual environment (next to the Miniconda base environment) created with conda with python 3.10 (let's call it myenv)
• Somewhere at the start I did conda install pip inside myenv
• (I think at some point I did a conda uninstall pip followed by conda install -n myenv python=3.10 pip I believe)
• Ubuntu on that particular machine
• system python happens to be 3.9 (not intending to use it in my user land)

Being inside the virtual environment myenv which again I created and activated with Miniconda,

now typing pip would execute a pip from ~/.local/bin/ which is not "tied" (whatever that means) to the python 3.10 but to a python 3.9 (the system python?). This led to unexpected results.

To work around this, I followed an advice found at https://adamj.eu/tech/2020/02/25/use-python-m-pip-everywhere/ and thus I added alias pip='python -m pip' to ~/.bashrc.

Not sure if this is a good solution, but for now I got my stuff working. I wanted to share my experience as well as that article.

[pfmoore]

Circling back round to this (thanks @pradyunsg for reminding me to re-read it, there's recent comments here I'd forgotten about).

1. I now have a possible plan for something like -p. See Add an option for pip to manage a different "environment" #11307¹.
2. We appear to have won the battle on switching people to python -m pip, based on this discussion. For better or worse, switching back to recommending the pip executable and deprecating python -m pip would be a really hard sell.
3. Nobody seems interested in the versioned executables any more.

Footnotes

1. Making -p mean "query the target Python and use its installation scheme" does the main part of the job. It doesn't emulate the way --user is layered on top of site-packages, but I'm not particularly interested in supporting that. ↩

[pfmoore]

One additional thought. The semantics of -p are not 100% clear to me. If we run a 64-bit Python on Windows, with -p pointing to a 32-bit interpreter, would you expect things like the supported wheel tags to be based on the target of -p? If so, then #11307 isn't sufficient (and indeed, I'm not sure how we'd approach this as it's essentially a form of cross-compilation support).

So on reflection, I'd prefer not to set misleading expectations by including -p in the (as yet unspecified) UI for #11307.

[merwok]

I am sure there’s a big group of experimented devs who consistently use virtual environments and run scripts in their simplest form, i.e. pip tox pytest ./manage.py and so on, never with python -m, without confusion or issues. After all, the rules followed by the shell/OS to resolve python are the same as the ones for pip.

Personally I look forward to managing a single pip with the zipapp method, but I will certainly have a symlink so that I’m still typing pip install in 10 years. I understand the reasons to emphasize python -m in docs, but please let scripts be scripts.

[sbidoul]

Now that pip has a --python option, and pip-less virtual environments are viable, the pip command still makes a lot sense.

[pfmoore]

Agreed, although I don't like the pip3 form. But I guess Unix users are used to having the 3 suffix by this point...

[wfjsw]

Does --python work on Windows? AFAIK pip.exe has a hardcoded Python path into the binary and it is not trivial to change that.