Adopting "working" scheme for every run

[pradyunsg]

Just carrying over my idea in #1056 (comment), for proper dedicated discussion.

AFAIK, there are 3 possible schemes for packages:

• "system" - for system/global packages (--system)
• "user" - for packages installed in user space (--user)
• "local" - for virtualenv packages (--local)

pip enforces a "working" scheme on every run. Outside a virtualenv, the default working scheme would be "system" (It should really be "user", that's another issue #1668). Inside a virtualenv, the default working scheme should be "local". Passing --system or --user or --local overrides the working scheme.

Only packages in the same scheme as the working scheme can be modified. By modifying, I mean installing or uninstalling a package. Trying to modify a package in a different scheme is not allowed and pip would print a message and error out.

So, modifying a package in system scheme with a "user" working scheme is not allowed. Nor is modifing a package in user scheme with a "system" working scheme. Niether are the other permutations with "local".

I think this results in a pretty simple behaviour model.

As @ncoghlan pointed out, this would need some logic to understand that user installs shadow system ones. Also, I would slightly change this to spell it like --scheme {global,user,venv} because I like how this signifies exclusiveness of the behaviours better than plain flags do and is consistent with some internal configuration stuff.

Additionally, while doing this:

• preparatory work of Default to --user  #1668 can also be done, meaning that defaulting to user would just be flipping the default (something like what we did for --upgrade-strategy).
• addresses pip install -U does not remember whether a package was installed with --user #1056 by requiring the user to specify the scheme that they are working with.

/ping @dstufft @pfmoore

[pradyunsg]

I would slightly change this to spell it like --scheme {global,user,venv}

Oh, and this would let you set a default scheme for yourself using the configuration toolchain.

[pradyunsg]

@dstufft @pfmoore I would like to know what you think about this. :)

[pfmoore]

In principle this sounds like a reasonable thing to do - but I'm afraid I've got some personal priorities at the moment that mean I don't really have time to think through the implications.

So count me as in favour in principle, but not really able to provide a detailed review at the moment, sorry.

[pradyunsg]

Cool. Thanks! :)

[ncoghlan]

Just noting that I briefly thought that --scheme may not be a good name for this option, due to the potential collision with the concept of installation schemes in sysconfig: https://docs.python.org/3/library/sysconfig.html#installation-paths

However, I subsequently realised that these uses are actually the same use case - the new pip level option is just a helper to select the desired scheme without having to specify the exact platform appropriate scheme name as defined in sysconfig.

[pradyunsg]

Linking to #2418 since I somehow always forget there was an attempt at making --user behaviour default.

---

concept of installation schemes in sysconfig

This is, actually, nice that the names here and there match. :)

[tuukkamustonen]

This came up in #4809 where I suggested being able to pip list only packges related to one scope (now it combines both system and user packages, making it impossible to say which package comes from where).