Default to --user

[dstufft]

When pip is installed on a system that has an OS Python install there is currently a problem where pip install foo will throw an error because it doesn't have file permissions. This causes people to instead run sudo pip install foo which globally installs to the system Python. This creates an issue where people are using pip to manage system level packages when they should likely be using the system package manager.

So my intention is that pip should default to --user however there are a few sticking points with this:

• How does this interact with Windows? Does this make sense there?
• How does this interact with altinstall'd Pythons? Specially such as are installed with tools like https://github.com/yyuu/pyenv
• What do we do for when people do invoke pip as root? Installing into /root/.local/ does not seem very useful.
• What does this mean for get-pip and the pypa install instructions?
• ~/.local/bin is not on many people's $PATH, is there anything that can be done about this?
• --user installs lack precedence to global easy_install'd packages, which can be quite unexpected.

There are a number of issues that are relevant here: #624 #1443 #1153 #1500 #1051

/cc @ncoghlan

[qwcode]

What does this mean for the pypa install instructions?

currently the mention of root or Administrator access is in a footnote, so the instructions would fundamentally stay the same (assuming get-pip automatically defaulted to --user as well). the change would be to explain that they will end up with installs that only apply to their current user.

regardless of the --user change, the current instructions need to explain that some distributions like debian, are disabling ensurepip, so they shouldn't go looking for it as a way to get pip.

[dstufft]

Long term I'm pretty sure they are going to keep it enabled, they are just figuring out how to do it.

[qwcode]

although this issue is described as being about where pip should manage packages by default, almost more important (to me at least) is that this is indirectly about where get-pip will place pip by default.

[qwcode]

How does this interact with altinstall'd Pythons?
Specially such as are installed with tools like https://github.com/yyuu/pyenv

the user scheme is .local/lib/pythonX.Y/site-packages, so if you're managing multiple pythons with the same major/minor version, you could end up with a mess I guess.

for tools like that, that manage real pythons, pinning pip to global mode would make sense for those.

[dstufft]

A relevant issue on the redhat/fedora bug tracker https://bugzilla.redhat.com/show_bug.cgi?id=662034#c10

[dstufft]

It looks like for Fedora/RedHat PATH=$PATH:$HOME/.local/bin:$HOME/bin is already in their /etc/skel/.bash_profile. This would make stuff installed with --user still show up by default, at least for bash users (default shell). Maybe other distros already have this?

[pfmoore]

There's nothing specially different about Windows that I know of. But we've only just got the standard Python directory onto people's PATH, I don't expect that getting a per-user scripts directory on is going to be particularly easy - and there may be technical difficulties as well, I'm not sure putting a user-level environment variable like %LOCALAPPDATA% into the system-level %PATH% even works :-(

I'm against rushing this. I would prefer waiting till we've had more experience with user installs and have managed to iron out the issues first. I know that's a chicken and egg problem, but I don't see rushing into a change as helping.

Also, with my Windows hat on, I'd have to say that making it harder for Unix users who haven't yet worked out that careless use of sudo is bad to do stupid things, isn't really that good a justification...

I'm assuming that this would only apply when running pip from a system Python.

[dstufft]

Well the different thing on Windows is there isn't a system provided Python like there is on *nix that also comes with system provided Python packages :)

I'm not particularly wanting to rush this either. I just wanted to start the discussion.

This doesn't really affect people who type sudo pip install as I think installing something as root should still go to the system site packages by default. This would be to guide people towards using --user when running as an unprivileged user. Right now what you get is that pip install as a regular user just bombs out with a permission error. Even if we improve that message to bomb out and suggest --user that's still not the greatest UX.

In talking to one of the Fedora people, I think the way it may make to do this is to implement a permissions check. If I have permission to write to the site-packages directory then I am a privledged user and pip installs to the global Python, If I do not then I am an unprivileged user and it installs to --user.

[ncoghlan]

As far as Windows goes, we usually duck the problem because we install Python to a non-privileged directory by default, so pip doesn't trigger UAC when installing globally. If someone changes that to install into Program Files instead, then UAC will trigger when they run pip (which is also OK). I'm not sure what happens if they select a per-user install in the installer.

As Paul notes, PATH on Windows doesn't include the user scripts directory yet (just the global one), so that's definitely worth taking into account.

I don't see any major barriers on the POSIX side though. How does this UX idea sound: for wheel based installs, check for permissions on the installation target directory, and if the user doesn't have write permissions, put up a prompt asking if they would like to do a --user install instead? A new "--global" or "--system" flag would force the "no" answer.

And then at some point in the future, we could skip the prompt and simply assume --user if the permissions weren't right for a global install.

[dstufft]

A prompt (except in the case of --no-input) probably makes sense for the transition step. I think checking permissions should make things work for Windows too since by default the location is in a spot where people have permissions on and so the only time they'd hit this would be if they are using a systems admin provided Python that explicitly didn't give them permission.

Essentially this makes our failure mode much better.

[qwcode]

making it harder for Unix users who haven't yet worked out that careless use of sudo
is bad to do stupid things, isn't really that good a justification..

sudo and security is really a secondary issue IMO.
this is primarily about preventing the conflict between OS packaging and pip in the system python.
Currently, linux users are often placed in an impossible situation.

1. OS Mandate: 'Use the OS pkg mgr; Don't infect your system with roque pip-installed packages (including a get-pip'd pip)'
2. Reality: "The OS packages (including pip) are too old, and I can't upgrade to the experimental release of the distro, just to get upgrades of some package. I'm going rogue...."

[qwcode]

another baby step is to document (assuming we test it) that you can get-pip.py --user, and have the PUG mention --user as a possibility for the install of virtualenv (and wheel, and twine too, if not in a virtualenv)

[ncoghlan]

FWIW, distro vendors also consider the status quo a problem and are looking at various ways to improve the tools for doing selective upgrades without impacting core OS components. Still worth us tackling from the upstream side though, since those efforts are at various stages of maturity and we want a consistent cross-platform solution for end users.

[pfmoore]

I have no problem with changing things to help co-operate with distros on Unix; if defaulting to --user on Unix and not on Windows is acceptable, that's fine with me. I just don't see switching from a bad experience on Unix to a bad experience on Windows as a good trade-off. And I'm yet to be convinced that --user on Windows is ready for prime time.

Note that triggering UAC when running pip is bad on Windows, because what it actually means is that pip won't run except in an elevated console window. It does not mean that pip users get a nice prompt which they can say "OK" to, unfortunately, that's only how GUI apps work...