Fix the install scheme to `posix_prefix` when `--prefix` is passed

[frostming]

What's the problem this feature will solve?

On Debian systems, due to the sysconfig patch, sysconfig.get_preferred_scheme('prefix') will return posix_local, which is also the default scheme. This causes problem when using --prefix <path> with pip install, where packages are installed to <prefix_path>/local/lib/pythonX.Y/dist-packages. This is apparently wrong.

Describe the solution you'd like

Add a special case to pip._internal.locations._sysconfig:get_scheme(), when posix_local is found in INSTALL_SCHEMES and prefix is not None, rewrite the scheme as posix_prefix.

Alternative Solutions

I am not sure if it is better to report it to Debian, but IMO it makes more sense to fix it in pip to be more robust.

Additional context

• https://salsa.debian.org/cpython-team/python3/-/blob/master/debian/patches/sysconfig-debian-schemes.diff
• python3.10+: global pip install layout does not match debian's due to sysconfig changes deadsnakes/issues#182
• Debian bug report

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

What is the behaviour of the distutils backend?

[pradyunsg]

I am not sure if it is better to report it to Debian, but IMO it makes more sense to fix it in pip to be more robust.

If this behaviour originates due to a Debian patch, it is usually a good idea to file an issue with them. Please file a bug with Debian, with reportbug python3-pip (their reporting documentation). You can link to this issue in your bug report.

[frostming]

What is the behaviour of the distutils backend?

They are different:

>>> from pip._internal.locations import _distutils, _sysconfig
>>> _sysconfig.get_scheme('click', prefix='myprefix').purelib
'myprefix/local/lib/python3.10/dist-packages'
>>> _distutils.get_scheme('click', prefix='myprefix').purelib
'myprefix/lib/python3.10/site-packages'

If this behaviour originates due to a Debian patch, it is usually a good idea to file an issue with them. Please file a bug with Debian, with reportbug python3-pip (their reporting documentation). You can link to this issue in your bug report.

Thanks for the guidance but I am not able to do so now(some problem with reportbug).

[uranusjr]

Hmm yeah at least they should match. I don’t think Debian can handle this though; the --prefix value is not passed to sysconfig so they have no way to magically choose a scheme based on it. We’ll likely need to somehow handle this in pip, or raise an issue to CPython to further enhance the sysconfig API.

[speth]

I'm seeing a similar problem on Fedora 37 (Rawhide) while trying to package a Python module (see Cantera/cantera#1233), using both the Fedora-packaged version of pip or pip installed from PyPI, so I don't think that this is only a Debian problem. In both cases this is using pip 22.0.4. There are some differences, though, compared to the behavior reported on Debian -- I see the local directory being incorrectly inserted through both routes:

>>> from pip._internal.locations import _distutils, _sysconfig
>>> _sysconfig.get_scheme('click', prefix='myprefix').purelib
'myprefix/local/lib/python3.10/site-packages'
>>> _distutils.get_scheme('click', prefix='myprefix').purelib
'myprefix/local/lib/python3.10/site-packages'

[uranusjr]

If it's injected for both implementations, this is not a bug but an intended behaviour. You need to check with Red Hat to understand their intention behind this and resolve the issue there, pip cannot change it here.

[henryptung]

Not to disrupt the conversation, but isn't the prefix install scheme fairly-explicitly specified in https://docs.python.org/3/install/index.html#alternate-installation-unix-the-prefix-scheme? Is this behavior changing, and is that documented somewhere?