Description
This is an accompanying issue to #10151 on the distutils-sysconfig switch, so discussions on concrete solutions are not drowned by the reports.
I categorised the reports we gathered in the past day:
https://gist.github.com/uranusjr/c03c2c452ee8faa11db996c6b5a3999d
(Some reports uncover multiple issues so a few rows are duplicated under different categories.)
From what I can tell, there are four main problematic environments:
On macOS, pip install --user in a framework build
I think I made a mistake in #9844 and the skip_osx_framework_user_special_case check isn’t working properly. This should be easy to fix.
PR ready in #10215.
On MSYS, sysconfig’s lib path does not contain site-packages
This is… I don’t know, extremely weird. Sounds like a bug in msys? There’s only one ill-formatted report about this, and the reporter did not respond to my request to clarify. So I think I’ll just ignore this unless someone else comes up with a better report.
On Linux, sudo pip install -U pip a system-provided pip and then attempt to sudo pip install a package under the default prefix
Account for ~50% of the reports. Since Linux distros patch both Python and pip to return different paths for system and user-land tools, a user-installed upstream pip have trouble understanding the scheme. I already told Debian devs about this and IIRC they already updated the patches to accomodate the sysconfig switch, but in any case, since this can only be solved by distro devs, so I’m going to just try to detect the platform and suppress the warning. I think this is doable by checking whether distutils contains deb_system and unix_local on Debian and derivatives. Red Hat derivatives are more difficult to detect, I think the only way is to check whether platlib and purelib are in different locations (Red Hat patches platlib to use lib64 instead of lib).
PR ready as a part of #10217.
On Homebrew and Linuxbrew, pip install into the Brew-managed system prefix
Homebrew uses a distutils.cfg file to overwrite the install scheme (similar patch for Linuxbrew). Since distutils.cfg is going to be deprecated and eventually removed with distutils itself, Homebrew needs to re-think how they manage the install schemes. I already raised this issue to Homebrew. On pip’s side, since the entire distutils config file thing is going away, we should raise a deprecation warning when it is detected (and suppress the normal location mismatch warning). I’m thinking something like
DEPRECATION: Overwriting package installation paths with distutils configuration files is deprecated
and will be removed in Python 3.12. If you are seeing this under Homebrew or Linuxbrew, please
see https://github.com/Homebrew/homebrew-core/issues/76621
Linuxbrew seems to have one additional setup that Homebrew does not have: #10151 (comment) But the root cause is likely the same (maybe just version differences in Brew?)
PR ready in #10218.
On Linux (potentially all POSIX), pip install into any site on a Python with pymalloc enabled
This is actually a bug in sysconfig. sysconfig’s purelib and platlib values fail to include sys.abiflag (the m thing in python3.6m if you’re not familiar with the topic; there are three suffixes in total IIRC). But Python removed those suffixes in 3.8 (I think) so there’s no point fixing sysconfig now (again, we are only going to switch to sysconfig when running on 3.10+). We should add some special cases to suppress the warning.
PR ready as a part of #10217.