pip uninstall calls normalize_path excessively, hurting performance on Windows

[DefaultRyan]

Description

When using pip's uninstall, either via pip uninstall or pip install --force-reinstall, I found that UninstallPathSet was generating thousands of calls to normalize_path(), the vast majority of which are being called on identical paths.

For instance, when running pip install --force-reinstall --no-deps .tox/.tmp/package/1/pip-23.1.dev0.tar.gz, I saw ~4000 calls to normalize_path().

• ~2500 were in calls to UninstallPathSet.add(), when normalizing the "head" of a file path. Since the vast majority of directories have several files in them, this meant we were repeating the path normalization of the "head" portion of the path.
• ~1500 were from calls to UninstallPathSet._permitted(), which calls is_local(). All but one of these were redundant, because is_local() calls normalize_path(sys.prefix), and sys.prefix shouldn't be changing while executing an uninstall.

In my testing on Windows, normalize_path() overwhelmingly spent its time calling realpath(). On Linux-based filesystems, realpath() might be a fast function, but it's more expensive on Windows, and all these calls were adding nearly 1 second of time to the uninstall.

Expected behavior

I would expect pip's uninstall logic to not normalize the same paths hundreds/thousands of times during the scope of a single uninstall/reinstall operation, degrading performance.

Instead, I would expect to see some level of caching as to not harm performance disproportionately on Windows platforms.

pip version

23.1

Python version

3.11

OS

Windows 11

How to Reproduce

1. Download the pip source distrubtion from https://pypi.org/project/pip/#files
2. Create a virtual environment
3. From that virtual environment run: python -m pip install --force-reinstall <path to the tar.gz you downloaded
4. If you perform these steps with Python profiling enabled, you'll see approximately 4000 calls made to normalize_path().

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[ichard26]

This can be closed by #11868 which was merged a while ago :D