17.0.4: LLVM uses deprecated `distutils` python module

[kloczek]

distutils is already deprecated https://peps.python.org/pep-0632/ and should no longer be used.

[tkloczko@devel-g2v lldb-13.0.1.src]$ grep -r distutils
CMakeLists.txt:          -c "import distutils.sysconfig; print(distutils.sysconfig.get_python_lib(True, False, ''))"
packages/Python/lldbsuite/test/decorators.py:from distutils.version import LooseVersion
packages/Python/lldbsuite/test/lldbtest.py:from distutils.version import LooseVersion
packages/Python/lldbsuite/test/lldbtest.py:import distutils.spawn
packages/Python/lldbsuite/test/lldbtest.py:        path = distutils.spawn.find_executable("clang", lldb_dir)
test/API/tools/lldb-server/TestAppleSimulatorOSType.py:            from distutils.version import LooseVersion
test/Shell/helper/build.py:        from distutils.version import StrictVersion
test/Shell/helper/build.py:            from distutils.version import LooseVersion
test/Shell/lit.cfg.py:from distutils.spawn import find_executable
third_party/Python/module/pexpect-4.6/setup.py:from distutils.core import setup

[llvmbot]

@llvm/issue-subscribers-lldb

[kloczek]

Just checked 14.0.0 and still I see used distutils

[tkloczko@devel-g2v lldb-14.0.0.src]$ grep -r distutils
packages/Python/lldbsuite/test/decorators.py:from distutils.version import LooseVersion
packages/Python/lldbsuite/test/lldbtest.py:from distutils.version import LooseVersion
packages/Python/lldbsuite/test/lldbtest.py:import distutils.spawn
packages/Python/lldbsuite/test/lldbtest.py:        path = distutils.spawn.find_executable("clang", lldb_dir)
test/API/tools/lldb-server/TestAppleSimulatorOSType.py:            from distutils.version import LooseVersion
test/Shell/helper/build.py:        from distutils.version import StrictVersion
test/Shell/helper/build.py:            from distutils.version import LooseVersion
test/Shell/lit.cfg.py:from distutils.spawn import find_executable
third_party/Python/module/pexpect-4.6/setup.py:from distutils.core import setup

[DavidSpickett]

To remove find_executable usage: https://reviews.llvm.org/D124601

[DavidSpickett]

https://reviews.llvm.org/D124670 is what switching to packaging for the version handling would look like. It would need us to make a copy of that in thirdparty/ (possible that we could pick out that one bit, haven't looked).

Needs testing on all platforms though, I'm not sure LegacyVersion matches LooseVersion in all the ways we expect.

Then we'd be left with:

$ grep -r distutils
third_party/Python/module/pexpect-4.6/setup.py:from distutils.core import setup

Which could be updated if we want. I think it is used just for the test suite not any modules you would find in an install.

[kloczek]

Only confirmation: lldb seems is no longer affected.
Feel free to close.

[DavidSpickett]

No longer affected how exactly. In that we've removed all the distutils usage that isn't related just to testing?

I'd keep this open just for removing distutils usage in general, testing or otherwise.

[kloczek]

Current pexpect no longer uses distutils.
I think that the best would be drop bundling that module code.

[DavidSpickett]

Ok I understand now.

I think (in that I've never seen it written down) the policy for lldb is to include copies of third party python packages. So to be clear by "updated" I mean getting a copy of a newer version of pexpect, not editing the current one to remove distutils.

When I get a chance I'll find out what that process looks like. We do have to be careful about API changes there.

[labath]

It's not just lldb -- llvm in general likes to absorb things into its repository. Although I don't really agree with the practice, I can't deny the practicality of not having to install a bunch of random packages in order to run tests.

On a practical note, our copy of pexpect was updated fairly recently, and we don't make any modifications to it, so pulling in a new version should be fairly straight-forward.

[mgorny]

@DavidSpickett, would you by any chance be interested in doing the find_executable() switch in other LLVM projects as well?

I'm wondering about the best replacement for LooseVersion/StrictVersion. One problem is that modern packaging no longer supplies LooseVersions, so unless we can strict to the strict variant (i.e. the one requiring conformance to PEP 440), we may end up having to parse the versions manually.

That said, I suppose most of LooseVersion uses are actually trivial.