pre-commit fails to initialize due to python 3.8 dependency

[soxofaan]

I wrestled with this pre-commit issue when trying to contribute a PR yesterday and could not find pointers in the issues or discussions, so I'm at least dumping my findings here. To be discussed if/how this has to be documented properly somewhere.

With #12747 (end of August) the pre-commit run explicitly depends on python 3.8:

pytest/.pre-commit-config.yaml

Lines 33 to 36 in a14c718

	- id: mypy
	files: ^(src/|testing/|scripts/)
	args: []
	language_version: "3.8"

(python 3.8 is EOL since this week, but that's not the actual problem here, I would have had the same issue with 3.9)

Running pre-commit failed on me during pre-commit update, e.g.:

$ pre-commit run
...
[INFO] Installing environment for https://github.com/pre-commit/mirrors-mypy.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/home/johndoe/.local/pipx/venvs/pre-commit/bin/python', '-mvirtualenv', '/home/johndoe/.cache/pre-commit/repodljwkw2k/py_env-3.8', '-p', '3.8')
return code: 1
stdout:
    RuntimeError: failed to find interpreter for Builtin discover of python_spec='3.8'
stderr: (none)
Check the log at /home/johndoe/.cache/pre-commit/pre-commit.log

After some rabbit hole crawling I figured out that I have to make sure that virtualenv (used by pre-commit) is able to resolve "3.8" to an actual interpreter on my system.

from https://virtualenv.pypa.io/en/latest/user_guide.html#python-discovery:

Given the specifier virtualenv will apply the following strategy to discover/find the system executable:
...

• Try to discover a matching python executable within the folders enumerated on the PATH environment variable. In this case we’ll try to find an executable that has a name roughly similar to the specification (for exact logic, please see the implementation code).

I'm using asdf (which uses pyenv under the hood) and luckily had still a 3.8 version around. I found the direct (non-shimmed) path to the executable with

$ (asdf shell python 3.8.15; python3.8 -c "import sys;print(sys.executable)")
/home/johndoe/.asdf/installs/python/3.8.15/bin/python3.8

And appended the containing folder to my PATH when triggering the pre-commit update:

$ export PATH=$PATH:/home/johndoe/.asdf/installs/python/3.8.15/bin/
$ pre-commit run

pre-commit then could successfully update/install the hooks, and I could commit again

[Pierre-Sassoulas]

Python 3.8 has been EOL for a few days, I think we could / should drop it.

[The-Compiler]

I don't think we should hard-code the Python version used to run mypy at all. This was done in #12747 so that mypy catches old typing usage that doesn't exist in newer versions - but mypy already supports setting that independently from the version used to run it.

@nicoddemus was this a deliberate decision, or were you just not aware of the mypy setting? To be fair I didn't test if it works with the scenario in #12744, but I'd expect it to.

[Pierre-Sassoulas]

Opened #12869 to test in CI

[nicoddemus]

@nicoddemus was this a deliberate decision, or were you just not aware of the mypy setting?

I was not aware at all, passing the argument is of course much better! 👍

[nicoddemus]

Fixed by #12869.