Allow installing stub packages in mypy's Python as a fallback

[ilevkivskyi]

Currently PEP 561 specifies that stub packages (like sqlalchemy-stubs) should be installed in the site packages of the Python version of the program that is being checked:

Type checkers that check a different Python version than the version they run on MUST find the type information in the site-packages/dist-packages of that Python version.

However, there is an important use case for stub packages that is incompatible with such path resolution scheme: single installation for stubs accompanied by a plugin. For example, all large frameworks like Django, NumPy, SQLAlchemy, etc. require both stubs and mypy plugins to properly type-check code using them. Ideally, a user should be able to just pip install them (stubs and mypy plugin) in one place, which is the place where mypy is installed.

What I propose is to add a sentence like (approximately): "If the package is not found in that version, it is searched in the version of Python where the type checker is running."

@ethanhs What do you think?

NOTE: Currently this accidentally works in mypy. I propose to make this an officially supported feature, and add proper support to mypy.

[ethanhs]

Hm, this makes me feel a bit uneasy. There are cases where this might not be possible (e.g. a Python 2 only dependency).

I would much rather have mypy look in the other Python executable for the plugin than have mypy break the model of "We look at the Python executable pointed at via python_executable or python_version for typed packages".

[gvanrossum]

That was my POV too, but Ivan's argument was that then you'd to execute two pip commands (using two different interpreters!) to install the typing support for such packages. That has much more of a chance of breaking, since it's likely that some kind of coordination between versions of the plugin and the stubs is needed.

[ethanhs]

@gvanrossum but if we search for plugins based on python_executable the package only needs to be installed in the correct Python install AFAICT?

[ilevkivskyi]

Also there seems to be some misunderstanding about what I am proposing:
a) I don't want to change anything about typed packages either inline or with stubs, since those contain actual runtime code, I am talking only about stub packages (number 3 in PEP 561 resolution order).
B) I don't want it as a default behavior, we still look in the dedicated version, but if we didn't find it, we fall back to the current version of Python.

[ilevkivskyi]

@ethanhs
A plugin is actual runtime code that will be executed by the interpreter, it may be even syntactically invalid for Python 2. On the other hand, stub files are essentially just text.

[ethanhs]

I suppose I am okay with this. It makes me feel uneasy, but I can't disagree that it has good ergonomic benefits.