Complete type analysis of variadic types

[ilevkivskyi]

This PR closes the first part of support for TypeVarTuple: the "static" analysis of types (of course everything is static in mypy, but some parts are more static): semanal/typeanal, expand_type(), map_instance_to_supertype(), erase_type() (things that precede and/or form foundation for type inference and subtyping). This one was quite tricky, supporting unpacks of forward references required some thinking.

What is included in this PR:

• Moving argument count validation from semanal_typeargs to typeanal. In one of previous PRs I mentioned that get_proper_type() may be called during semantic analysis causing troubles if we have invalid aliases. So we need to move validation to early stage. For instances, this is not required, but I strongly prefer keeping instances and aliases similar. And ideally at some point we can combine the logic, since it gets more and more similar. At some point we may want to prohibit using get_proper_type() during semantic analysis, but I don't want to block TypeVarTuple support on this, since this may be a significant refactoring.
• Fixing map_instance_to_supertype() and erase_type(). These two are straightforward, we either use expand_type() logic directly (by calling it), or following the same logic.
• Few simplifications in expandtype and typeops following previous normalizations of representation, unless there is a flaw in my logic, removed branches should be all dead code.
• Allow (only fixed) unpacks in argument lists for non-variadic types. They were prohibited for no good reason.
• (Somewhat limited) support for forward references in unpacks. As I mentioned this one is tricky because of how forward references are represented. Usually they follow either a life cycle like: Any -> <known type>, or <Any> -> <placeholder> -> <known type> (second one is relatively rare and usually only appears for potentially recursive things like base classes or type alias targets). It looks like <placeholder> can never appear as a valid unpack target, I don't have a proof for this, but I was not able to trigger this, so I am not handling it (possible downside is that there may be extra errors about invalid argument count for invalid unpack targets). If I am wrong and this can happen in some valid cases, we can add handling for unpacks of placeholders later. Currently, the handling for Any stage of forward references is following: if we detect it, we simply create a dummy valid alias or instance. This logic should work for the same reason having plain Any worked in the first place (and why all tests pass if we delete visit_placeholder_type()): because (almost) each time we analyze a type, it is either already complete, or we analyze it from scratch, i.e. we call expr_to_unanalyzed_type(), then visit_unbound_type() etc. We almost never store "partially analyzed" types (there are guards against incomplete references and placeholders in annotations), and when we do, it is done in a controlled way that guarantees a type will be re-analyzed again. Since this is such a tricky subject, I didn't add any complex logic to support more tricky use cases (like multiple forward references to fixed unpacks in single list). I propose that we release this, and then see what kind of bug reports we will get.
• Additional validation for type arguments position to ensure that TypeVarTuples are never split. Total count is not enough to ban case where we have type variables [T, *Ts, S, U] and arguments [int, int, *Us, int]. We need to explicitly ensure that actual suffix and prefix are longer or equal to formal ones. Such splitting would be very hard to support, and is explicitly banned by the PEP.
• Few minor cleanups.

Some random comments:

• It is tricky to preserve valid parts of type arguments, if there is an argument count error involving an unpack. So after such error I simply set all arguments to Any (or *tuple[Any, ...] when needed).
• I know there is some code duplication. I tried to factor it away, but it turned out non-trivial. I may do some de-duplication pass after everything is done, and it is easier to see the big picture.
• Type applications (i.e. when we have A[int, int] in runtime context) are wild west currently. I decided to postpone variadic support for them to a separate PR, because there is already some support (we will just need to handle edge cases and more error conditions) and I wanted minimize size of this PR.
• Something I wanted to mention in one of previous PRs but forgot: Long time ago I proposed to normalize away type aliases inside Unpack, but I abandoned this idea, it doesn't really give us any benefits.

As I said, this is the last PR for the "static part", in the next PR I will work on fixing subtyping and inference for variadic instances. And then will continue with remaining items I mentioned in my master plan in #15924

Fixes #15978

[mehdigmira]

FYI, this PR seems to be introducing some regressions, see example below

from typing import Generic
from typing_extensions import TypeVarTuple
from typing import Unpack

Ts = TypeVarTuple("Ts")


class Array(Generic[Unpack[Ts]]):
    def _close(self) -> None:
        ...

    def close(self) -> None:
        self._close()  # error: Invalid self argument "Array[Unpack[Ts]]" to attribute function "_close" with type "Callable[[Array[Unpack[Ts]]], None]"

[ilevkivskyi]

@mehdigmira This is kind of expected. Subtyping is quite broken now (especially for instances), and this PR exposes this. As I mentioned in PR description, I am not going to work on subtyping in this PR, I am going to fix it in the next one(s).

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

[JukkaL]

Looks good! This fixes a bunch of important issues with TypeVarTuples.

Maybe reword the beginning of the summary, since the initial "Fixes #..." kind of downplays the impact of this PR, as it fixes other issues as well.

[ilevkivskyi]

@JukkaL OK 👍 I updated the PR description. (Btw the next part is almost ready, I will make a PR likely tomorrow.)