Invalid infer generic type for contravariant TypeVar inside function call

[unknownlighter]

from typing import Generic, TypeVar


T_contra = TypeVar('T_contra', contravariant=True)


class Animal:
    pass


class Cat(Animal):
    pass


class AnimalBox(Generic[T_contra]):
    def __init__(self, animal: T_contra):
        pass


def print_box(a: AnimalBox[Cat]):
    pass


animal = Animal()

print_box(AnimalBox(animal))  # error: Argument 1 to "AnimalBox" has incompatible type "Animal"; expected "Cat"

box = AnimalBox(animal)
print_box(box)  # OK

Reproduces only on contravariants. Covariants seems to be OK.
Tested on: mypy-0.680+dev.69eaf88a0b03e9f491cc22534c2d30f1d21832b2

[unknownlighter]

Seems to be invariant TypeVar also affected. Inside function call they works as covariants

from typing import Generic, TypeVar


T = TypeVar('T')


class Animal:
    pass


class Cat(Animal):
    pass


class UglyCat(Cat):
    pass


class AnimalBox(Generic[T]):
    def __init__(self, animal: T):
        pass


def print_box(a: AnimalBox[Cat]):
    pass


print_box(AnimalBox(Animal()))  # error: Argument 1 to "AnimalBox" has incompatible type "Animal"; expected "Cat"

print_box(AnimalBox(UglyCat()))  # OK, but expected error

box = AnimalBox(UglyCat())
print_box(box)  # works as excepted: error: Argument 1 to "print_box" has incompatible type "AnimalBox[UglyCat]"; expected "AnimalBox[Cat]"

[Tishka17]

The same is for invariants.

[ilevkivskyi]

It looks like here are several misconceptions about what variance is, and about how inference works. Let me clarify the situation:

1. Variance is not a property of a type variable -- this is just an (unfortunate) historical artifact. Variance is a property of a generic class. The only thing that one can conclude from your first example is that AnimalBox[Animal] is a subtype of AnimalBox[Cat].
2. Type inference doesn't affect type safety. In other words, there can't be false negatives because of bad inference (unless you infer Any somewhere, but mypy should never do this). Consider your second example, you say:

   print_box(AnimalBox(UglyCat()))  # OK, but expected error

   But this call is type-safe. To understand why, consider this call print_box(AnimalBox[Cat](UglyCat())) and then take into account that at runtime type application essentially returns the same class object.
3. These misunderstandings may be amplified by the way how inference works in mypy: currently mypy always uses external (return) type context first. To see this, consider this code:

   def func(x: T) -> T: ...
   x: int
   y: str = func(x)

   In this call, there are two options for an error: incompatible types in assignment, or incompatible argument type. Trying it in mypy you can see it is the latter. This is because mypy infers the type argument T of func using external context and then type checks the call. It may be surprising, but in practice, using external context typically generates better error messages.
4. The known downside of such scheme is that it may infer unexpected types for nested calls, we have a bunch of issues about this. The solution is to put constraints inferred from external and internal (argument) context in the same bin and solving them together. For instance, in your first example the problematic call would give T_contra :> Animal (from inner context) plus T_contra :> Cat (from the outer one). This has a perfect solution Animal for T_contra. Unfortunately, this is a big refactoring and it is not clear when we will do this.
5. Essentially, this is very similar to Mypy is still too eager about outer context #5874 but I would prefer to keep this open, because it is a "rectified" example of why single-bin inference is needed. Current workaround is to simply explicitly provide type arguments instead of relying on inference.

[karlicoss]

@ilevkivskyi thanks for a comprehensive explanation! I'm still not sure if I got some things right though, could you elaborate please?

I used a different example because the original example by @unknownlighter didn't have any specific examples of variance violation (i.e. assignment), perhaps it would be clear for other people if one includes them.

from typing import Generic, TypeVar
X = TypeVar('X')

class Box(Generic[X]):
    def __init__(self, x: X) -> None:
        self.x = x

def consume(v: Box[BaseException]) -> None:
    v.x = Exception("boom!")

consume(Box(SystemExit(1))) # I also expected type error here, but it passes mypy

2. "this call is type-safe"

By that, do you mean that because Box(SystemExit(1)) is a temporary here, there is nothing that can examine from it the outside and observe that x is set to an incompatible type?
In particular, that got me thinking about the walrus operator (because its whole purpose is 'leaking' temporaries), I tried it, and indeed, mypy complains now:

consume(b := Box(SystemExit(123))) # mypy gives type error here
print(b.x.code) # that would be AttributeError in runtime, Exception doesn't have 'code'

I guess there are two potential options:

• b is assigned type according to the right hand side of the walrus operator, i.e. Box[SystemExit], and then you get 'incompatible argument type' in consume (which is what actually happens)
• external context (i.e. consume argument type) imposes type Box[BaseException] on b in the walrus operator. Then you'd expect mypy complaining at missing 'code' attribute in print(b.x.code), but that's not what happens.

So the external context thing only applies for return values, not arguments?

And finally, if I make T contravariant, surprisingly mypy stops complaining (even though the code fails at runtime). I guess the root of problem here is that covariant type is an argument for Box.__init__, which is a different issue: #734 ?

Thanks for you time!

[ilevkivskyi]

I'm still not sure if I got some things right though, could you elaborate please?

I appreciate the gratitude, but no, sorry. I don't have tine to engage in lengthy philosophical discussions. If you have complete concrete examples of false negatives or false positives, please open separate issues. If you want to learn more about mypy to became a contributor, the better alternative is to try fixing some issues.