false positive: classmethod constructor on generic class

[jpassaro]

I'm inclined to think this is a bug, but could be argued it's a missing feature.

How to reproduce

# makes_bug.py

from typing import Generic, Type, TypeVar

T = TypeVar('T')

class Foo(Generic[T]):
    def __init__(self, value: T) -> None: ...
    @classmethod
    def construct(cls) -> Foo[str]:
        return cls('bar')

Expected outcome

This code should pass type checks.

Actual outcome

mypy, to my great surprise, fails:

$ python --version
Python 3.8.1
$ mypy --version
mypy 0.790+dev.65186ae1e23fd44f1d7e6aa2c4458bdf55640742
$ mypy src/makes_bug.py
makes_bug.py:11: error: Incompatible return value type (got "Foo[T]", expected "Foo[str]")
makes_bug.py:11: error: Argument "value" to "Foo" has incompatible type "str"; expected "T"
Found 2 errors in 1 file (checked 1 source file)

Workaround

Changing line 10 to def construct(cls: Type[Foo]) -> Foo[str]: fixes it; or, probably more or less equivalently, def construct(cls: Type[Foo[Any]]) -> Foo[str]:.

Investigation / suggested fix

Adding reveal_type(cls) indicates that cls is inferred to have type Foo[T], which cannot return a Foo[str].

It's not clear to me why classmethods should have the lead argument inferred to a bound type. I'd propose that my workaround be made a default: class methods should infer the type of the lead argument type as the unbound class type, or bound with all Anys.

I'd imagine the change would go here: https://github.com/python/mypy/blob/65186ae1e23fd44f1d7e6aa2c4458bdf55640742/mypy/semanal.py#L609,L610
or (more likely) in the referenced method class_type():
https://github.com/python/mypy/blob/65186ae1e23fd44f1d7e6aa2c4458bdf55640742/mypy/semanal.py#L4790,L4791

Happy to take a stab at it, but wanted to solicit some feedback first. This is a complex system and it's possible I'm missing some potential consequence here.

[jpassaro]

On further inspection, this appears to be intentional behavior, per case testGenericClassDirectCall:

from typing import TypeVar, Generic

T = TypeVar('T')

class C(Generic[T]):
    def __init__(self, item: T) -> None: ...
    @classmethod
    def f(cls) -> None:
        cls(1)  # E: Argument 1 to "C" has incompatible type "int"; expected "T"

For the sake of enlightening any future readers of this question, here's what this leads me to. The code I had, which originally failed these tests, looked something like this:

class C(Generic[T]):
    def __init__(self, processor: Callable[[Any], T]) -> None: ...
    @classmethod
    def forstr(cls) -> C[str]:
        return cls(processor=lambda x: cast(str, x))

cstr: C[str] = C.forstr()

I assumed cls would have type C[Any]. What I forgot to take into account is that classmethods can be called on instances as well (e.g. c_int: C[int] = C(int); c_int.forstr()), for which reason the type has to be inferred as C[T]. For that reason, the workaround, which I originally thought was a hack, is actually a reasonably straightforward way of clarifying the intention:

class C(Generic[T]):
    def __init__(self, processor: Callable[[Any], T]) -> None: ...
    @classmethod
    def forstr(cls: Type[C[Any]], *args) -> C[str]:
        return cls(*args, processor=lambda x: cast(str, x))

cstr: C[str] = C.forstr(...)

[jpassaro]

I've been thinking about this further and I think I must be missing some context here.

When mypy forced the code I had before to define @classmethod def forstr(cls: Type[C[Any]]) -> C[str]: instead of the implicitly type cls variable, it was pointing out that the class method could be invoked not just as C.forstr or C[...].forstr, but as C(...).forstr(). It's not clear to me what runtime errors this is designed to avoid, or what other purpose this strictness serves. The underlying constructor is valid no matter what the type parameters are; Cint works. Is there something I'm missing?

If it's not that this is specifically problematic, but that this check arises from other contexts in which this strictness is useful, I wonder if it would make sense to carve out an exception for classmethods that happen to be constructors, since this is a fairly common occurrence. Something like detecting a return type whose name refers to the same node in the local symbol table as the surrounding class def. Would anybody find this valuable?

[JukkaL]

The mypy behavior looks strange, perhaps because the type variable T is considered bound in the class method. Maybe we can change the type of cls if there is no annotation.