Poor error message due to positional-or-keyword incompatibility

[DanielNoord]

Bug Report

When using Callable to type a method instead of defining it (in my use case it is defined and assigned in another file) mypy doesn't seem to follow the same rules for child methods with and without parameters.
I asked on gitter and was told this is probably a bug. I also searched and was unable to find another report for this, sorry if this is still a duplicate.

To Reproduce

Use the following code:

from typing import Callable, Generator


class Parent:
    def method_without(self) -> "Parent":
        ...

    def method_with(self, param: str) -> "Parent":
        ...


class Child(Parent):
    method_without: Callable[["Child"], "Child"]
    method_with: Callable[["Child", str], "Child"]

Expected Behavior

Not sure. Either both typings of the child methods should be rejected or both should be accepted.

Actual Behavior

test.py:14: error: Incompatible types in assignment (expression has type "Callable[[str], Child]", base class "Parent" defined the type as "Callable[[str], Parent]") [assignment]

Your Environment

• Mypy version used: 0.931
• Mypy command-line flags: --strict
• Mypy configuration options from mypy.ini (and other config files):

no_implicit_optional = True
scripts_are_modules = True
warn_unused_ignores = True
show_error_codes = True

• Python version used: 3.10
• Operating system and version: macOS

[hauntsaninja]

Thanks for the report. Looks like Eric figured out what was going on in python/typing#1036 — the difference is that param can be passed by keyword in the parent. So changing the title to reflect this as a diagnostics / usability issue.

[DanielNoord]

Ah sorry, should have recognised that that issue was also what was causing this. Thanks for updating! Should the label be updated as well?

[A5rocks]

I made a simple diagnostic (ParamSpec-only and definitely improvable) because this behaviour + concatenate was confusing: #11847 (comment)

However, something to wonder is if we should allow self arguments to be inferred as pos-only, since that seems to be a common problem and really, not much safety is gained.

[ktbarrett]

It's not just the positional-only issue. The original code is not type safe.

from typing import Callable, Generator

class Parent:
    def method_without(self) -> "Parent":
        return Parent()  # returning Parent here
    def method_with(self, param: str, /) -> "Parent":
        return Parent()

class Child(Parent):
    method_without: Callable[["Child"], "Child"]
    method_with: Callable[["Child", str], "Child"]    

c = Child()
reveal_type(c.method_without())  # says it's a Child... 

[hauntsaninja]

@ktbarrett The return type seems type safe to me. I don't see a situation in which if you thought you were receiving a Parent and instead received a Child you'd get a type error.

[ktbarrett]

@hauntsaninja And if Child defines the method extra_method?

[hauntsaninja]

Maybe I'm being really dumb, but that seems fine to me too.

[ktbarrett]

c = Child()
d = c.method_without()  # says it's a Child, it's actually a Parent
d.extra_method()  # No error here, Child has extra_method

[hauntsaninja]

But c is a Child, so c.method_without() returns a Child, so d is a Child. Where do you get the "it's actually a Parent" from?

[ktbarrett]

What method is it dispatching to at runtime? Parent.method_without. Unless Child has a new definition of method_without that actually does return a Child, that attribute is an unsafe cast of Parent.method_without.

[JelleZijlstra]

That's a different (and much broader) problem, where mypy doesn't check that declared instance attributes actually exist. You could get the same effect if the parent had attr: object = object() and the child had attr: int.

[ktbarrett]

That's a different (and much broader) problem, where mypy doesn't check that declared instance attributes actually exist. You could get the same effect if the parent had attr: object = object() and the child had attr: int.

Hmmm... Is it not reasonable to prevent users from defining overlapping and incompatible attributes in subclasses? If the superclass defined the attribute, the child class doesn't need to redefine it... it simply needs to make sure that it fills in an implementation.

EDIT 1: I actually think there might not ever be a good reason to redefine an instance attribute without an initial value...
EDIT 2: Expounded invalid uses.
EDIT 3: Ehh... this is hard. It seems like it's an easy fix to simply not allow redefining instance attributes that don't have a value associated with them. I haven't yet thought of a need to ever do that.

[sliedes]

IIUC, I think this is a simpler piece of code that illustrates this problem—I ran into this when trying to type 3rd party code that monkey-patches some stdlib functions:

def foo(x: int) -> str | None:
    return None

foo = lambda _: None  # Incompatible types in assignment (expression has type "Callable[[int], None]", variable has type "Callable[[int], str | None]")  [assignment]

The error message seems rather confusing and doesn't even mention what's wrong—the named parameter x.

Apparently there is some effort to be more helpful, because if I change foo's return value to None, I get this more helpful message instead; however, this apparently fails if the return type is narrowed:

Incompatible types in assignment (expression has type "Callable[[Arg(int, '_')], None]", variable has type "Callable[[Arg(int, 'x')], None]") [assignment]