Only use annotations of @overloaded functions to type check the implementation

[oakkitten]

Consider the following real-world example of a function. The type of one of the parameters of this function depends on another one:

from typing import *

K = TypeVar("K", bound=Hashable)
V = TypeVar("V")


@overload
def dict_get_factory(dictionary: Dict[K, V], 
                     key: K, 
                     factory: Callable[[], V], 
                     takes_key: Literal[False]) -> V: ...

@overload
def dict_get_factory(dictionary: Dict[K, V], 
                     key: K, 
                     factory: Callable[[K], V], 
                     takes_key: Literal[True]) -> V: ...

def dict_get_factory(dictionary: Dict[K, V], 
                     key: K, 
                     factory: Union[Callable[[], V], Callable[[K], V]], 
                     takes_key: bool=False) -> V:
    """Like dict.setdefault(), but uses a factory

    >>> dict_get_factory(d := {}, "foo", int)
    0
    >>> dict_get_factory(d, "bar", lambda key: len(key), takes_key=True)
    3
    >>> d
    {'foo': 0, 'bar': 3}
    """
    try:
        return dictionary[key]
    except KeyError:
        return dictionary.setdefault(key, 
                factory(key) if takes_key else factory())
                
i1: int = dict_get_factory({"foo": 1}, "bar", lambda: 1, False)   # ok
i2: int = dict_get_factory({"foo": 1}, "bar", lambda s: 1, True)  # ok

It doesn't typecheck as if you only look at the annotations of the implementation, the signature of the factory is ambiguous:

main.py:36: error: Too many arguments
main.py:36: error: Too few arguments

I don't think it's possible to annotate the implementation in such a way that it would typecheck. So it would be nice if mypy used the signatures of the @overloaded methods, and one wouldn't have to annotate the signature of the implementation at all. Even if you could annotate it in such a way that it would typecheck, wouldn't it be redundant?

[JelleZijlstra]

You probably have to annotate the implementation with Callable[..., V]. Overload implementations often can't be typechecked precisely.

An alternative way could be to use separate keyword-only arguments for factories with and without keys. That way, I believe you don't even need the overloads.

[Dreamsorcerer]

The problem with the current solution is that you end up needing to use Any which means you basically lose any typing benefits on the implementation.

Proposed solution would be to ignore annotations on the implementation and type check it with the overloads. i.e. Once for each overload, just like how TypeVars are handled.

Which is also how the documentation looks like (no annotations on the implementation):
https://docs.python.org/3/library/typing.html?highlight=annotations#typing.overload

[vnmabus]

I am currently having the same problem, and I agree with the proposed solution. The implementation should be checked against all overload variants, and the errors should be reported (including in which overloads the errors appear). Otherwise it is impossible to typecheck the implementation of functions where the types of some parameters change together, instead of only the return type.