**kwargs checked against wrong type with defaults or *args

[kcr]

def a(*args: str, **kw: int) -> None:
    b(**kw)
    b('', **kw)
    c(**kw)
    c('', **kw)

def b(args: str='', **kw: int) -> None:
    pass

def c(*args: str, **kw: int) -> None:
    pass

results, with 0.4.3:

/tmp/minimize.py: note: In function "a":
/tmp/minimize.py:2: error: Argument 1 to "b" has incompatible type **Dict[str, int]; expected "str"
/tmp/minimize.py:4: error: Argument 1 to "c" has incompatible type **Dict[str, int]; expected "str"

[gvanrossum]

Can you try again with the latest revision from the repo? I fixed some things in this area recently.

[gvanrossum]

Never mind. I can still repro this.

[gnprice]

The logic the code might be implementing here, at least in the call to b, is that it's possible 'args' is a key in kw, in which case we'd be passing an int to a parameter of type str.

That logic doesn't seem to apply in the second case, though, which would make that a bug.

In the b case it's possible this is the kind of unsoundness we'd rather allow, because it's probably cumbersome to explicitly prevent it in this code. Even if we do give an error for it, this error message isn't really on point about what the issue is, and if nothing else we should make it clearer.

[gnprice]

And thanks, Karl, for the report!

[gvanrossum]

In particular, calling a(args=0) would end up calling b(args=0) which is a bug in your code. But the same reasoning doesn't apply to c(**kw), so that's a bug in mypy.

[elazarg]

This can be fixed by adding at checkexpr.py#L1980 (right below the comment referring to that situation)

-    if ((callee_names[j] and no_certain_match)
                    or callee_kinds[j] == nodes.ARG_STAR2):
+    if ((callee_names[j] and no_certain_match and callee_kinds[j] != nodes.ARG_STAR)
                    or callee_kinds[j] == nodes.ARG_STAR2):

However, it raises another error for the following file. I wonder whether this is the expected behavior:

def a(**kw: int) -> None:
    a(**kw)
    c(**kw)
    d(**kw)

def b(a: int) -> None: pass
def c(*args: str) -> None: pass
def d() -> None: pass

def x(*args: int) -> None:
    y(*args)
    z(*args)
    q(*args)

def y() -> None: pass
def z(**kw: int) -> None: pass
def q(a: int) -> None: pass

Result:

tmp.py: note: In function "a":
tmp.py:3: error: Too many arguments for "c"
tmp.py:4: error: Too many arguments for "d"
tmp.py: note: In function "x":
tmp.py:16: error: Too many arguments for "y"
tmp.py:17: error: Too many arguments for "z"

Perhaps that's intentional, but I don't see why the empty *args and the empty **kw are discriminated.
The situation with *args is the current behavior, irrelevant to the change. It's explicitly checked for in the code for being an empty tuple, emitting error otherwise. why?

If the suggested solution seems valid, I can open a PR.

[ilevkivskyi]

(This appeared two more times so raise priority to normal)

[ilevkivskyi]

#5834 has another example of this with some details.

[ilevkivskyi]

Raised priority to high, since this keeps coming.

As noticed in #6092 this bug only appears when the type of kwargs is Dict, but it doesn't when the type is Mapping.

[JukkaL]

I think that I know how to fix this since I just recently worked on the relevant code.

[JukkaL]

Actually the suggested fix by @elazarg above seems to be the correct one. I'll prepare a PR.

[snejus]

mypy 0.782 and python 3.6 here - I'm having it as well. Example:

@dataclass
class ContextData:
    """Hi."""
    account: Optional[Dict[str, str]] = None
    module: Optional[str] = None
    data: Optional[MutableMapping[str, Any]] = None


class Context(ContextData, ABC):
    """Hello."""

    @classmethod
    def create(
        cls, 
        data: MutableMapping[str, Any], 
        module: str, 
        attribs: Optional[MutableMapping[str, Any]] = None
):
        to_save = {
            "account": data.pop("account"),
            "module": module,
            "data": data,
        }
        if attribs:
            to_save.update(attribs)
        return cls(**to_save)

src/file.py:67: error: Argument 1 to "Context" has incompatible type "**Dict[str, Collection[str]]"; expected "Optional[Dict[str, str]]"  [arg-type]

Could it be related to dataclasses?

[JelleZijlstra]

@snejus this doesn't look like the original problem. You might have to just declare to_save as Dict[str, Any]. If that's not sufficient, feel free to open a new issue.

I'm kind of curious where the Collection in your error message comes from.

[snejus]

Yup - you're right. Figured that out this morning and had to facepalm myself. :) It was related to the way the arguments are ordered.
The Collection thingie - this factory was getting called with a super from one of the derived classes - I may have used an OrderedDict there - can't tell for sure now since it's already been refactored.

[matt2000]

Seems like there's still a challenge here, in that, mypy seems to me to be giving a misleading error message. Is there are any way to report the specific kwarg dictionary item that is mistyped rather than reporting on the **Dict ? Should another issue be opened for that?

[ClementWalter]

Maybe a related example:

from typing import Union


class Parent:
    def __init__(self, a: int, b: str) -> None:
        self.a = a
        self.b = b


class Child(Parent):
    def __init__(self, c: float, **kwargs: Union[int, str]) -> None:
        super().__init__(**kwargs)
        self.c = c


child = Child(a="a", b=2, c=2.3)

# tmp.py:12: error: Argument 1 to "__init__" of "Parent" has incompatible type "**Dict[str, Union[int, str]]"; expected "int"
# tmp.py:12: error: Argument 1 to "__init__" of "Parent" has incompatible type "**Dict[str, Union[int, str]]"; expected "str"
# Found 2 errors in 1 file (checked 1 source file)

[JukkaL]

@ClementWalter In your example there is no guarantee that the argument names and types match the called function. For example, you have b=2 but it's declared as b: str in the function. I think that mypy is right in reporting an error here, though the error message is not super helpful. You can create a new issue about the confusing error message if you'd like.

[ClementWalter]

actually Union[str, int] is not a good typing but I was wondering how to handle this case (inheritance) with mypy and google the error I found this issue. I guess that ideally we could provide a TypedDict to kwargs typing, something like:

ParentType(TypedDict):
    a: int
    b: str

class Child(Parent):
    def __init__(self, c: float, **kwargs: ParentType) -> None:
        ....