Odd error message when kwargs passed as **dict

[scop]

Checking this with mypy 0.620:

from concurrent.futures import ThreadPoolExecutor
import sys

opts = {'max_workers': None}
if sys.version_info >= (3, 6):
    opts['thread_name_prefix'] = 'SyncWorker'
executor = ThreadPoolExecutor(**opts)

...yields:

mypy_bug.py:6: error: Incompatible types in assignment (expression has type "str", target has type "None")
mypy_bug.py:7: error: Argument 1 to "ThreadPoolExecutor" has incompatible type "**Dict[str, None]"; expected "str"

Let's ignore the first error for now. But the second seems to have an incorrect message, specifically I think the **Dict[str, None] in it is weird. Just guessing, is it trying to assign something to the ThreadPoolExecutor's thread_name_prefix arg which is a str? Maybe the None from max_workers because passing kwargs as a dict like this might result in random/varying order of the dict items? But if that's the case, I'd expect it to say something else than **Dict[str, None] as the culprit, maybe None or int, but not the whole **dict.

Another thought is that when kwargs is passed as **dicts like this, their order is not really defined, so "Argument 1" isn't very helpful. Would be better to say the argument's name.

But maybe I'm just all off track here -- will leave the rest to someone who actually has an informed opinion :)

[scop]

(I just tried defining opts as OrderedDict and the error went away, but anyway I think the points/suggestions above still hold.)

[radzak]

@scop Hey, take a look at this SO question. It should help you understand why you're getting the first error.

The answer to this question will explain you that the type of the opts dictionary mypy inferred is Dict[str, None]. If you take a look at a stub for ThreadPoolExecutor, you can notice that the expected type for thread_name_prefix is str:

        def __init__(self, max_workers: Optional[int] = ...,
                     thread_name_prefix: str = ...) -> None: ...

but mypy thinks you're passing **Dict[str, None], which causes the incompatible type error (None is not str).

[akaptur]

I've hit the same issue with passing a dict of kwargs. I'm also on mypy 0.620, python 3.6.5. Here's a small repro:

def test(a: int = 1, b: str = 'hi', c: bool = True) -> None:
    return None

kwargs = {"a": 1, "b": "hello", "c": False}
test(**kwargs)

This code is correct at runtime, but generates the following mypy errors:

test.py:9: error: Argument 1 to "test" has incompatible type "**Dict[str, object]"; expected "int"
test.py:9: error: Argument 1 to "test" has incompatible type "**Dict[str, object]"; expected "str"
test.py:9: error: Argument 1 to "test" has incompatible type "**Dict[str, object]"; expected "bool"

I can suppress this error by either:

• annotating kwargs as Dict[str, Any]
• adding a #type: ignore
• calling the function with the kwargs specified (test(a=1, b="hello", c=False))

Something that I might expect to help, but doesn't, is annotating kwargs as Dict[str, Union[str, bool, int]].

The first two ways are not really fixes, and the third is not always an option. Is there a good way to keep mypy from emitting this false positive?

[ilevkivskyi]

Is there a good way to keep mypy from emitting this false positive?

class Options(TypedDict):
    a: int
    b: str
    c: bool

kwargs: Options = {"a": 1, "b": "hello", "c": False}
test(**kwargs)  # OK

Note however that this may trade false positive for false negative because of #5198

[ilevkivskyi]

Also this is not really a false positive. There is no way mypy can guarantee that a key will be e.g. deleted from this dictionary (it will however prevent you from deleting keys if this is a TypedDict).

[ilevkivskyi]

This happened again, so raising priority to normal. Btw we can at least infer TypedDict types for dict literals if they appear directly in ** context in function call. Finally, the error message should be more clear (explaining that a variable is of type dict with unknown keys, and maybe suggesting TypedDict).

[ilevkivskyi]

This appeared three more times, so raising priority to high.

[w-p]

Hit this today, inconsistently, in several places. I can confirm that adding an explicit Dict[str, Any] resolves the issue but it's still unclear why.

[stevenjackson121]

If the general problem is hard to solve. Would it be easier (and still valuable) to add support for something like:

# OK iff test(a=1,b="hello",c=False) is OK
kwargs: Final[frozendict] = frozendict({"a": 1, "b": "hello", "c": False})
test(**kwargs)  

Final[frozendict] removes the need to handle any changes to kwargs between definition and usage (or the need to infer changes arent made), and would cover all my use cases (described in more detail in #8485)

[ilevkivskyi]

Supporting frozen typed dicts is also a possible solution. We can even support both things (this and the older idea of inferring typed dict types for dict literals appearing directly in a call).

Would it be easier (and still valuable) to add support for something like

The core team has very limited resources, if you want to work on this, please go ahead.

[patrick91]

I got this today as well, using a TypedDict fixed the mypy error :)

[zachliu]

I got this today as well. Unfortunately using a TypedDict doesn't work in my case 😢

@type_checked_constructor(skip=True)
@dataclass
class MyOtherStuff():
    ...

@type_checked_constructor(skip=True)
@dataclass
class File():
    ...

@type_checked_constructor(skip=True)
@dataclass
class MyStuff():
    """MyStuff Dataclass"""
    kind: str
    is_ready: bool
    request: MyOtherStuff
    files: Optional[List[File]]

data = {...}

MyStuff(**data)

causes a new error
error: Expected keyword arguments, {...}, or dict(...) in TypedDict constructor

Using mypy==0.782 with Python 3.8.4

[derrix060]

Similar to #8890

[wereii]

I've just hit this error with dataclasses

Example

from dataclasses import dataclass


@dataclass(slots=True, kw_only=True, frozen=True)
class TestClass:
    ID: int
    name: str
    domain: str
    secure: bool


data = {
    "ID": 1,
    "name": "example",
    "domain": "example.com",
    "secure": True,
}


TestClass(**data)

Output

> mypy test_my.py
test_my.py:20: error: Argument 1 to "TestClass" has incompatible type "**Dict[str, object]"; expected "int"
test_my.py:20: error: Argument 1 to "TestClass" has incompatible type "**Dict[str, object]"; expected "str"
test_my.py:20: error: Argument 1 to "TestClass" has incompatible type "**Dict[str, object]"; expected "bool"
Found 3 errors in 1 file (checked 1 source file)

Versions

Python 3.10.1 (main, Dec 18 2021, 23:53:45) [GCC 11.1.0]
mypy 0.930

[adamwojt]

I run into this problem today with below case:

error_kwargs = {
    "tags": {
        "foo": bar,
    },
    "contexts": {"foo": {"bar": "bar"}},
}
if error_msg in self.non_retryable_errors:
    raise NonRetryableResponseError(**error_kwargs)
else:
    raise RetryableResponseError(**error_kwargs)

I chose to just annotate error_kwargs with Dict[str, Any] as TypedDict seemed like overkill for me.

[mastercoms]

Hit this with pydantic's create_model:

fields = {
  "test": (str, "")
}
create_model("model", **fields)

[aripollak]

We can now at least check kwargs with differing types in a function definition by using Unpack with TypedDict, though the error message without that is still a little confusing. https://github.com/python/mypy/blob/master/CHANGELOG.md#mypy-17