Mypy 1.3.0 crashes when using dictionary unpacking

[eabruzzese]

Crash Report

Mypy 1.3.0 crashes with an internal error when it encounters dictionary unpacking, e.g. **mydict.

Traceback

From 1.3.0:

mypy_unpacking_crash.py:7: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.3.0
Traceback (most recent call last):
  File "mypy/checkexpr.py", line 4890, in accept
  File "mypy/nodes.py", line 2230, in accept
  File "mypy/checkexpr.py", line 4352, in visit_dict_expr
  File "mypy/checker.py", line 6134, in named_generic_type
  File "mypy/checker.py", line 6141, in lookup_typeinfo
  File "mypy/checker.py", line 6219, in lookup_qualified
KeyError: '_typeshed'
mypy_unpacking_crash.py:7: : note: use --pdb to drop into pdb

From master:

mypy_unpacking_crash.py:7: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.4.0+dev.acce2709bac5cd65e983b694ce162f42e4a87176
Traceback (most recent call last):
  File "/Users/eric/.pyenv/versions/3.11.0/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/main.py", line 95, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/main.py", line 174, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/build.py", line 194, in build
    result = _build(
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/build.py", line 267, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/build.py", line 2926, in dispatch
    process_graph(graph, manager)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/build.py", line 3324, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/build.py", line 3425, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/build.py", line 2311, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 472, in check_first_pass
    self.accept(d)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/nodes.py", line 786, in accept
    return visitor.visit_func_def(self)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 962, in visit_func_def
    self._visit_func_def(defn)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 966, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 1038, in check_func_item
    self.check_func_def(defn, typ, name, allow_empty)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 1234, in check_func_def
    self.accept(item.body)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/nodes.py", line 1220, in accept
    return visitor.visit_block(self)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 2682, in visit_block
    self.accept(s)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/nodes.py", line 1401, in accept
    return visitor.visit_for_stmt(self)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 4562, in visit_for_stmt
    self.accept_loop(s.body, s.else_body)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 602, in accept_loop
    self.accept(body)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/nodes.py", line 1220, in accept
    return visitor.visit_block(self)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 2682, in visit_block
    self.accept(s)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/nodes.py", line 1307, in accept
    return visitor.visit_assignment_stmt(self)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 2730, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 2903, in check_assignment
    rvalue_type = self.check_simple_assignment(lvalue_type, rvalue, context=rvalue)
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 3953, in check_simple_assignment
    rvalue_type = self.expr_checker.accept(
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checkexpr.py", line 4879, in accept
    typ = node.accept(self)
          ^^^^^^^^^^^^^^^^^
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/nodes.py", line 2242, in accept
    return visitor.visit_dict_expr(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checkexpr.py", line 4330, in visit_dict_expr
    self.chk.named_generic_type("_typeshed.SupportsKeysAndGetItem", [kt, vt])
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 6226, in named_generic_type
    info = self.lookup_typeinfo(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 6233, in lookup_typeinfo
    sym = self.lookup_qualified(fullname)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/eric/.pyenv/versions/3.11.0/lib/python3.11/site-packages/mypy/checker.py", line 6311, in lookup_qualified
    n = self.modules[parts[0]]
        ~~~~~~~~~~~~^^^^^^^^^^
KeyError: '_typeshed'
mypy_unpacking_crash.py:7: : note: use --pdb to drop into pdb

To Reproduce

Using this file:

# mypy_unpacking_crash.py
from typing import Any

def get_annotations(cls: type) -> dict[str, Any]:
    """Return a dict of all annotations for a class and its parents."""
    annotations: dict[str, Any] = {}
    for ancestor in cls.__mro__:
        annotations = {**annotations, **getattr(ancestor, "__annotations__", {})}
    return annotations

And this minimal pyproject.toml (thanks @AlexWaygood!)

[tool.mypy]
follow_imports = "skip"
follow_imports_for_stubs = true

And running this command:

mypy --show-traceback mypy_unpacking_crash.py

I get the traceback above.

Your Environment

• Mypy version used: 1.3.0
• Mypy command-line flags: --show-traceback
• Mypy configuration options from mypy.ini (and other config files):

# pyproject.toml

[tool.mypy]
check_untyped_defs = true
color_output = true
disallow_incomplete_defs = true
disallow_untyped_calls = true
disallow_untyped_decorators = false
disallow_untyped_defs = true
error_summary = true
follow_imports = "skip"
follow_imports_for_stubs = true
ignore_missing_imports = true
no_implicit_optional = true
# I'm using django-stubs, but I've also tried without it and the crash still occurs
plugins = ["mypy_django_plugin.main"]
pretty = true
show_error_context = true
strict_equality = true
strict_optional = true
warn_no_return = true
warn_redundant_casts = true
warn_return_any = false
warn_unreachable = true
warn_unused_ignores = true

• Python version used: 3.11
• Operating system and version: MacOS 12.5.1

Theories

• Could this PR be related?

[AlexWaygood]

Simpler repro: run mypy on this snippet:

from typing import Any

def get_annotations(cls: type) -> dict[str, Any]:
    """Return a dict of all annotations for a class and its parents."""
    annotations: dict[str, Any] = {}
    for ancestor in cls.__mro__:
        annotations = {**annotations, **getattr(ancestor, "__annotations__", {})}
    return annotations

With this pyproject.toml file:

[tool.mypy]
follow_imports = "skip"
follow_imports_for_stubs = true

Both of the above flags appear to be required to reproduce the crash, but none of the others are.

[hauntsaninja]

Lol, probably prevents mypy from figuring out what _typeshed is. The relevant change in 1.3 is #14990 , my guess is need to find what special casing we do for other things in the builtins SCC to make follow imports not apply and do it for _typeshed as well

[eabruzzese]

Thanks @AlexWaygood! 🙏 I added that minimal example to the repro instructions in the description

[hauntsaninja]

Thanks for the report!

Note that global follow_imports = "skip" + follow_imports_for_stubs = true is an unusual setting that the vast majority of users don't actually ever want. Just mentioning since global follow_imports is amongst the most misused mypy configuration options :-)

[eabruzzese]

@hauntsaninja I didn't even notice that we had follow_imports set to skip; thanks so much for pointing that out. I think we probably want normal, since we're going for the strictest-possible mypy configuration (within reason) for most of our projects. I'll have to read more about those options to get a better understanding of how they work.