New Semantic Analyzer: INTERNAL ERROR on aiohttp

[sk-]

This is only a potential bug report, as I found that upgrading the triggering library solves the issue.

When analyzing code with the new semantic analyzer, mypy fails processing the client module of aiohttp (v3.4.4).

The difference between the affected version and the latest code is how they annotate the dataclasses using attr.ib. In v3.4.4 they have total = attr.ib(type=float, default=None), while on v3.5.4 they use total = attr.ib(type=Optional[float], default=None).

Compare https://github.com/aio-libs/aiohttp/blob/v3.4.4/aiohttp/client.py#L49-L53 and https://github.com/aio-libs/aiohttp/blob/v3.5.4/aiohttp/client.py#L132-L136

Below is the output of the run with the latest development version

/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/aiohttp/client.py:49: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.rtfd.io/en/latest/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.720+dev.262fe3dc4d5d14492c6fd4009d91c555c406ac04
Traceback (most recent call last):
  File "/Users/skreft/.virtualenvs/zapship/bin/mypy", line 10, in <module>
    sys.exit(console_entry())
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/__main__.py", line 8, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/main.py", line 83, in main
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/build.py", line 164, in build
    result = _build(sources, options, alt_lib_path, flush_errors, fscache, stdout, stderr)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/build.py", line 224, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/build.py", line 2567, in dispatch
    process_graph(graph, manager)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/build.py", line 2880, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/build.py", line 2978, in process_stale_scc
    semantic_analysis_for_scc(graph, scc, manager.errors)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/newsemanal/semanal_main.py", line 74, in semantic_analysis_for_scc
    process_top_levels(graph, scc, patches)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/newsemanal/semanal_main.py", line 193, in process_top_levels
    patches)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/newsemanal/semanal_main.py", line 315, in semantic_analyze_target
    active_type=active_type)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/newsemanal/semanal.py", line 360, in refresh_partial
    self.refresh_top_level(node)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/newsemanal/semanal.py", line 371, in refresh_top_level
    self.accept(d)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/newsemanal/semanal.py", line 4435, in accept
    node.accept(self)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/nodes.py", line 923, in accept
    return visitor.visit_class_def(self)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/newsemanal/semanal.py", line 978, in visit_class_def
    self.analyze_class(defn)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/newsemanal/semanal.py", line 1049, in analyze_class
    self.analyze_class_body_common(defn)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/newsemanal/semanal.py", line 1058, in analyze_class_body_common
    self.apply_class_plugin_hooks(defn)
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/newsemanal/semanal.py", line 1103, in apply_class_plugin_hooks
    hook(ClassDefContext(defn, decorator, self))
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/plugins/attrs.py", line 214, in attr_class_maker_callback
    if info[attr.name].type is None and not ctx.api.final_iteration:
  File "/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/mypy/nodes.py", line 2413, in __getitem__
    raise KeyError(name)
KeyError: 'total'
/Users/skreft/.virtualenvs/zapship/lib/python3.6/site-packages/aiohttp/client.py:49: : note: use --pdb to drop into pdb

[ilevkivskyi]

Just for the record here is a simple repro test case:

[case testTypeInAttrDeferredStar]
import lib
[file lib.py]
import attr
from other import *

@attr.s
class C:
    total = attr.ib()

[file other.py]
import lib
[builtins fixtures/bool.pyi]

I think this is high priority because it also highlights a wider problem: NewSemanticAnanlyzer.missing_names is not per-scope, while it probably should be. Otherwise a star import at top level blocks adding any names to a class for example.

[ilevkivskyi]

Also we should check whether dataclasses don't have the same problem.