incremental: Crash after deleting a function.

[timabbott]

I ran into this crash with 0.501 today on the Zulip codebase when running mypy with this version of the code zulip/zulip@09f66b5 (which deleted the below function).

I've attached the cache state tarball in case it's helpful: mypy.tar.gz

Traceback (most recent call last):
  File "/srv/zulip-py3-venv/bin/mypy", line 6, in <module>
    main(__file__)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/main.py", line 42, in main
    res = type_check_only(sources, bin_dir, options)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/main.py", line 86, in type_check_only
    options=options)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 183, in build
    dispatch(sources, manager)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1531, in dispatch
    process_graph(graph, manager)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1761, in process_graph
    process_fresh_scc(graph, prev_scc)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1830, in process_fresh_scc
    graph[id].fix_cross_refs()
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1311, in fix_cross_refs
    fixup_module_pass_one(self.tree, self.manager.modules)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/fixup.py", line 21, in fixup_module_pass_one
    node_fixer.visit_symbol_table(tree.names)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/fixup.py", line 79, in visit_symbol_table
    stnode = lookup_qualified_stnode(self.modules, cross_ref)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/fixup.py", line 255, in lookup_qualified_stnode
    assert key in names, "Cannot find %s for %s" % (key, name)
AssertionError: Cannot find do_set_realm_description for zerver.lib.actions.do_set_realm_description

[refi64]

FWIW, I couldn't actually reproduce this with (inside the Zulip directory):

rm -rf var/mypy_cache
git checkout 09f66b5c6dfc5c8bf5d52fb51259189ade0f336e^
tools/run-mypy
git checkout 09f66b5c6dfc5c8bf5d52fb51259189ade0f336e
tools/run-mypy

[gvanrossum]

Yeah the missing info is all the mtimes of the files in Tim's working directory when it failed. :-(

[gvanrossum]

(FWIW if this occurs again on an open source package, as of mypy 0.521 the mtimes in the .mypy_cache directory are no longer a problem, as we now use a hash of the source file.)

[timabbott]

I think @hackerkid saw this earlier today in the Zulip project; but we may be a bit behind 0.521 (haven't updated since PyCon).

[hackerkid]

I got the error after I switched from a branch invite-link which had the model MultiUserInvitation to the branch get-user-2g which didn't had the model. But interestingly I was not able to reproduce the error when I switched from invite-link to master. I tried the switching multiple times and was able to obtain the same behavior. All the branches were rebased on top of upstream/master.

During each switch I did

• ./tools/provision
• ./tools/run-mypy

I have attached the tar of my Zulip directory including the cache here.

Update: fixed the get-user-2g-link. Thanks ethanhs

[emmatyping]

(the correct get-user-2g link)

@hackerkid did this happen consistently? Could you try updating the version of Mypy that you use? It appears your version is old.

[gvanrossum]

I downloaded the tarball but could not repro the issue with mypy 0.521, so I think it's not worth looking into further. Let us know whether you still see this with master.

[timabbott]

Hmm, I've definitely seen this since 0.521. I just upgraded to the latest mypy master in case that helps; will let you know when I next see it (it's about once a week, usually after a series of switches between branches including one that deletes things).

[gvanrossum]

If and when you see it, tarring up your entire zulip tree (.git and all) like @hackerkid did would be very helpful.

[gvanrossum]

We've seen a similar crash internally, where the assert referenced a class that was deleted by the change. Sadly it was also impossible to repro. Which means that at least the work-around of deleting the cache shouldn't be too painful.

[gvanrossum]

An internal user claims this can be reproduced as follows:

0. Start with a clean slate
1. Run mypy
2. Make an edit
3. Run it again, with -i
4. Revert the edit (e.g. git stash)
5. Run it again, with -i -- BOOM

The edit in step 2 could be a whitespace edit, as long as it's a type-checked file.