Typeshed CI tests fail with error about __add__ vs. __iadd__

[gvanrossum]

Example: https://travis-ci.org/python/typeshed/builds/112511679

The specific error is:

stdlib/3/builtins.pyi: note: In class "list":
stdlib/3/builtins.pyi:476: error: Signatures of "__iadd__" and "__add__" are incompatible

I cannot repro this locally and the Travis CI tests pass when the PR is merged: https://travis-ci.org/python/typeshed/builds/112653577

I've seen this error before but only in my "incremental" branch. It's pretty shy -- sometimes I get it consistently, sometimes it's irreproducible (in fact I haven't seen it in a while). I think the cause of the error is a subtlety in the order in which the import cycle (builtins, typing, abc) is type-checked. My theory is that in my branch the processing order is dependent on Python's hash seed; but I've also made some subtle changes to the processing of cycles in that branch.

I have two hypotheses about why this is now happening in the Travis CI tests for typeshed:

• either somehow my branch leaked into some Travis CI test machine and pip install doesn't properly overwrite this
• or this has always been a problem, depending on the hash seed, and we've somehow never caught it in master

[JukkaL]

I've never seen this before.

Do you think that it's possible to reproduce this issue with an empty Python file that only depends on builtins, typing and/or abc? If so, it shouldn't be too hard to manually verify the potential places where hash seed could make a difference.

[gvanrossum]

Sadly I've not been able to repro this yet. I'll keep an eye open.

[gvanrossum]

Another theory: it may depend on the order in while os.listdir() returns the files in a package. The traveis tests run on Linux, I only test on OSX. I think OSX returns files in sorted order.

[JukkaL]

Should we sort the result of os.listdir() for better predictability?

[gvanrossum]

This was because runtests.py passes typing.pyi on the command line before builtins.pyi. I've checked in a hack for typeshed/runtests.py but we need to leave this open for a better fix. It's rstill reproducible as follows:

mypy stdlib/3/typing.pyi stdlib/3/builtins.pyi

[ethanhs]

I cannot seem to replicate @gvanrossum's error using that command. Perhaps this has been fixed?

[ilevkivskyi]

I cannot seem to replicate @gvanrossum's error using that command. Perhaps this has been fixed?

On the other hand, the same command for Python 2:

mypy stdlib/2/typing.pyi stdlib/2/builtins.pyi

crashes with:

Traceback (most recent call last):
  File "/usr/local/bin/mypy", line 11, in <module>
    load_entry_point('mypy===0.520-dev-b7636cbd6566b6872a80abe3d2b731aa0a684ada', 'console_scripts', 'mypy')()
  File "/usr/local/lib/python3.4/dist-packages/mypy/__main__.py", line 7, in console_entry
    main(None)
  File "/usr/local/lib/python3.4/dist-packages/mypy/main.py", line 50, in main
    res = type_check_only(sources, bin_dir, options)
  File "/usr/local/lib/python3.4/dist-packages/mypy/main.py", line 97, in type_check_only
    options=options)
  File "/usr/local/lib/python3.4/dist-packages/mypy/build.py", line 192, in build
    graph = dispatch(sources, manager)
  File "/usr/local/lib/python3.4/dist-packages/mypy/build.py", line 1761, in dispatch
    process_graph(graph, manager)
  File "/usr/local/lib/python3.4/dist-packages/mypy/build.py", line 2004, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/usr/local/lib/python3.4/dist-packages/mypy/build.py", line 2099, in process_stale_scc
    graph[id].semantic_analysis()
  File "/usr/local/lib/python3.4/dist-packages/mypy/build.py", line 1656, in semantic_analysis
    self.manager.semantic_analyzer.visit_file(self.tree, self.xpath, self.options, patches)
  File "/usr/local/lib/python3.4/dist-packages/mypy/semanal.py", line 296, in visit_file
    self.accept(d)
  File "/usr/local/lib/python3.4/dist-packages/mypy/semanal.py", line 3605, in accept
    node.accept(self)
  File "/usr/local/lib/python3.4/dist-packages/mypy/nodes.py", line 754, in accept
    return visitor.visit_class_def(self)
  File "/usr/local/lib/python3.4/dist-packages/mypy/semanal.py", line 669, in visit_class_def
    defn.defs.accept(self)
  File "/usr/lib/python3.4/contextlib.py", line 66, in __exit__
    next(self.gen)
  File "/usr/local/lib/python3.4/dist-packages/mypy/semanal.py", line 721, in analyze_class_body
    self.setup_type_promotion(defn)
  File "/usr/local/lib/python3.4/dist-packages/mypy/semanal.py", line 790, in setup_type_promotion
    promote_target = self.named_type_or_none(promotions[defn.fullname])
  File "/usr/local/lib/python3.4/dist-packages/mypy/semanal.py", line 1202, in named_type_or_none
    assert isinstance(node, TypeInfo)
AssertionError: 

[ilevkivskyi]

The latter crash however is a duplicate of #2386. So that I think we can close this.