Untangle most of the big import cycle

[msullivan]

Currently mypy has a 47 module import cycle, which is not great.

This untangles things, leaving the following cycles:

• Size 2: mypy.build, mypy.semanal_main
• Size 2: mypy.plugin, mypy.interpreted_plugin
• Size 4: mypy.checker, mypy.checkexpr, mypy.checkmember, mypy.checkstrformat. The type checker itself. This could be untangled by having the sub-checkers operate on an interface of the checker, like the semantic analyzer does.
• Size 5: mypy.nodes, mypy.types, mypy.type_visitor, mypy.visitors, mypy.strconv. This can't really be untangled.
• Size 14: mypy.messages, mypy.expandtype, mypy.erasetype, mypy.typevars, mypy.maptype, mypy.typeops, mypy.sametypes, mypy.subtypes, mypy.constraints, mypy.applytype, mypy.join, mypy.meet, mypy.solve, mypy.infer. The "typeops tangle". Untangling a little further is probably possible but will require some thought and actual logic changes to code. Messages can definitely be removed.

The untangling done here is pretty simple and consists of two main parts:

• Remove mypy.types's dependency on the type ops tangle by moving all of its functions that depend on it into mypy.typeops.
• Remove the dependency of the type ops tangle on the type checker by moving some functions from mypy.checkmember to mypy.typeops.

[gvanrossum]

Yay! Are we going to use black next? :-)

[gvanrossum]

I know this will break Dropbox's Edgestore plugin. It's easily fixed, but I wonder how many other 3rd party plugins will be affected?

[JukkaL]

We should document the change in the issue that tracks plugin API changes (and potentially in mypy release notes).

The introduction of recursive types will break many plugins as well, probably more than this change.

[ilevkivskyi]

I think sometimes it may be useful. For example we probably want Union[List[T], List[S]] become just List[Any], not Union[List[Any], List[Any]]. I think a similar applies to the question below.

[ilevkivskyi]

This could be untangled by having the sub-checkers operate on an interface of the checker, like the semantic analyzer does.

I think this makes sense.

The "typeops tangle". Untangling a little further is probably possible but will require some thought and actual logic changes to code.

I would like to point out one nasty dependency: mypy.subtypes.unify_generic_callable() pulls in whole lot of things. This may be not easy to fix however.

Messages can definitely be removed.

Good idea!

[JukkaL]

This is great! I've been thinking about doing this myself for a long time but never got around to doing it.

[JukkaL]

We should document the change in the issue that tracks plugin API changes (and potentially in mypy release notes).

The introduction of recursive types will break many plugins as well, probably more than this change.

[ilevkivskyi]

See #6329 (likely caused by the one I mentioned). Also this PR essentially fixes #93

[msullivan]

I went and mostly fixed #6329; we can now import any module first except for type_visitor.py. (That one is fixable, it's just a bit more work).

Would it be worth adding a test at some point for that?

[ilevkivskyi]

@msullivan

Would it be worth adding a test at some point for that?

If it is not too tricky I think it makes sense to add some tests that would prevent us from recreating (or enlarging) those import cycles.

[JukkaL]

We could perhaps run another incremental build after self check in verbose mode and assert that the largest import cycle is below some size limit. Alternatively, we could modify verbose mode to write more details about SCCs in incremental mode and make more specific assertions, such as requiring that the nodes, typeops and checker cycles are disjoint.