Preserve annotations and inferred types when generating stubs

[JukkaL]

If a user maintains a 3rd party library module that is type checked using mypy, there is no easy to way to automatically generate annotated typeshed stubs for that library, even though this could be pretty useful. We could perhaps extend stubgen to do this job.

Here's how this could work internally:

• Do the equivalent of a mypy run against a target module/package.
• Generate stubs based on the mypy AST, including inferred types.

The main difference from previous stubgen related proposals would be the inclusion of inferred types in the stubs. For example, consider this program:

x = 0
a = [x]

The generated stub could look like this:

from typing import List

x: int
a: List[int]

Here are a few things that complicate this task:

• It may be hard to generate stubs that work with both Python 2 and 3. However, maybe it's okay to generate separate stubs for 2 and 3, since the stubs hopefully won't need a lot of manual review as they are automatically generated.
• It may be hard to include platform-specific definitions. One approach would be to generate separate ASTs for all supported platforms and somehow merge them, generating conditional platform checks automatically.

[ilevkivskyi]

There is an existing PR #3169 for improving stubgen. But it only preserves existing annotations, type aliases, and TypeVars (i.e. no inferred annotations).

[JelleZijlstra]

I was thinking about this in the context of python/typing#84.

An alternative for fully annotated libraries could be to just install their full code in the "stubs" directory, so we don't have to generate separate stub files. However, that may slow down mypy and it may cause problems when user code is type checked with more restrictive flags than the library.

[JukkaL]

User code may also be fail to type check because of differences between mypy versions. Stubs are more likely to be compatible across mypy versions than actual code.