Stub generation and module re-exports, #noqa

[rtpg]

Many larger libraries break up their packages into multiple modules, then "re-export" the modules in the package's __init__.py

For example, in django.db.models:

from django.db.models.fields import *  # NOQA
from django.db.models.fields.files import FileField, ImageField  # NOQA

This allows you to import FileField and CharField (from django.db.models.fields) directly from django.db.models

stubgen currently provides these re-exports in the case of import *, but not in the second import. So FileField is missing from the django.db.models stub.

This makes sense, since all imports are re-exports, but that might not be the wanted API (should I really be importing requests from some random module in the middle of my application? probably not). But, of course, sometimes it's on purpose.

This does come up a lot, though, so I was thinking of 3 possible solutions:

• Re-export all imports when doing stub generation
  This will cause generated stubs to best represent reality. This solution will make initial annotations easier, but also makes the output messier than what authors might want their public API to look like.
• Re-export "unused" imports.
  This is tricky because intent is hard to decipher. In django's case, unused imports (with # noqa) are almost certainly meant for re-exporting. We could look for such comments on import lines, or figure out some other kind of heuristic.
• Do nothing
  This issue comes up with big libraries, but not so much in application code. Big libraries usually have enough people to power through this sort of thing.

I tried looking in the issues and didn't see this coming up, but the easier the stub generation output is, the more likely people will get the third party annotation work done, so it would be cool to get this right. I think the second one is reasonable, first one is the easiest to implement.

[gvanrossum]

The question is really how does stubgen know which modules are meant for
re-export? I don't want to export all imported modules automatically --
not even when they're submodules or when they're unused (the unused-less
seems a very weak signal).

In manually generated stubs we just do import Foo as Foo.

I don't understand your use of # NOQA -- is that what it signals? (It
could mean many things in general.)

If you can come up with a PR to stubgen that implements a reasonable
heuristic we'll consider it, otherwise the status quo wins (3).