Repository navigation
Should __init__.py imports be treated as exports with --no-implicit-reexport behaviour? #10198
Description
Activity
I personally believe here mypy is in the right. And mypy should fail as it's doing now. However, the project might want to opt-in to an implicit re-export behavior if they don't want to use an explicit export instead. This setting should be done though by the project and not the downstream users though. Perhaps
py.typefile could contain some metadata related to this.To put it in scope why mypy (rightly) complains here, consider the following example:
from typing import List def generate_list() -> List[int]: return [1]
This module will provide both
generate_listandListas available to import for the runtime. I think we can agree though thatListshould not be imported from this module, but rather keep importing it fromtyping. We need some way to explicitly state what's just used by the module, and what is meant to be imported by end-users.Historically the
__all__was expressing this. Linters/IDEs used this to suggest imports as such. For__init__.pythefrom x import y as ywas also used/worked.Reacted by David Tucker, Arseniy Terekhin, Jürgen Gmach and Anton AgestamWhat is bad about using
__all__?Some people find it odd that you have to:
- have to define twice your names (once to import/define, once in
__all__) and these definitions are not close to each other, - the definition entries within
__all__is a string (important because typos are not caught early because of this)
In practice means making sure your
__all__list is up to date and correct is an error-prone job. I can live with these issues personally, but I understand the concern.Reacted by Jürgen Gmach, Joshua Bronson, jessekv, Edaqa Mortoray, Trevor Gross, Yossi Rozantsev, Michael Blaß, Roger Gonzalez and John Hagen- have to define twice your names (once to import/define, once in
I wonder if it would make sense for
mypyto detect something like__all__ = list(locals())
as a way to disable
no_implicit_reexportfrom the package side 🤔Since Pallets ended up reintroducing this conversation, just wanted to chime in that after looking at the issue more we agree with @gaborbernat that mypy is right here, so we'll make bugfix releases with explicit exports in the form
import name as name.Either method is a bit noisy to me, so I think we could still brainstorm how this could be improved to make it easier / more natural to maintain.
Reacted by jessekv and Maciej DąbrowskiI don't think
__init__.pyshould be treated specially for two reasons:- It's inconsistent and inconsistency is confusing.
- Often,
__init__.pyis not just a collection of re-exports, but contains the bulk of the implementation, with a few utilities or special cases in submodules. In this case, re-exports are as problematic as in any other file. - Edit: A third reason is that
foo.pyandfoo/__init__.pyshould work the same.
Reacted by David Lord, Bernát Gábor, Shantanu, dquitmann-op, Julian Berman, Roger Gonzalez and Anton AgestamEither method is a bit noisy to me, so I think we could still brainstorm how this could be improved to make it easier / more natural to maintain.
I'd imagine we should enquire the python core developers about this 🤔 so probably someone should open a topic on discuss.python.org
- have to define twice your names (once to import/define, once in
__all__) and these definitions are not close to each other,
It's not limited to defining them twice, it's more than that. Consider I have a file
abc.pyin a module, and I use__all__ = ['One', 'Two']. Now I wish to export those public names from the module as a whole, so in my__init__.pyfile I have to list the names explicitly again.I could live with having explicit exports be mentioned in
__all__, but I think ti should be only once. There should be a way to re-export all those public names again.The use of strings instead of symbols in
__all__is also an issue since it delays detection of defects.- have to define twice your names (once to import/define, once in
At this point I'm kind of wishing we had an explicit
exportdirective that could list individual names, rename, or all public symbols from a module. I guess that's why we need to get the Python code developers in the discussion.Reacted by Andrei Nesterov, Trevor Gross, Michael Blaß, Marc Bresson, Anzhari Purnomo and Steve ByerlyI wish star imports would re-export. Consider the following:
$ head main.py p/* ==> main.py <== import p def main() -> None: print(p.x + p.y) ==> p/a.py <== x = 1 ==> p/b.py <== y = 2 ==> p/__init__.py <== from .a import * from .b import * $ mypy --strict main.py main.py:4: error: Module has no attribute "x" main.py:4: error: Module has no attribute "y" Found 2 errors in 1 file (checked 1 source file)While star imports are a bit controversial in general, I think re-exporting names is definitely a common pattern for them, and we use it a lot in typeshed. It would be nice if it would also work outside typeshed.
Reacted by Maciej DąbrowskiThinking about how it's done in Pallets, if we ever use an import in the module, then it's not intended for export. The only time we want imports to be exported is if they're not used in the module, which happens to only be done in
__init__files.Perhaps mypy could use this as a heuristic. We'd still be using flake8 to detect unused imports (or ignore them in
__init__files, so it wouldn't hide real issues with imports. If necessary, it could be behind anexport_unused_importconfig.#10826 is similar to this, ie.
__all__usage in__init__.py.I think @erictraut's comment could be very useful in this context as well: #10826 (comment)
But summarizing, there is a proposal for the indentifiable
__all__idioms: https://github.com/python/typing/blob/master/docs/source/libraries.rst#library-interfaceReacted by Hash and Michael Blaß- added a commit that references this issue
on Jul 8, 2022
I'm uncertain whether
--no-implicit-reexporthas a "correct" behaviour when it comes to__init.py__files.If I have this
__init__.pyfile:My intent is to export
MyNamefrom the module. Currently this does not work however, and I need to do either:or
Both of which are redundant.
I saw this issue on stub-generation that seems to imply that imports in an init (or otherwise unused imports) should be considered exports.
Shouldn't
--no-implicit-reexporttreat init files as doing exports for the module without the code having to use the redundant...as...or__all__form?