Unable to track Literal type conditionally imported from third party library

[johnthagen]

Bug Report

mypy seems unable to track a Literal type when the Literal is conditionally imported from a third party library.

To Reproduce

# main.py
from drf_spectacular.utils import Direction


def go(direction: Direction) -> int:
    if direction == "request":
        return 1
    elif direction == "response":
        return 2

Run:

mypy --strict main.py
main.py:4: error: Missing return statement
Found 1 error in 1 file (checked 1 source file)

Expected Behavior

No type error because mypy can detect this is exhaustive.

Actual Behavior

Error printed above

Your Environment

$ pip freeze
asgiref==3.4.1
attrs==21.2.0
click==8.0.1
Django==3.2.9
djangorestframework==3.12.4
drf-spectacular==0.21.0
inflection==0.5.1
jsonschema==4.2.1
mypy==0.910
mypy-extensions==0.4.3
pep517==0.10.0
pip-tools==6.4.0
pyrsistent==0.18.0
pytz==2021.3
PyYAML==6.0
sqlparse==0.4.2
toml==0.10.2
typing_extensions==4.0.0
uritemplate==4.1.1

• Mypy version used: 0.910
• Mypy command-line flags: --strict
• Mypy configuration options from mypy.ini (and other config files): N/A
• Python version used: 3.9.6
• Operating system and version: macOS 10.15

Third party code of interest:

• https://github.com/tfranzel/drf-spectacular/blob/03d315ced245db71cef1e45fd05a082b7dedc7aa/drf_spectacular/utils.py#L16

from drf_spectacular.drainage import (
    Final, Literal, error, get_view_method_names, isolate_view_method, set_override, warn,
)
...
Direction = Literal['request', 'response']

• https://github.com/tfranzel/drf-spectacular/blob/03d315ced245db71cef1e45fd05a082b7dedc7aa/drf_spectacular/drainage.py#L7

if sys.version_info >= (3, 8):
    from typing import Final, Literal, _TypedDictMeta  # type: ignore[attr-defined] # noqa: F401
else:
    from typing_extensions import (  # type: ignore[attr-defined] # noqa: F401
        Final, Literal, _TypedDictMeta,
    )

I tried modifying the third party package locally to remove some of the conditional imports, but I could never get the mypy error to go away. The only way I was able to get the error to stop was to redeclare (duplicate) the type locally:

from typing import Literal

Direction = Literal['request', 'response']


def go(direction: Direction) -> int:
    if direction == "request":
        return 1
    elif direction == "response":
        return 2

[sobolevn]

I'm trying to reproduce this:

import sys

if sys.version_info >= (3, 8):
    from typing import Literal
else:
    from typing_extensions import Literal

Direction = Literal['request', 'response']

x: Direction
reveal_type(x)  # N: Revealed type is "Union[Literal['request'], Literal['response']]"

Works fine. Let's try something else. Putting this into a different file also works for the latest master.

Can you please add reveal_type(direction) somewhere in go()?

[johnthagen]

Can you please add reveal_type(direction) somewhere in go()?

mypy --strict main.py
main.py:4: error: Missing return statement
main.py:5: note: Revealed type is "Any"
Found 1 error in 1 file (checked 1 source file)

Without --strict it works as expected:

mypy  main.py        
main.py:5: note: Revealed type is "Union[Literal['request'], Literal['response']]"

---

I too saw that when all put into a single file, everything worked fine.

My suspicion has been that because --strict is more picky about re-exporting items without them being placed in __all__ perhaps that was part of the root cause. Note that utils.py imports Literal from drainage.py, which does not export Literal.

I played around with modifying the third party library (adding __all__ = ["Literal"] to drainage.py) but was not successful is simplifying the error any more.

[sobolevn]

Yes, this is what I was missing 🙂

It won't work in --strict, because it implies --no-implicit-reexport flag.

Repro:

# a.py
import sys

if sys.version_info >= (3, 8):
    from typing import Literal
else:
    from typing_extensions import Literal


# b.py
from b import Literal

Direction = Literal['request', 'response']


# c.py
from a import Direction

x: Direction
reveal_type(x)  # Any

Solution

import sys

if sys.version_info >= (3, 8):
    from typing import Literal as Literal
else:
    from typing_extensions import Literal as Literal

After this change even with --no-implicit-reexport the revealed type is Union[Literal['request'], Literal['response']], not Any

I am not sure this is fixable outside of 3rd party lib. The only thing you can do is to disable implicit-reexport check.

[johnthagen]

@sobolevn Shouldn't another (better?) fix be to set __all__ appropriately? Would that satisfy --no-implicit-reexport?

I'm also curios why Literal as Literal works? Is as a signal to mypy that you mean to export something (perhaps with a different name)?

[sobolevn]

Yes, __all__ also works fine 👍
You can use either!

Literal as Literal works because it was decided that this syntax is going to highlight that something is "re-exported" from a module. It is quite compact and runtime friendly.

Any other names won't work. Only matching names count.

See https://mypy.readthedocs.io/en/latest/config_file.html?highlight=implicit-reexport#confval-implicit_reexport

[johnthagen]

Weird, when I change drainage.py in the third party repo like:

if sys.version_info >= (3, 8):
    from typing import Final as Final, Literal as Literal, _TypedDictMeta as _TypedDictMeta
else:
    from typing_extensions import (
        Final as Final, Literal as Literal, _TypedDictMeta as _TypedDictMeta,
    )

I still see the mypy error. (Was planning to submit a PR if I can get a fix to work locally).

[hauntsaninja]

Thanks sobolevn! I think the remaining thing that can be done here is #13965, so closing as a duplicate of that issue