Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with @overload and Iterator/Generator #17927

Open
NoamNol opened this issue Oct 12, 2024 · 2 comments
Open

Problem with @overload and Iterator/Generator #17927

NoamNol opened this issue Oct 12, 2024 · 2 comments
Labels
bug mypy got something wrong

Comments

@NoamNol
Copy link

NoamNol commented Oct 12, 2024

See last line (Playground):

from collections.abc import Iterator, Generator
from typing import Any, Iterable, TypeVar, overload
from typing_extensions import assert_type

T = TypeVar("T")
IterableT = TypeVar("IterableT", bound=Iterable)


@overload
def read_iterator(iterable: Iterator[T]) -> list[T]: ...


@overload
def read_iterator(iterable: IterableT) -> IterableT: ...


def read_iterator(iterable: Iterable) -> Iterable:
    if isinstance(iterable, Iterator):
        return list(iterable)
    else:
        return iterable


def _create_generator() -> Generator[int, Any, None]:
    for i in range(3):
        yield i


def test_read_iterator() -> None:
    an_iterator = iter(range(3))
    assert_type(read_iterator(an_iterator), list[int])

    a_range, a_list, a_tuple, a_set, a_dict = range(3), [1, 2, 3], (1, 2, 3), {1, 2, 3}, {"a": 1}
    assert_type(read_iterator(a_range), range)
    assert_type(read_iterator(a_list), list[int])
    assert_type(read_iterator(a_tuple), tuple[int, int, int])
    assert_type(read_iterator(a_set), set[int])
    assert_type(read_iterator(a_dict), dict[str, int])
    
    a_generator = _create_generator()
    assert_type(a_generator, Generator[int, Any, None])
    assert_type(read_iterator(a_generator), list[int])  # error: Expression is of type "Any", not "list[int]"  [assert-type]

Mypy doesn't like this code:

error: Expression is of type "Any", not "list[int]"  [assert-type]

Please remember that Generator is a subtype of Iterator:

>>> isinstance(a_generator, Iterator)
True

I think this is a bug in Mypy. The Generator should match the signature of def read_iterator(iterable: Iterator[T]) -> list[T]


Versions:

> python -V             
Python 3.11.1

> mypy --version
mypy 1.11.2 (compiled: yes)

❯ system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: macOS 13.6.7 (22G720)
      Kernel Version: Darwin 22.6.0

Related: python/typing#253

@NoamNol NoamNol added the bug mypy got something wrong label Oct 12, 2024
@NoamNol
Copy link
Author

NoamNol commented Oct 12, 2024

In VSCode the returned type of read_iterator(a_generator) is list[int] with not problem:

VSCode-generator-list-type 2024-10-12 at 22 29 39

@NoamNol NoamNol closed this as completed Oct 12, 2024
@NoamNol NoamNol reopened this Oct 12, 2024
@hauntsaninja
Copy link
Collaborator

This looks related to the case mypy has where if multiple overloads match due to the presence of Any, in some situations it will infer Any. It looks like that code path is getting triggered by the Any in Generator's send type. (I think the semantics here are a little ad hoc, the Typing Council does have specifying behaviour here on its list)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong
Projects
None yet
Development

No branches or pull requests

2 participants