Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MyPy does not understand types exclusion using operator "is" #15788

Closed
socketpair opened this issue Jul 31, 2023 · 4 comments
Closed

MyPy does not understand types exclusion using operator "is" #15788

socketpair opened this issue Jul 31, 2023 · 4 comments
Labels
bug mypy got something wrong

Comments

@socketpair
Copy link

Real example:

from typing import Any, Collection, Generator, Iterable, TypeVar, cast
from itertools import chain, islice

T = TypeVar('T')  


def _split_on_batch_gen(items: Iterable[T]) -> Generator[Iterable[T], Any, None]:
    items = iter(items)
    stopper = object()
    while True:
        first_item = next(items, stopper)
        if first_item is stopper:
            break
        batch: chain[T] = chain((first_item,), islice(items, 0, 99))
        yield batch

or, reduced:

from typing import Any, Collection, Generator, Iterable, TypeVar, cast
from itertools import chain, islice

items = iter(range(1000))
stopper = object()
first_item = next(items, stopper)
if first_item is not stopper:
    batch: Iterable[int] = chain((first_item,), islice(items, 0, 99))

check here: https://mypy-play.net/?mypy=latest&python=3.11

gives:

main.py:8: error: Argument 1 to "chain" has incompatible type "tuple[object]"; expected "Iterable[int]"  [arg-type]

Which is obviously incorrect, since we checked that first_item is not an object(). I can not check for isinstance(fist_item, object) because every object is object :). Seems, MyPy understands isinstance() assertions, but does not understand is.

Seems, type of next(xxx, yyy) should be xxx[0] | Literal[yyy] instead of xxx[0] | yyy (pseudocode, I don't know how to express correctly)

@socketpair socketpair added the bug mypy got something wrong label Jul 31, 2023
@eli-schwartz
Copy link
Contributor

Which is obviously incorrect, since we checked that first_item is not an object().

Not really, no. All you checked was that it isn't the stopper sentinel by identity. It can still be any other object() object, so that comparison doesn't prove object | T can't be object anymore.

https://mypy.readthedocs.io/en/stable/type_narrowing.html suggests that for type narrowing you can use type(first_item) is object but this doesn't work either, perhaps because:

first_item = next(items, stopper)
reveal_type(first_item)

suggests that first_item is just object and that's it, it appears to have forgotten all about T. Perhaps object destroys union types.

For test purposes using dict() as a sentinel reveals that type(x) is dict type narrows inside the if, but doesn't type narrow in an else branch. type(x) is not dict doesn't appear to narrow at all?

@socketpair
Copy link
Author

What I said, return type of next should not be T| type(sentinel). It should be T | literally(sentinel).

Is it possible in Python type system ? In MyPy?

So check is stopper would reduce the union down to just T.

@TeamSpen210
Copy link
Contributor

typing.Literal[] is what you want, but it doesn't support arbitrary object() sentinels. The best method right now is a 1-item enum, or alternatively a @final-ed class if/once #15646 is merged. See also python/typing#689, and the draft PEP 661.

Or for this code in particular, you could use a try-except StopIteration, which would be fully typesafe.

@erictraut
Copy link

I think mypy is working correctly (as designed) in this case. Is there anything actionable from this issue, or should it be closed?

@JelleZijlstra JelleZijlstra closed this as not planned Won't fix, can't repro, duplicate, stale Aug 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong
Projects
None yet
Development

No branches or pull requests

5 participants