TypeGuard functions not properly recognizing generic types

[wbolster]

Bug Report

It seems the typing.TypeGuard support is not able to properly discriminate a union of two (generic) types.

To Reproduce

• The example below defines two completely distinct types, Left[L] and Right[R], both of which happen to be generic and have a .value attribute.
• The LeftOrRight is a Union of those types.
• x is a variable of type LeftOrRight[str, int]. In this example it has value Left("")
• Using isinstance() works fine to figure out whether x is either Left or Right, and within the if/else block the types are correct, including the types L and R (both TypeVar)
• The is_left() and is_right() helpers use typing.TypeGuard, but Mypy does not recognize the types like it did with the isinstance() check.

from typing import Generic, TypeGuard, TypeVar, Union

L = TypeVar("L")
R = TypeVar("R")


class Left(Generic[L]):
    value: L

    def __init__(self, value: L):
        self.value = value


class Right(Generic[R]):
    value: R

    def __init__(self, value: R):
        self.value = value


LeftOrRight = Union[Left[L], Right[R]]


def is_left(obj: LeftOrRight[L, R]) -> TypeGuard[Left[L]]:
    return isinstance(obj, Left)


def is_right(obj: LeftOrRight[L, R]) -> TypeGuard[Right[R]]:
    return isinstance(obj, Right)


x: LeftOrRight[str, int] = Left("")
reveal_type(x)  # Revealed type is "Union[Left[builtins.str], Right[builtins.int]]"

if isinstance(x, Left):
    reveal_type(x)  # Revealed type is "Left[builtins.str]"
    x.value + ""  # only valid for str
else:
    reveal_type(x)  # Revealed type is "Right[builtins.int]"
    x.value + 10  # only valid for int

if is_left(x):
    reveal_type(x)  # Revealed type is "Left[L`-1]"
    x.value + ""  #  Unsupported left operand type for + ("L")  [operator]
else:
    reveal_type(x)  # Revealed type is "Union[Left[builtins.str], Right[builtins.int]]"
    x.value + 10  # Unsupported operand types for + ("str" and "int")  [operator]

Expected Behavior

is_left() and is_right() act as type-safe ways to discriminate between the two members of the Union, including the actual type of the (generic) .value attribute.

Actual Behavior

• reveal_type() returns wrong results
• type errors reported in what seems to be perfectly fine and very strict code

Your Environment

This happens with both Mypy 0.910 (current release as time of writing) and today's git checkout of the main branch.

[wbolster]

friendly ping 👋🏼 @hauntsaninja @ilevkivskyi, since you were involved with TypeGuard related issues fairly recently, e.g. #11061, #10899, #10647, #11007

[erictraut]

The current behavior is correct and intended. User-defined type guards intentionally do not provide type narrowing for the negative ("else") case. To quote from PEP 647:

User-defined type guards apply narrowing only in the positive case (the if clause). The type is not narrowed in the negative case.

This decision was based on experience with user-defined type guards in TypeScript. It supports the negative case, and this leads to problems in a number of real-world cases. The TypeScript authors admitted that if they able to go back in time, they would probably reverse their decision. We learned from this when drafting PEP 647.

Some built-in type guards (such as isinstance checks) can be safely narrowed in the negative case because the semantics are well known. In the case of user-defined type guards, the semantics depend on the use case, and a type checker cannot assume that positive and negative checks are symmetrical. There are many examples where they are not.

[erictraut]

I was too hasty in my response. I think I misunderstood the problem you were reporting.

You are correct that mypy isn't correctly applying the solved type variables in the case of the type guard.

    if is_left(x):
        reveal_type(x)  # Mypy: Left[L`-1], Pyright: List[str]
        x.value + ""
    elif is_right(x):
        reveal_type(x)  # Mypy: Left[str] | Right[int]], Pyright: Right[int]
        x.value + 10

[wbolster]

thanks for the response, @erictraut 🙏🏼

the lack of support for the negative case would account for the errors seen in the else clause of the example.

however, the positive case also fails currently:

if is_left(x):
    reveal_type(x)  # Revealed type is "Left[L`-1]"
    x.value + ""  #  Unsupported left operand type for + ("L")  [operator]

... even though the type should be fully know at this point: x is either Left[L] or Right[R], and the typeguard check should turn it into Left[L]. since the type of L is known, namely a str, i would expect that x.value + "" works without type errors.

the reveal_type() line seems to indicate this information is lost:

    reveal_type(x)  # Revealed type is "Left[L`-1]"

...which would make TypeGuard quite useless for generic types, specifically for things like Result (which is what triggered me to look into this in the first place).

[erictraut]

Yes, I agree. You can address the negative case by adding an elif check like I did in my last response, but that still leaves a problem.

[wbolster]

for full transparency: the problem i'm trying to solve is this one: rustedpy/result#69

...and it seems that this is not possible to achieve currently.

[MicaelJarniac]

I believe I've stumbled upon the same issue, but when using TypeGuard from typing_extensions: #11780

[3ok]

Here is a simpler example, taken from PEP647. I ran it using mypy v0.942 in Python 3.8.

from typing import TypeVar, Set, Any, Type
from typing_extensions import TypeGuard

_T = TypeVar("_T")

def is_set_of(val: Set[Any], type: Type[_T]) -> TypeGuard[Set[_T]]:
    return all(isinstance(x, type) for x in val)

def foo() -> Set[Any]:
    ...

some_set = foo()
if is_set_of(some_set, str):
    reveal_type(some_set)  # Revealed type is "builtins.set[_T`-1]"

I would have expected the revealed type to be Set[str], which is correctly inferred by pyright.

[Fulguritude]

Same problem on my end, using mypy 0.941. FYI, I'm trying to write a generic isinstance_builder(value, type_or_callback) which can either take in a type, or a custom isinstance-like callback of a single value.

I even tried the (very simple) PEP647 example:

_T = TypeVar("_T")

def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]:
    return len(val) == 2

def func(names: Tuple[str, ...]):
    if is_two_element_tuple(names):
        reveal_type(names)  # Tuple[str, str]
    else:
        reveal_type(names)  # Tuple[str, ...]

which returns

> python3.10 -m mypy playground2.py
playground2.py:22: note: Revealed type is "Tuple[_T`-1, _T`-1]"
playground2.py:24: note: Revealed type is "builtins.tuple[builtins.str, ...]"

instead of the expected

> python3.10 -m mypy playground2.py
playground2.py:22: note: Revealed type is "builtins.tuple[builtins.str, builtins.str]"
playground2.py:24: note: Revealed type is "builtins.tuple[builtins.str, ...]"