bool is not treated the same as Union[Literal[True], Literal[False]] in overloads

[MarcoGorelli]

Bug Report

The Type checking calls to overload docs read

if multiple variants match due to one or more of the arguments being a union, mypy will make the inferred type be the union of the matching variant returns

To Reproduce

from typing import overload, Optional, Literal, Union


@overload
def foo(inplace: Literal[True]) -> None: ...

@overload
def foo(inplace: Literal[False]) -> int: ...

def foo(inplace: bool) -> Optional[int]: ...

inplace_foo: bool
reveal_type(foo(inplace_foo))  # Revealed type is 'Any'



@overload
def bar(inplace: Literal[True]) -> None: ...

@overload
def bar(inplace: Literal[False]) -> int: ...

def bar(inplace: Union[Literal[True], Literal[False]]) -> Optional[int]: ...

inplace_bar: Union[Literal[True], Literal[False]]
reveal_type(bar(inplace_bar))  # Revealed type is 'Union[None, builtins.int]'

https://mypy-play.net/?mypy=latest&python=3.9&gist=051875ec08b32acb049577f7cd6085f7

Expected Behavior

I would've expected that typing a variable as bool would make it behave the same way as Union[Literal[True], Literal[False]]

Actual Behavior

bool is not treated the same as Union[Literal[True], Literal[False]]. However, that's all that bool can be, right, either True or False?

Your Environment

• Mypy version used: latest
• Mypy command-line flags:
• Mypy configuration options from mypy.ini (and other config files):
• Python version used: 3.9
• Operating system and version:

See the link to the mypy playground above to reproduce

[DevilXD]

It seems that the final type declaration of [bool] -> Optional[int] is simply not included as one of the overload options - and sure enough, adding an explicit overload, matching the signature of the function itself, makes this case type-check as expected:

@overload
def foo(inplace: Literal[True]) -> None: ...

@overload
def foo(inplace: Literal[False]) -> int: ...

# Added overload
@overload
def foo(inplace: bool) -> Optional[int]: ...

# Actual definition
def foo(inplace: bool) -> Optional[int]: ...

inplace_foo: bool
reveal_type(foo(inplace_foo))  # Revealed type is 'Union[builtins.int, None]'

This solution could be used for the time being. Still, I think that the non-overload signature typings should be included by default, if present.

[MarcoGorelli]

Thanks @DevilXD - yes, for the time being I'm adding in an extra overload of the bool case and it works fine

[DevilXD]

It looks like this is intended to work like that, based on the Python documentation (and PEP 484):

The @overload-decorated definitions are for the benefit of the type checker only, since they will be overwritten by the non-@overload-decorated definition, while the latter is used at runtime but should be ignored by a type checker.

Adding an overload like this is just the way to go here, even if it feels like boilerplate code.

[NeilGirdhar]

@DevilXD In theory, MyPy could use the overloads to figure out that all of possible Boolean values are covered. It may not be worth the trouble though compared with adding the overload you showed.

[DevilXD]

MyPy could use the overloads to figure out that all of possible Boolean values are covered.

This adds a lot of complexity. Right now, it's quite simple - look at all the overloads, and figure out the one matching, and use that. Once you get into "process all overloads to see if every possibility is covered", it gets unnecessarily complex.

Again, adding an overload like this, is the way to go, even if it feels like boilerplate code ¯\_(ツ)_/¯

[NeilGirdhar]

@DevilXD Makes sense, I agree 😄

[MarcoGorelli]

closing as it seems that bool isn't the union of Literal[True] and Literal[False] in any case, it's not an overload-specific thing:

$ cat t.py 
from typing import Literal, Union

def foo(x: Literal[True, False]) -> None:
    return None

x: bool
foo(x)
$ mypy t.py
t.py:7: error: Argument 1 to "foo" has incompatible type "bool"; expected "Union[Literal[True], Literal[False]]"
Found 1 error in 1 file (checked 1 source file)

[DevilXD]

I think it is an overload-specific thing - actually a bug even. Union[Literal[True], Literal[False]] should be compatible with bool, in every case. This isn't the same issue you've originally reported though, as it's been established that having to add an explicit overload is by design - the type-checker won't "combine" overloads, to figure out the return type, nor take the final definition typing into account when picking one, hence why you need to explicitly add it. Still, the example you've just provided, should type check correctly.

EDIT: It's slightly the same: bool is not treated the same as Union[Literal[True], Literal[False]] in general, not just in overloads.

[MarcoGorelli]

it's been established that having to add an explicit overload is by design - the type-checker won't "combine" overloads

Not sure what you mean here - it does take unions:

$ cat t.py 
from typing import Literal, Union, overload, Sequence

@overload
def foo(x: str) -> Sequence[int]: ...

@overload
def foo(x: int) -> int: ...

def foo(x): ...

x: Union[int, str]
reveal_type(foo(x))

$ mypy t.py
t.py:12: note: Revealed type is 'Union[builtins.int, typing.Sequence[builtins.int]]'

[DevilXD]

Well... I have no idea how MyPy does that then, as it technically shouldn't do that. On the linked issue, you can see someone explaining that it'd lead to quadratic complexity - not sure how true it is now.

Only one overload should be matched, picked, and used, for each invocation. It seems like MyPy is doing some additional work under the hood, to provide this union functionality - which is even more reason you should keep this issue open, as it's clearly not working as intended (with the bool example you've provided).

[MarcoGorelli]

It's behaving as documented here https://mypy.readthedocs.io/en/stable/more_types.html#type-checking-calls-to-overloads:

Second, if multiple variants match due to one or more of the arguments being a union, mypy will make the inferred type be the union of the matching variant returns:

The reason this doesn't happen in the example I originally posted is that bool isn't the union of Literal[True] and Literal[False].

On the linked issue, you can see someone explaining that it'd lead to quadratic complexity

I think they were referring to when the parameter you're overloading has parameters which comes before it which take defaults, see here for some examples of where you have to provide lots of overloads. Even then though, I think the complexity would be 2^n rather than n^2 (for each default parameter preceding the one you're overloading, you need to take care of the case where it's present and the one where it's absent)

[A5rocks]

FYI as of ef43416 this program now type checks:

from typing import Literal, Union

def foo(x: Literal[True, False]) -> None:
    return None

x: bool
foo(x)

IMO this issue should be reopened.

[NeilGirdhar]

What if T < bool, then shouldn't that require an explicit overload added in #10194 (comment)?

I'm no expert, but seems like this should fail. As @MarcoGorelli says, bool is not the union of true and false.

Edit: bool is not subclassable, so my argument doesn't apply.

[A5rocks]

ATM bool is (in mypy's eyes) a union of true and false, as shown by aforementioned code sample passing now.

To be more specific, in mypy's eyes bool is a subtype of Literal[True, False]. (Ever since ef43416)

[NeilGirdhar]

@A5rocks Yes, I agree with you that that I think that may be incorrect and the issue should be reopened. You may be interested to know that pyright also passes on your test, and it might be worthwhile asking the expert @erictraut what he thinks?

[erictraut]

From a type perspective, bool is equivalent to Literal[True, False], so it's fine for a type checker to treat the two as equivalent. The same is true for an enum, which is equivalent to the union of its constituent enumerated value literals. And the same is true for None which is equivalent to Literal[None]. Other literal types (int, str, bytes) cannot be expanded in this manner because it's not possible to exhaustively enumerate all of the literals for those types.

I think it's therefore fine for the code sample posted above by @A5rocks to pass type checking, as it currently does with mypy and pyright.

from typing import Literal

def foo(x: Literal[True, False]) -> None:
    return None

def func(x: bool):
    foo(x) # No Error (for mypy or pyright)

Although bool is equivalent to Literal[True, False], a type checker may choose not to expand the former to the latter. There are cases where it's advantageous to do so and others where it's not. In particular, I think it makes sense for type narrowing operations where you are testing for exhaustive matching, like this:

def func(val: bool):
    match val:
        case True:
            print("True!")
        case False:
            print("False!")
        case _:
            assert_never(val) # No error (for mypy or pyright)

In this case, pyright and mypy expand bool into Literal[True, False] for purposes of type narrowing and exhaustion checking. The same applies to an enum.

I'm not convinced that bool and enums should be expanded for purposes of overload matching. Such cases are rare and would add significant performance overhead for overload matching. I therefore think it's reasonable to ask developers who create polymorphic functions whose return types that depend on bool inputs to provide a fallback overload to handle the case where the bool argument is not statically determined to be either True or False at analysis time. The same applies to enums.

from typing import Literal, overload

@overload
def func(x: Literal[True]) -> int: ...
@overload
def func(x: Literal[False]) -> str: ...
def func(x: bool) -> int | str:
    return 0 if x else ""

def foo(x: bool):
    func(x)  # Error: no overload matches (both mypy and pyright)

So, from my perspective, mypy is doing the right thing currently, and I would leave this bug closed. Mypy and pyright behaviors are currently in alignment in this regard.

[NeilGirdhar]

In this case, pyright and mypy expand bool into Literal[True, False] for purposes of type narrowing and exhaustion checking. The same applies to an enums.

I think that's perfectly fine, but one of the key ingredients that needs to be mentioned is that both Enum and bool cannot be extended in Python (I just learned this). That is,

from enum import Enum

class Color(Enum):
    red = 1
    green = 2

class T(Color):  # Raises TypeError.
    pass

class U(bool):  # Raises TypeError.
    pass

If these types were subclassable, then the assumption that these types are a known exhaustive set of literals would not hold.