'in' operator rejected with union right operand

[JukkaL]

Mypy generates a false positive for this fragment:

from typing import Iterable, Container, Union

i: Iterable[str]
c: Container[str]
u: Union[Iterable[str], Container[str]]

'x' in i
'x' in c
'x' in u  # Unsupported right operand type for in ("Union[Iterable[str], Container[str]]")

This should be easy to fix.

[warsaw]

I think I've run into a similar problem, but given the error message, it's hard to tell. Here's the code sample:

def exclude_disabled(employees: List[Person]) -> List[Person]:
    """Excludes disabled employees."""
    return [employee for employee in employees if 'OU=zDisabled' not in employee.dn]

the error I get (with mypy 0.620) is:

error: Unsupported right operand type for in ("None")

My guess is that the conditional makes mypy think that employee may be None when it will never be so.

[ilevkivskyi]

There must be something else, I just tried a complete version of your sample:

from typing import List

class Person:
    dn: List[str]

def exclude_disabled(employees: List[Person]) -> List[Person]:
    """Excludes disabled employees."""
    return [employee for employee in employees if 'OU=zDisabled' not in employee.dn]

and it type-checks cleanly on master. What is the type of employee.dn?

[ariciputi]

Hi,
I have a similar use case involving Unions.

• Are you reporting a bug, or opening a feature request?

AFAICT this is a bug.

• Please insert below the code you are checking with mypy,
  or a mock-up repro if the source is private. We would appreciate
  if you try to simplify your case to a minimal repro.

Here it is a minimal code snippets:

from typing import Collection, Union

def function(a: Union[bool, Collection[str]], b: str) -> int:
    if a is True:
        return 1
    if a is False:
        return 2

    if b in a:
        return 3

    return 4

• What is the actual behavior/output?

example.py:10: error: Unsupported right operand type for in ("Union[bool, Collection[str]]")

• What is the behavior/output you expect?

This is a perfectly valid Python code, so mypy shouldn't complain.

• What are the versions of mypy and Python you are using?

$ mypy --version
mypy 0.720

Do you see the same issue after installing mypy from Git master?

Haven't tried

• What are the mypy flags you are using? (For example --strict-optional)

None

Hope you can help.

[JukkaL]

@ariciputi Your issue is different. Mypy doesn't understand that the is True and is False checks together are enough to narrow away the bool type from the union. Could you create a new issue about this (you can just copy your comment to a new issue)?

[ariciputi]

Hi @JukkaL,
I've opened this as you suggested.

Thanks.

[zomglings]

@JukkaL : This is an interesting case. Did some digging. This works:

from typing import Container, Iterable, List, Set, Union

c: Container[str]
i: Iterable[str]
l: List[str]
s: Set[str]
u: Union[Container[str], List[str], Set[str]]

'x' in c
'x' in i
'x' in l
'x' in s
'x' in u

As in your example, adding the Iterable[str] type to the Union kills it.

It seems like it has to do with the fact that the Iterable interface doesn't define a __contains__ method. That's pretty much all that a Container is, and in mypy/typeshed/stdlib/2and3/builtins.pyi, we see that the list stub as well as the the set stub both define the __contains__ method.

If we look in mypy/typeshed/stdlib/3/typing.pyi for the Iterable stuff, sure enough, no __contains__.

But, of course, if you define some dumb iterable, you can use in to check for containment successfully at runtime:

import collections.abc
from typing import Iterable

class MyIterable(collections.abc.Iterable):
    def __iter__(self) -> int:
        for i in range(10):
            yield i

a = MyIterable()
print(2 in a) # True
print(12 in a) # False

in is complicated -- https://stackoverflow.com/questions/12244074/python-source-code-for-built-in-in-operator/12244378#12244378

Simply adding a __contains__ to the Iterable stub fixes the behaviour in your example:

@runtime_checkable
class Iterable(Protocol[_T_co]):
    @abstractmethod
    def __iter__(self) -> Iterator[_T_co]: ...
    @abstractmethod
    def __contains__(self, x: object) -> bool: ...

I hesitate to make this change -- it seems like messing fundamentally with the protocol. I guess I'll push up a PR on typeshed and link to this issue and maybe carry the discussion forward there?

Any direction would be helpful, I've just started trying to understand this codebase.

[zomglings]

Yeah, changing Iterable to contain __contains__ breaks all kinds of tests even just in mypy. My naive attempt at resolution is inadvisable.

Would certainly appreciate guidance from someone more experienced with this code.

[MrCreosote]

I think this is also a case of this error (?), but is really odd because there's no need to upcast the value entries of the dict to object, as all the values are sets.

foo = {'a': set(),
       'b': {'c', 'd'}}

'z' in foo['a']

Results in

error: Unsupported right operand type for in ("object")

If a set literal rather than set() is used for the value of the a key, type checking is successful.

The workaround is to add an explicit type definition for foo.

[JelleZijlstra]

@MrCreosote that's a different problem, basically that mypy infers foo to be Dict[str, object]. As you say, an explicit type definition (or use of a TypedDict) solves it.

[JelleZijlstra]

This still reproduces. The weird part here to me isn't that we reject x in Iterable | Container, it's that we accept x in Iterable, since the Iterable protocol doesn't have __contains__. Is there some special casing that happens only at the top level but not inside a union?

[erictraut]

The in operator works with both __contains__ and iterables. See https://docs.python.org/3/reference/expressions.html#membership-test-operations for details. So x in Iterable should type check without errors, as should x in Iterable | Container.

[JelleZijlstra]

I know that; what's weird to me is that mypy knows about that too, but only if the Iterable is not in a Union. (For completeness, in also works with objects that define __getitem__.)