Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Type of TypedDict is ambiguous" despite tag fields being distinct Literal types #8533

Closed
florimondmanca opened this issue Mar 13, 2020 · 4 comments · Fixed by #14505
Closed

Comments

@florimondmanca
Copy link

florimondmanca commented Mar 13, 2020

Hi there,

Thanks for all the awesome work on mypy! It's brought incredible improvements to my workflow. :-)

I'm reporting what could be a bug with tagged unions, which were released in 0.770. As of today this is also reproducible against master.

Unfortunately this bug makes usage of tagged unions impractical. Last time I beta-tested it (a few days before the 0.770 release — perhaps around March 1st), it worked though.

Let me know if there's anything I can do to help resolve!

Reproduction case

# example.py
from typing import TypedDict, Literal, Union

# TypedDict interfaces with distinct tag fields.
A = TypedDict('A', {'@type': Literal['type-a'], 'value': str})
B = TypedDict('B', {'@type': Literal['type-b'], 'value': str})

# Tagged union of A and B.
C = Union[A, B]

c: C = {
    '@type': 'type-a',
    'value': 'test',
}

Expected behavior

$ mypy example.py
# OK - no errors

Actual behavior

$ mypy example.py
example.py:7:8: error: Type of TypedDict is ambiguous, could be any of ("A", "B")
example.py:7:8: error: Incompatible types in assignment (expression has type "Dict[str, str]", variable has type "Union[A, B]")
Found 2 errors in 1 file (checked 1 source file)

Version info

  • Python 3.8.1
  • Mypy 0.770
  • Mypy flags: none (and no mypy.ini in the current workdir).

Additional context

I notice an ambiguous case is tested here:

[case testTypedDictUnionAmbiguousCase]
from typing import Union, Mapping, Any, cast
from typing_extensions import TypedDict, Literal
A = TypedDict('A', {'@type': Literal['a-type'], 'a': str})
B = TypedDict('B', {'@type': Literal['a-type'], 'a': str})
c: Union[A, B] = {'@type': 'a-type', 'a': 'Test'} # E: Type of TypedDict is ambiguous, could be any of ("A", "B") \
# E: Incompatible types in assignment (expression has type "Dict[str, str]", variable has type "Union[A, B]")
[builtins fixtures/dict.pyi]

But the snippet above is unambiguous: Literal['type-a'] and Literal['type-b'] don't overlap at all. Is the test above passing in master?

@florimondmanca
Copy link
Author

florimondmanca commented Mar 13, 2020

Ah, so I realize this might actually be a feature request… It seems we can't use a tagged union to declare variable, but only when declaring a function parameter, and then pattern-match inside the function based on whichever exact type is passed as a parameter, eg:

from typing import TypedDict, Literal, Union

# TypedDict interfaces with distinct tag fields.
A = TypedDict('A', {'@type': Literal['type-a'], 'value': str})
B = TypedDict('B', {'@type': Literal['type-b'], 'value': str})

# Tagged union of A and B.
C = Union[A, B]

def process(c: C): pass

a: A = {'@type': 'type-a', 'value': 'test'}
process(a)  # OK

I think (?) it would make sense to allow declaring a variable typed after a tagged union, given that we can do it with regular unions, eg…

from typing import TypedDict, Literal, Union

class A: pass
class B: pass

C = Union[A, B]

c: C = A()  # OK

@msullivan
Copy link
Collaborator

I think that I agree that this is a bug.

There is a pretty easy workaround, though:

a: A = ...
c: C = a

works

@msullivan msullivan added bug mypy got something wrong priority-2-low labels Mar 14, 2020
@florimondmanca
Copy link
Author

Hi @msullivan, thanks :)

FWIW, my precise use case was yielding dicts that match a tagged union inside a generator function:

from typing import TypedDict, Literal, Union, Iterator

A = TypedDict('A', {'@type': Literal['type-a'], 'value': str})
B = TypedDict('B', {'@type': Literal['type-b'], 'value': str})
C = Union[A, B]

def stream() -> Iterator[C]:
    yield {'@type': 'type-a', 'value': 'test'}
$ mypy example.py    
example.py:9: error: Incompatible types in "yield" (actual type "Dict[str, str]", expected type "Union[A, B]")
example.py:9: error: Type of TypedDict is ambiguous, could be any of ("A", "B")
Found 2 errors in 1 file (checked 1 source file)

In that case your workaround works too, although pretty clunky:

def stream() -> Iterator[C]:
    a: A = {'@type': 'type-a', 'value': 'test'}
    yield a

I'm new to the mypy codebase, but if you've got any pointers on where this might come from I can try and investigate a bit more.

@Michael0x2a
Copy link
Collaborator

The strategy I personally use is to grep for part of the error message. That usually points you to some location in messages.py, which is responsible for constructing and reporting error messages. Then, you can work backwards to figure out how exactly we ended up reaching that particular point in the code.

For example, grepping for "Type of TypedDict is ambiguous" should point you to typed_dict_ambiguous in messages.py. And from there, we can see this is only called by find_typeddict_context in checkexpr.py.

And finally, if we step through what this function is doing, we discover that mypy is comparing the TypedDicts against the dict expression only by their keys and bails out early if there are multiple potentially valid TypedDict contexts. (And the caller of find_typeddict_context is what's actually responsible for checking the values).

This is where the bug is coming from: this filtering strategy is probably too aggressive/the whole flow of logic here could probably do with a second look.

pgjones added a commit to pgjones/hypercorn that referenced this issue Dec 24, 2020
This makes use of the TypedDict (added to Python in 3.8 and available
in the typing_extensions package) to specify the types of the key
values in the ASGI messages. This then ensures that the messages are
correctly constructued and used in the code.

Note the type ignores are due to this issue
python/mypy#8533.
bluetech added a commit to bluetech/mypy that referenced this issue Sep 26, 2021
Fixes python#8533.

Previously, given a union of TypedDicts, e.g. `A|B` in

```py
from typing import TypedDict, Literal, Union

class A(TypedDict):
    tag: Literal['A']
    extra_a: str

class B(TypedDict):
    tag: Literal['B']
    extra_b: str
```

when needing to disambiguate the union, e.g.

```
td: A|B = {
    'tag': 'A',
    'extra_a': 'foo',
}
```

mypy would only consider the *keys* of the dict expression and
TypedDict, e.g. 'tag' and 'extra_a'. But if multiple members of the
union have the same shape, only distinguished by a value type, the
disambiguation fails, e.g.

```py
class A(TypedDict):
    tag: Literal['A']

class B(TypedDict):
    tag: Literal['B']

td: A|B = {  # E: Type of TypedDict is ambiguous, could be any of ("A", "B")
    'tag': 'A',
}
```

To allow this, also consider the types of the dict expression's values
when narrowing the candidates from the union.
bluetech added a commit to bluetech/mypy that referenced this issue Sep 26, 2021
Fixes python#8533.

Previously, given a union of TypedDicts, e.g. `A|B` in

```py
from typing import TypedDict, Literal, Union

class A(TypedDict):
    tag: Literal['A']
    extra_a: str

class B(TypedDict):
    tag: Literal['B']
    extra_b: str
```

when needing to disambiguate the union, e.g.

```
td: A|B = {
    'tag': 'A',
    'extra_a': 'foo',
}
```

mypy would only consider the *keys* of the dict expression and
TypedDict, e.g. 'tag' and 'extra_a'. But if multiple members of the
union have the same shape, only distinguished by a value type, the
disambiguation fails, e.g.

```py
class A(TypedDict):
    tag: Literal['A']

class B(TypedDict):
    tag: Literal['B']

td: A|B = {  # E: Type of TypedDict is ambiguous, could be any of ("A", "B")
    'tag': 'A',
}
```

To allow this, also consider the types of the dict expression's values
when narrowing the candidates from the union.
JukkaL pushed a commit that referenced this issue Jan 23, 2023
Fixes #14481 (regression)
Fixes #13274
Fixes #8533

Most notably, if literal matches multiple items in union, it is not an
error, it is only an error if it matches none of them, so I adjust the
error message accordingly.

An import caveat is that an unrelated error like `{"key": 42 + "no"}`
can cause no item to match (an hence an extra error), but I think it is
fine, since we still show the actual error, and avoiding this would
require some dirty hacks.

Also note there was an (obvious) bug in one of the fixtures, that caused
one of repros not repro in tests, fixing it required tweaking an
unrelated test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants