Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use join to calculate better fallback for tuples #3575

Closed
JukkaL opened this issue Jun 20, 2017 · 7 comments · Fixed by #3952
Closed

Use join to calculate better fallback for tuples #3575

JukkaL opened this issue Jun 20, 2017 · 7 comments · Fixed by #3952

Comments

@JukkaL
Copy link
Collaborator

JukkaL commented Jun 20, 2017

Currently the fallback of named tuple types and tuples is Tuple[Any, ...] but we could use join of items to get a more precise fallback type after #3560 has been merged.

Pointers:

@JukkaL JukkaL changed the title Use join to calculate better fallback for named tuples Use join to calculate better fallback for tuples Jun 20, 2017
@gvanrossum
Copy link
Member

Remind me where the fallback type is used? I already think we're being too strict in some cases, e.g.

a = (1, 2)
b = a + ('', '')

give an error on the second line Unsupported operand types for + ("Tuple[int, int]" and "Tuple[str, str]") but I think it should just make the type of b: Tuple[int, int, str, str].

Also, this is still an error:

from typing import Tuple
a = (1, 2)
b = a + ('', '')  # type: Tuple[int, int, str, str]

There are actually two errors even:

__tmp__.py:3: error: Incompatible types in assignment (expression has type Tuple[int, ...], variable has type "Tuple[int, int, str, str]")
__tmp__.py:3: error: Unsupported operand types for + ("Tuple[int, int]" and "Tuple[str, str]")

@ilevkivskyi
Copy link
Member

Remind me where the fallback type is used?

The most important places are probably in subtypes.py where subtyping relationships between TupleTypes and Instances are decided (e.g. for named tuples), and in checkmember.py for look-up of attributes on tuples (like __add__ in your example).

We of course could be more flexible with tuples, and there are issues for this like #224, this could be done by special-casing. A more "broad" solution for such situations may be variadic generics. Anyway, both these ways are mostly unrelated to the joins in fallbacks.

@JukkaL
Copy link
Collaborator Author

JukkaL commented Jun 21, 2017

The fallback Any item type seems to used at least for multiplication and addition. It doesn't affect subtyping universally, since tuple subtyping has been special cased, but it might affect some special cases. Other operations might not be affected since they are special cased.

JukkaL pushed a commit that referenced this issue Sep 27, 2017
Forward references didn't work with anything apart from classes, for example 
this didn't work:

```
x: A
A = NamedTuple('A', [('x', int)])
```

The same situation was with `TypedDict`, `NewType`, and type aliases. The 
root problem is that these synthetic types are neither detected in first pass, 
nor fixed in third pass. In certain cases this can lead to crashes (first six issues 
below are various crash scenarios). This fixes these crashes by applying some 
additional patches after third pass.

Here is the summary of the PR:

* New simple wrapper type `ForwardRef` with only one field `link` is introduced 
  (with updates to type visitors)
* When an unknown type is found in second pass, the corresponding 
  `UnboundType` is wrapped in `ForwardRef`, it is given a "second chance" in 
  third pass.
* After third pass I record the "suspicious" nodes, where forward references and 
  synthetic types have been encountered and append patches (callbacks) to fix 
  them after third pass. Patches use the new visitor `TypeReplacer` (which is the 
  core of this PR).

Fixes #3340
Fixes #3419
Fixes #3674
Fixes #3685
Fixes #3799
Fixes #3836
Fixes #3881
Fixes #867
Fixes #2241
Fixes #2399
Fixes #1701
Fixes #3016
Fixes #3054
Fixes #2762
Fixes #3575
Fixes #3990
@ilevkivskyi
Copy link
Member

Hm, actually this issue was fixed by #3952 only partially: the first bullet in the first message is still missing. Precise fallbacks are generated only for named tuples, not for "normal" tuples, the latter still have Any in fallback.

@ilevkivskyi ilevkivskyi reopened this Sep 27, 2017
@ilevkivskyi ilevkivskyi self-assigned this Sep 29, 2017
@ilevkivskyi
Copy link
Member

ilevkivskyi commented Sep 29, 2017

Just a note (mostly to myself), when updating the fallbacks in patches we should take care that it doesn't interfere with subclassing "normal" tuples:

class B:
    attr: int

class C(B, Tuple[int, str]):
    label: str

(The patch is therefore probably should be made in semananl.py, where we know the context for TupleType, not in typeanal.py.)

@ilevkivskyi
Copy link
Member

Adding the patch is easy here, but this reveals several other errors/bugs, so it is not easy after all. Also tuple[Any] fallback is probably fine for now.

@ilevkivskyi
Copy link
Member

This was essentially fixed by #6442

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants