Fix tuple[Any, ...] subtyping #16108

hauntsaninja · 2023-09-14T00:09:09Z

Follow up to #16073 and #16076
Fix needed for https://github.com/python/mypy/pull/16053/files#r1316481395

I add test cases that would have caught my previous incorrect PR. I add an explicit case for the new desirable behaviour we see with zip.

Follow up to python#16073 and python#16076 Fix needed for https://github.com/python/mypy/pull/16053/files#r1316481395 I add test cases that would have caught my previous incorrect PR. I add an explicit case for the new desirable behaviour we see with zip.

hauntsaninja · 2023-09-14T00:50:32Z

Hmm, this breaks commutativity of meet. Easy to make an ad hoc patch...

ilevkivskyi

Generally looks good, but there are some suspicious parts, let's see what primer output will be after fixing these.

ilevkivskyi · 2023-09-14T09:48:07Z

mypy/meet.py

@@ -90,10 +90,15 @@ def meet_types(s: Type, t: Type) -> ProperType:
            return t

    if not isinstance(s, UnboundType) and not isinstance(t, UnboundType):
+        swap = isinstance(t, TupleType) and not isinstance(s, TupleType)
+        if swap:
+            s, t = t, s


This looks suspicious, the check below is for proper subtype, that should not now anything about Any.

ilevkivskyi · 2023-09-14T09:49:17Z

mypy/subtypes.py

+                    and len(left.args) == 1
+                    and isinstance(get_proper_type(left.args[0]), AnyType)
+                ):
+                    return True


I think this should return not self.proper_subtype similar to above.

ilevkivskyi · 2023-09-14T09:50:15Z

mypy/subtypes.py

-            # TODO: we need a special case similar to above to consider (something that maps to)
-            # tuple[Any, ...] a subtype of Tuple[<whatever>].
+                if (
+                    left.type.fullname == "builtins.tuple"


I think this should use has_base() + map_instance_to_supertype() logic like above. Otherwise this breaks transitivity of subtyping.

ilevkivskyi · 2023-09-14T09:54:08Z

test-data/unit/check-tuples.test

+takes_tuple_aa(inst_tuple_aa)
+takes_tuple_aa(inst_tuple_aa_subclass)
+# This next one should maybe not be an error
+takes_tuple_aa(inst_tuple_any_subclass)  # E: Argument 1 to "takes_tuple_aa" has incompatible type "tuple_any_subclass"; expected "Tuple[A, A]"


This should be allowed (see above, if A <: B and B <: C, then A must be <: C).

This reverts commit c2250f8.

hauntsaninja · 2023-09-14T19:51:57Z

Thanks for the review! We lose several of the true positive warnings on pandas etc (the case tested by testTupleOverloadZipAny), but this is a right change and we should fix by changing overload return inference.

ilevkivskyi · 2023-09-14T20:31:49Z

mypy/subtypes.py

-            # TODO: we need a special case similar to above to consider (something that maps to)
-            # tuple[Any, ...] a subtype of Tuple[<whatever>].
+                if (
+                    mapped.type.has_base("builtins.tuple")


Why did you need the last commit? This condition is always true because of the surrounding if (each tuple fallback has builtins.tuple in MRO), and also now this check looks wrong because it may trigger in unrelated cases involving something like class M(Generic[T], tuple[int, ...]): ....

What you had before was almost right, except you didn't need the check len(mapped.args) == 1, it should be always true if mapped.type.fullname == "builtins.tuple".

I misinterpreted your comment here #16108 (comment)

~~Ugh, I tried to test the class M(Generic[T], tuple[int, ...]): ... case but I did it backwards~~ I can't actually find a test, even with various NamedTuples

If you will not be able to come up with a test case, don't try to hard. It may be not easy to trigger such edge case (because of how user defined tuple types are represented). You already have enough test cases :-)

(I would still remove the unneeded len check)

This reverts commit 9840c37.

for more information, see https://pre-commit.ci

github-actions · 2023-09-14T21:17:45Z

Diff from mypy_primer, showing the effect of this PR on open source code:

mitmproxy (https://github.com/mitmproxy/mitmproxy)
+ mitmproxy/coretypes/multidict.py:153: error: Unused "type: ignore" comment  [unused-ignore]
+ mitmproxy/coretypes/multidict.py:167: error: Unused "type: ignore" comment  [unused-ignore]
+ mitmproxy/tools/console/grideditor/base.py:170: error: Unused "type: ignore" comment  [unused-ignore]

Expression (https://github.com/cognitedata/Expression)
- expression/collections/block.py:814: error: Overloaded function implementation cannot produce return type of signature 1  [misc]
- expression/collections/block.py:814: error: Overloaded function implementation cannot produce return type of signature 2  [misc]
- expression/collections/block.py:814: error: Overloaded function implementation cannot produce return type of signature 3  [misc]
- expression/extra/parser.py:219: error: Overloaded function implementation cannot produce return type of signature 1  [misc]
- expression/extra/parser.py:219: error: Overloaded function implementation cannot produce return type of signature 2  [misc]

steam.py (https://github.com/Gobot1234/steam.py)
- steam/trade.py:321: error: Incompatible types in assignment (expression has type "tuple[Any, ...]", base class "Asset" defined the type as "tuple[str, str, str, str, str, str, str]")  [assignment]
- steam/ext/csgo/backpack.py:263: error: Incompatible types in assignment (expression has type "tuple[Any, ...]", base class "Asset" defined the type as "tuple[str, str, str, str, str, str, str]")  [assignment]
- steam/ext/csgo/backpack.py:330: error: Incompatible types in assignment (expression has type "tuple[Any, ...]", base class "Asset" defined the type as "tuple[str, str, str, str, str, str, str]")  [assignment]
- steam/ext/csgo/backpack.py:330: error: Cannot determine type of "REPR_ATTRS"  [has-type]
- steam/ext/tf2/backpack.py:80: error: Incompatible types in assignment (expression has type "tuple[Any, ...]", base class "Asset" defined the type as "tuple[str, str, str, str, str, str, str]")  [assignment]

mypy (https://github.com/python/mypy)
+ mypy/main.py:1362: error: Unused "type: ignore" comment  [unused-ignore]

operator (https://github.com/canonical/operator)
- ops/storage.py:191: error: Incompatible types in "yield" (actual type "tuple[Any, ...]", expected type "tuple[str, str, str]")  [misc]

jax (https://github.com/google/jax)
+ jax/_src/scipy/special.py:293: error: Unused "type: ignore" comment  [unused-ignore]

mongo-python-driver (https://github.com/mongodb/mongo-python-driver)
+ pymongo/mongo_client.py:1681: error: Unused "type: ignore" comment  [unused-ignore]

ibis (https://github.com/ibis-project/ibis)
- ibis/util.py:126: error: Incompatible return value type (got "tuple[Any, ...]", expected "tuple[V]")  [return-value]
- ibis/backends/oracle/__init__.py:51: error: Incompatible types in assignment (expression has type "tuple[Any, ...]", base class "ExprTranslator" defined the type as "tuple[type[Lag], type[Lead], type[DenseRank], type[MinRank], type[FirstValue], type[LastValue], type[PercentRank], type[CumeDist], type[NTile]]")  [assignment]
- ibis/backends/trino/compiler.py:23: error: Incompatible types in assignment (expression has type "tuple[Any, ...]", base class "ExprTranslator" defined the type as "tuple[type[Lag], type[Lead], type[DenseRank], type[MinRank], type[FirstValue], type[LastValue], type[PercentRank], type[CumeDist], type[NTile]]")  [assignment]
- ibis/backends/snowflake/__init__.py:58: error: Incompatible types in assignment (expression has type "tuple[Any, ...]", base class "ExprTranslator" defined the type as "tuple[type[Lag], type[Lead], type[DenseRank], type[MinRank], type[FirstValue], type[LastValue], type[PercentRank], type[CumeDist], type[NTile]]")  [assignment]

hauntsaninja · 2023-09-14T21:28:03Z

Thank you!

hauntsaninja added 2 commits September 13, 2023 17:07

Fix tuple[Any, ...] subtyping

2044720

Follow up to python#16073 and python#16076 Fix needed for https://github.com/python/mypy/pull/16053/files#r1316481395 I add test cases that would have caught my previous incorrect PR. I add an explicit case for the new desirable behaviour we see with zip.

error

6a4df2b

This comment has been minimized.

Sign in to view

fix meet

c2250f8

hauntsaninja force-pushed the tuple-any-subtyping branch from 2f1264b to c2250f8 Compare September 14, 2023 01:02

This comment has been minimized.

Sign in to view

hauntsaninja marked this pull request as ready for review September 14, 2023 05:10

ilevkivskyi reviewed Sep 14, 2023

View reviewed changes

hauntsaninja added 5 commits September 14, 2023 03:07

Revert "fix meet"

e64a6cd

This reverts commit c2250f8.

suggestion

17ad1ca

use mapped type

d5812b6

zip

03fda8b

base

9840c37

This comment has been minimized.

Sign in to view

ilevkivskyi reviewed Sep 14, 2023

View reviewed changes

hauntsaninja and others added 3 commits September 14, 2023 13:33

Revert "base"

2c4f791

This reverts commit 9840c37.

check

e99d2f0

[pre-commit.ci] auto fixes from pre-commit.com hooks

5c03731

for more information, see https://pre-commit.ci

ilevkivskyi approved these changes Sep 14, 2023

View reviewed changes

hauntsaninja merged commit 2c2d126 into python:master Sep 14, 2023
17 checks passed

hauntsaninja deleted the tuple-any-subtyping branch September 14, 2023 21:27

hauntsaninja mentioned this pull request Sep 15, 2023

Fix inference for overloaded __call__ with generic self #16053

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix tuple[Any, ...] subtyping #16108

Fix tuple[Any, ...] subtyping #16108

hauntsaninja commented Sep 14, 2023

This comment has been minimized.

hauntsaninja commented Sep 14, 2023 •

edited

Loading

This comment has been minimized.

ilevkivskyi left a comment

ilevkivskyi Sep 14, 2023

ilevkivskyi Sep 14, 2023

ilevkivskyi Sep 14, 2023

ilevkivskyi Sep 14, 2023

This comment has been minimized.

hauntsaninja commented Sep 14, 2023

ilevkivskyi Sep 14, 2023

hauntsaninja Sep 14, 2023 •

edited

Loading

ilevkivskyi Sep 14, 2023

github-actions bot commented Sep 14, 2023

hauntsaninja commented Sep 14, 2023

Fix tuple[Any, ...] subtyping #16108

Fix tuple[Any, ...] subtyping #16108

Conversation

hauntsaninja commented Sep 14, 2023

This comment has been minimized.

hauntsaninja commented Sep 14, 2023 • edited Loading

This comment has been minimized.

ilevkivskyi left a comment

Choose a reason for hiding this comment

ilevkivskyi Sep 14, 2023

Choose a reason for hiding this comment

ilevkivskyi Sep 14, 2023

Choose a reason for hiding this comment

ilevkivskyi Sep 14, 2023

Choose a reason for hiding this comment

ilevkivskyi Sep 14, 2023

Choose a reason for hiding this comment

This comment has been minimized.

hauntsaninja commented Sep 14, 2023

ilevkivskyi Sep 14, 2023

Choose a reason for hiding this comment

hauntsaninja Sep 14, 2023 • edited Loading

Choose a reason for hiding this comment

ilevkivskyi Sep 14, 2023

Choose a reason for hiding this comment

github-actions bot commented Sep 14, 2023

hauntsaninja commented Sep 14, 2023

hauntsaninja commented Sep 14, 2023 •

edited

Loading

hauntsaninja Sep 14, 2023 •

edited

Loading