Allow fancy self-types #7860

ilevkivskyi · 2019-11-02T22:30:00Z

Fixes #3625
Fixes #5305
Fixes #5320
Fixes #5868
Fixes #7191
Fixes #7778
Fixes python/typing#680

So, lately I was noticing many issues that would be fixed by (partially) moving the check for self-type from definition site to call site.

This morning I found that we actually have such function check_self_arg() that is applied at call site, but it is almost not used. After more reading of the code I found that all the patterns for self-types that I wanted to support should either already work, or work with minimal modifications. Finally, I discovered that the root cause of many of the problems is the fact that bind_self() uses wrong direction for type inference! All these years it expected actual argument type to be supertype of the formal one.

After fixing this bug, it turned out it was easy to support following patterns for explicit self-types:

Structured match on generic self-types
Restricted methods in generic classes (methods that one is allowed to call only for some values or type arguments)
Methods overloaded on self-type
(Important case of the above) overloaded __init__ for generic classes
Mixin classes (using protocols)
Private class-level decorators (a bit hacky)
Precise types for alternative constructors (mostly already worked)

This PR cuts few corners, but it is ready for review (I left some TODOs). Note I also add some docs, I am not sure this is really needed, but probably good to have.

ilevkivskyi · 2019-11-02T23:07:15Z

Few extra comments:

I just updated the PR description with a list of issues this will close, I didn't do a thorough search however.
Potentially this PR can be split in few parts, if needed (I would prefer to keep it single however).
I am going to push couple more tests that mimic tricky examples in one issue (just to be sure they are fixed). If you think it is too much I can remove them.

ilevkivskyi · 2019-11-02T23:50:12Z

(Also note that we can count #3625 as fixed only if remove all checks at the definition site).

Michael0x2a

Maybe you'll want to get a second opinion from Jukka/Sully, but FWIW these changes all look fine to me: the overall approach seems sound and nicely minimal, and my comments were mostly nits and smaller suggestions.

docs/source/more_types.rst

mypy/checker.py

mypy/typeops.py

test-data/unit/check-selftype.test

Co-Authored-By: Michael Lee <michael.lee.0x2a@gmail.com>

…kyi/mypy into fix-bind-self-direction

ilevkivskyi · 2019-11-03T01:33:11Z

(Also note that we can count #3625 as fixed only if remove all checks at the definition site).

I changed my mind about this. With a tiny extra fix this issue will be now also closed by this PR.

msullivan

This is a really exciting improvement. I love when so much falls out of one fix.

msullivan · 2019-11-04T19:28:31Z

docs/source/more_types.rst

@@ -561,6 +561,140 @@ with ``Union[int, slice]`` and ``Union[T, Sequence]``.
   to returning ``Any`` only if the input arguments also contain ``Any``.


+.. _advanced_self:
+
+Advanced uses of self-types


Is it worth documenting the use in __init__?

I noticed it is mostly useful for typeshed stubs, but didn't find this patter in user code, I will add a short sentence at the end, where I discuss overloads.

msullivan · 2019-11-04T19:43:36Z

mypy/typeops.py

-    # (info). For example
+    # We first need to record all non-trivial (explicit) self types in __init__,
+    # since they will not be available after we bind them. Note, we use explicit
+    # self-types only in the defining class, similar to __new__ (see below).


__new__ can use explicit returns in subclasses, but kind of as a technicality, since it only works using bound type vars, I think.

Yes, I will make the comment more clear.

msullivan · 2019-11-04T19:53:45Z

mypy/typeops.py

+    """
+    if isinstance(typ, TypeType):
+        return supported_self_type(typ.item)
+    return (isinstance(typ, TypeVarType) or


This doesn't seem to check that there is an instance upper bound? Should there be?

Also my inclination would be to throw parens around the and clause.

This doesn't seem to check that there is an instance upper bound? Should there be?

I would say it is safe to keep it as is. We can restrict this later if we will find problems.

Also my inclination would be to throw parens around the and clause.

OK.

ilevkivskyi added 7 commits November 2, 2019 16:42

Start work

07de26b

Some polish

56c5b34

Add some overloads

5870ce9

Some fixes

082f4f4

Add some mixin fun

869e15e

Fix self-check

a6060da

Add one more TODO

6dc8126

ilevkivskyi requested review from msullivan, JukkaL and Michael0x2a November 2, 2019 22:30

Add two more tricky tests

48bc5e3

Michael0x2a approved these changes Nov 3, 2019

View reviewed changes

ilevkivskyi and others added 8 commits November 3, 2019 00:36

Fix a corner case for Type[...] vs metaclass; add tests

f287273

Update tests

e696834

Update docs/source/more_types.rst

72b9c8a

Co-Authored-By: Michael Lee <michael.lee.0x2a@gmail.com>

Update docs/source/more_types.rst

c913a7f

Co-Authored-By: Michael Lee <michael.lee.0x2a@gmail.com>

Update docs/source/more_types.rst

3cb098c

Co-Authored-By: Michael Lee <michael.lee.0x2a@gmail.com>

Update mypy/checker.py

52158e7

Co-Authored-By: Michael Lee <michael.lee.0x2a@gmail.com>

Merge branch 'fix-bind-self-direction' of https://github.com/ilevkivs…

247ce64

…kyi/mypy into fix-bind-self-direction

Address CR

0127c00

ilevkivskyi added 2 commits November 3, 2019 01:40

Prettify one import

56f42bd

Fix typo

ea6376d

This was referenced Nov 3, 2019

False negative for restricted self in subclass #7861

Open

decorator as class member raises "self-argument missing" #7778

Closed

ilevkivskyi added 3 commits November 3, 2019 09:42

Add a comment

eb3b2b7

Simplify one test; use better example in docs

9a3454e

More doc tweaks

ab304e5

ilevkivskyi added 3 commits November 3, 2019 14:03

Some docstring, comments and cleanups

64c6727

Two more tests for overloaded __init__

5e5b133

Fix self-check

ed58024

msullivan approved these changes Nov 4, 2019

View reviewed changes

Address CR

b33059e

ilevkivskyi merged commit 673468c into python:master Nov 5, 2019

ilevkivskyi deleted the fix-bind-self-direction branch November 5, 2019 13:53

ethanhs mentioned this pull request Nov 5, 2019

Implement mypy annotations (Resolves #549) ocf/ocfweb#557

Merged

This was referenced Nov 11, 2019

Regression involving generic class methods #7925

Closed

Regression involving overloaded classmethods #7926

Closed

suutari mentioned this pull request Nov 25, 2019

docs/more_types: Fix typo in advanced self-types #8006

Merged

sbdchd mentioned this pull request Nov 29, 2020

Typevar can't be inferred when overloading __init__ with annotated self param microsoft/pyright#1211

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow fancy self-types #7860

Allow fancy self-types #7860

ilevkivskyi commented Nov 2, 2019 •

edited

Loading

ilevkivskyi commented Nov 2, 2019 •

edited

Loading

ilevkivskyi commented Nov 2, 2019

Michael0x2a left a comment

ilevkivskyi commented Nov 3, 2019

msullivan left a comment

msullivan Nov 4, 2019

ilevkivskyi Nov 5, 2019

msullivan Nov 4, 2019

ilevkivskyi Nov 5, 2019

msullivan Nov 4, 2019

ilevkivskyi Nov 5, 2019

Allow fancy self-types #7860

Allow fancy self-types #7860

Conversation

ilevkivskyi commented Nov 2, 2019 • edited Loading

ilevkivskyi commented Nov 2, 2019 • edited Loading

ilevkivskyi commented Nov 2, 2019

Michael0x2a left a comment

Choose a reason for hiding this comment

ilevkivskyi commented Nov 3, 2019

msullivan left a comment

Choose a reason for hiding this comment

msullivan Nov 4, 2019

Choose a reason for hiding this comment

ilevkivskyi Nov 5, 2019

Choose a reason for hiding this comment

msullivan Nov 4, 2019

Choose a reason for hiding this comment

ilevkivskyi Nov 5, 2019

Choose a reason for hiding this comment

msullivan Nov 4, 2019

Choose a reason for hiding this comment

ilevkivskyi Nov 5, 2019

Choose a reason for hiding this comment

ilevkivskyi commented Nov 2, 2019 •

edited

Loading

ilevkivskyi commented Nov 2, 2019 •

edited

Loading