Skip to content

Self Type #2193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 56 commits into from
Oct 27, 2016
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
04934c3
move things around
elazarg Sep 28, 2016
4eb5f75
first test pass. no override yet
elazarg Sep 29, 2016
181ff09
selftypeClass test pass
elazarg Sep 29, 2016
25e104a
Simple override
elazarg Sep 29, 2016
5e72623
Simple override
elazarg Sep 29, 2016
29d2e0f
test super() and rename self_type()
elazarg Sep 29, 2016
4d5d479
more tests
elazarg Sep 29, 2016
ea6c485
unneeded changes
elazarg Sep 29, 2016
00596a6
merge
elazarg Sep 29, 2016
39615c2
some more tests
elazarg Sep 29, 2016
f8875f9
recursive instantiation
elazarg Sep 29, 2016
9ae3bf9
add implicit bound
elazarg Sep 30, 2016
b29bf87
add tests: prohibit overriding without selftype
elazarg Sep 30, 2016
f097967
Merge
elazarg Sep 30, 2016
747c5b2
Merge
elazarg Sep 30, 2016
9b68a2f
minor
elazarg Sep 30, 2016
65d6428
Merge
elazarg Sep 30, 2016
6c161b9
fix comment
elazarg Sep 30, 2016
1cae034
rename: self->cls
elazarg Oct 1, 2016
733edc1
Fix tests, separate binding from the classes
elazarg Oct 4, 2016
91946e2
Merge upstream into self_type
elazarg Oct 4, 2016
abcb094
add Guido's test
elazarg Oct 5, 2016
9f831c4
move tests to dedicated file
elazarg Oct 7, 2016
aade1ed
move tests to dedicated file
elazarg Oct 7, 2016
a6a0a3b
Merge upstream
elazarg Oct 7, 2016
8e81051
pass report_type to instantiate
elazarg Oct 7, 2016
a9b0b68
send report_type to classmethod
elazarg Oct 7, 2016
ec0f33a
add missing file
elazarg Oct 7, 2016
09cd7bc
more report type
elazarg Oct 7, 2016
8b242e2
support super()
elazarg Oct 8, 2016
a8be2a6
avoid partial types when testing static access
elazarg Oct 9, 2016
3055ab0
remove comment
elazarg Oct 9, 2016
72af252
Merge remote-tracking branch 'upstream/master' into self_type
elazarg Oct 11, 2016
bb2ac78
use existing machinery; still partial types
elazarg Oct 14, 2016
5f26cac
do not trigger the .erased flag
elazarg Oct 14, 2016
1f44c73
do not trigger the .erased flag
elazarg Oct 14, 2016
fcbcf4b
Fix issue with t.variables. adapt SelfTypeBound
elazarg Oct 14, 2016
860c96a
minor
elazarg Oct 15, 2016
b078d61
some override checking: instantiate with fillvars(self)
elazarg Oct 15, 2016
fc29a3a
Merge remote-tracking branch 'upstream/master' into self_type
elazarg Oct 18, 2016
2299a00
forgotten file
elazarg Oct 18, 2016
137f452
initialize mutable in __init__
elazarg Oct 19, 2016
4bcbf59
Rename report_type, Add docstrings etc.
elazarg Oct 20, 2016
6155c7f
minor doc fix
elazarg Oct 20, 2016
47e5e2f
remove binder hack
elazarg Oct 20, 2016
00ae83d
Merge remote-tracking branch 'upstream/master' into self_type
elazarg Oct 26, 2016
30f847e
example for actual_type
elazarg Oct 26, 2016
c152c20
avoid crush in super(); document
elazarg Oct 26, 2016
4f2017e
lint
elazarg Oct 26, 2016
fddbba0
comment for fill_typevars
elazarg Oct 26, 2016
3cf8ecf
comment for fill_typevars
elazarg Oct 26, 2016
707da4e
rename actual_self, use declared_type in analyze_super
elazarg Oct 26, 2016
48e139d
test for erased types
elazarg Oct 26, 2016
25b84f8
lint
elazarg Oct 26, 2016
8944ff1
Revert accidental deletion from doc
elazarg Oct 27, 2016
9221001
add warning for unsafe testcase
elazarg Oct 27, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
rename actual_self, use declared_type in analyze_super
  • Loading branch information
elazarg committed Oct 26, 2016
commit 707da4e4d46998263d2c456e38e3885a0aaa913a
18 changes: 6 additions & 12 deletions mypy/checkexpr.py
Original file line number Diff line number Diff line change
Expand Up @@ -1608,24 +1608,18 @@ def analyze_super(self, e: SuperExpr, is_lvalue: bool) -> Type:
return AnyType()
if not self.chk.in_checked_function():
return AnyType()
# If the class A is not generic, and we declare `self: T` in the signature:
# * declared_self will be T
# * fill_typevars(e.info) will be A
# If the class has a type arguments Q, and we declare `self: A[T]`:
# * declared_self will be A[T]
# * fill_typevars(e.info) will be A[Q]
# If the we declare `self: T` in a generic class, declared_self is still T.
# TODO: selftype does not support generic classes yet
filled_self = fill_typevars(e.info)
args = self.chk.function_stack[-1].arguments
# An empty args with super() is an error; we need something in declared_self
declared_self = args[0].variable.type if args else filled_self
return analyze_member_access(name=e.name, typ=filled_self, node=e,
if not args:
self.chk.fail('super() requires at least on positional argument', e)
return AnyType()
declared_self = args[0].variable.type
return analyze_member_access(name=e.name, typ=declared_self, node=e,
is_lvalue=False, is_super=True, is_operator=False,
builtin_type=self.named_type,
not_ready_callback=self.not_ready_callback,
msg=self.msg, override_info=base, chk=self.chk,
actual_self=declared_self)
original_type=declared_self)
else:
# Invalid super. This has been reported by the semantic analyzer.
return AnyType()
Expand Down
93 changes: 41 additions & 52 deletions mypy/checkmember.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,28 +33,17 @@ def analyze_member_access(name: str,
not_ready_callback: Callable[[str, Context], None],
msg: MessageBuilder,
override_info: TypeInfo = None,
actual_self: Type = None,
original_type: Type = None,
chk: 'mypy.checker.TypeChecker' = None) -> Type:
"""Analyse attribute access.
"""Return the type of attribute `name` of typ.

This is a general operation that supports various different variations:

1. lvalue or non-lvalue access (i.e. setter or getter access)
2. supertype access (when using super(); is_super == True and
override_info should refer to the supertype)

actual_self is the type of E in the expression E.foo - the most precise
information available for mypy at the point of accessing E.foo
For example,

class D(str): pass
a = D()
reveal_type(a.replace)

during checking `a.replace`, typ will be str, whereas actual_self will be D.
This helps for error reporting and for implementing methods where self is generic.
original_type is the most precise inferred or declared type of the base object
that we have available. typ is generally a supertype of original_type.
When looking for an attribute of typ, we may perform recursive calls targeting
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please retain This is general ... and the following numbered list as they are still relevant.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. Sorry.

the fallback type, for example.
original_type is always the type used in the initial call.
"""
actual_self = actual_self or typ
original_type = original_type or typ
if isinstance(typ, Instance):
if name == '__init__' and not is_super:
# Accessing __init__ in statically typed code would compromise
Expand Down Expand Up @@ -88,15 +77,15 @@ class D(str): pass
# the first argument.
pass
else:
signature = bind_self(signature, actual_self)
signature = bind_self(signature, original_type)
typ = map_instance_to_supertype(typ, method.info)
return expand_type_by_instance(signature, typ)
else:
# Not a method.
return analyze_member_var_access(name, typ, info, node,
is_lvalue, is_super, builtin_type,
not_ready_callback, msg,
actual_self=actual_self, chk=chk)
original_type=original_type, chk=chk)
elif isinstance(typ, AnyType):
# The base object has dynamic type.
return AnyType()
Expand All @@ -106,7 +95,7 @@ class D(str): pass
# The only attribute NoneType has are those it inherits from object
return analyze_member_access(name, builtin_type('builtins.object'), node, is_lvalue,
is_super, is_operator, builtin_type, not_ready_callback, msg,
actual_self=actual_self, chk=chk)
original_type=original_type, chk=chk)
elif isinstance(typ, UnionType):
# The base object has dynamic type.
msg.disable_type_names += 1
Expand Down Expand Up @@ -143,24 +132,24 @@ class D(str): pass
# See https://github.com/python/mypy/pull/1787 for more info.
result = analyze_class_attribute_access(ret_type, name, node, is_lvalue,
builtin_type, not_ready_callback, msg,
actual_self=actual_self)
original_type=original_type)
if result:
return result
# Look up from the 'type' type.
return analyze_member_access(name, typ.fallback, node, is_lvalue, is_super,
is_operator, builtin_type, not_ready_callback, msg,
actual_self=actual_self, chk=chk)
original_type=original_type, chk=chk)
else:
assert False, 'Unexpected type {}'.format(repr(ret_type))
elif isinstance(typ, FunctionLike):
# Look up from the 'function' type.
return analyze_member_access(name, typ.fallback, node, is_lvalue, is_super,
is_operator, builtin_type, not_ready_callback, msg,
actual_self=actual_self, chk=chk)
original_type=original_type, chk=chk)
elif isinstance(typ, TypeVarType):
return analyze_member_access(name, typ.upper_bound, node, is_lvalue, is_super,
is_operator, builtin_type, not_ready_callback, msg,
actual_self=actual_self, chk=chk)
original_type=original_type, chk=chk)
elif isinstance(typ, DeletedType):
msg.deleted_as_rvalue(typ, node)
return AnyType()
Expand All @@ -176,33 +165,33 @@ class D(str): pass
# See comment above for why operators are skipped
result = analyze_class_attribute_access(item, name, node, is_lvalue,
builtin_type, not_ready_callback, msg,
actual_self=actual_self)
original_type=original_type)
if result:
return result
fallback = builtin_type('builtins.type')
return analyze_member_access(name, fallback, node, is_lvalue, is_super,
is_operator, builtin_type, not_ready_callback, msg,
actual_self=actual_self, chk=chk)
original_type=original_type, chk=chk)

if chk and chk.should_suppress_optional_error([typ]):
return AnyType()
return msg.has_no_attr(actual_self, name, node)
return msg.has_no_attr(original_type, name, node)


def analyze_member_var_access(name: str, itype: Instance, info: TypeInfo,
node: Context, is_lvalue: bool, is_super: bool,
builtin_type: Callable[[str], Instance],
not_ready_callback: Callable[[str, Context], None],
msg: MessageBuilder,
actual_self: Type = None,
original_type: Type = None,
chk: 'mypy.checker.TypeChecker' = None) -> Type:
"""Analyse attribute access that does not target a method.

This is logically part of analyze_member_access and the arguments are similar.

actual_self is the type of E in the expression E.var
original_type is the type of E in the expression E.var
"""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Describe report_type. It looks like there's a new use for it -- the name doesn't look very descriptive any more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I have commented about it above. Should I rename it to actual_self? Or perhaps report_type and actual_self just happen to coincide and they are actually different things.

actual_self = actual_self or itype
original_type = original_type or itype
# It was not a method. Try looking up a variable.
v = lookup_member_var_or_accessor(info, name, is_lvalue)

Expand All @@ -219,7 +208,7 @@ def analyze_member_var_access(name: str, itype: Instance, info: TypeInfo,
method = info.get_method('__getattr__')
if method:
function = function_type(method, builtin_type('builtins.function'))
bound_method = bind_self(function, actual_self)
bound_method = bind_self(function, original_type)
typ = map_instance_to_supertype(itype, method.info)
getattr_type = expand_type_by_instance(bound_method, typ)
if isinstance(getattr_type, CallableType):
Expand All @@ -235,20 +224,20 @@ def analyze_member_var_access(name: str, itype: Instance, info: TypeInfo,
else:
if chk and chk.should_suppress_optional_error([itype]):
return AnyType()
return msg.has_no_attr(actual_self, name, node)
return msg.has_no_attr(original_type, name, node)


def analyze_var(name: str, var: Var, itype: Instance, info: TypeInfo, node: Context,
is_lvalue: bool, msg: MessageBuilder,
not_ready_callback: Callable[[str, Context], None],
actual_self: Type = None) -> Type:
original_type: Type = None) -> Type:
"""Analyze access to an attribute via a Var node.

This is conceptually part of analyze_member_access and the arguments are similar.

actual_self is the type of E in the expression E.var
original_type is the type of E in the expression E.var
"""
actual_self = actual_self or itype
original_type = original_type or itype
# Found a member variable.
itype = map_instance_to_supertype(itype, var.info)
typ = var.type
Expand All @@ -273,7 +262,7 @@ def analyze_var(name: str, var: Var, itype: Instance, info: TypeInfo, node: Cont
# class.
functype = t
check_method_type(functype, itype, var.is_classmethod, node, msg)
signature = bind_self(functype, actual_self)
signature = bind_self(functype, original_type)
if var.is_property:
# A property cannot have an overloaded type => the cast
# is fine.
Expand Down Expand Up @@ -349,8 +338,8 @@ def analyze_class_attribute_access(itype: Instance,
builtin_type: Callable[[str], Instance],
not_ready_callback: Callable[[str, Context], None],
msg: MessageBuilder,
actual_self: Type = None) -> Type:
"""actual_self is the type of E in the expression E.var"""
original_type: Type = None) -> Type:
"""original_type is the type of E in the expression E.var"""
node = itype.type.get(name)
if not node:
if itype.type.fallback_to_any:
Expand All @@ -373,7 +362,7 @@ def analyze_class_attribute_access(itype: Instance,
if isinstance(t, PartialType):
return handle_partial_attribute_type(t, is_lvalue, msg, node.node)
is_classmethod = is_decorated and cast(Decorator, node.node).func.is_class
return add_class_tvars(t, itype, is_classmethod, builtin_type, actual_self)
return add_class_tvars(t, itype, is_classmethod, builtin_type, original_type)
elif isinstance(node.node, Var):
not_ready_callback(name, context)
return AnyType()
Expand All @@ -394,7 +383,7 @@ def analyze_class_attribute_access(itype: Instance,

def add_class_tvars(t: Type, itype: Instance, is_classmethod: bool,
builtin_type: Callable[[str], Instance],
actual_self: Type = None) -> Type:
original_type: Type = None) -> Type:
"""Instantiate type variables during analyze_class_attribute_access,
e.g T and Q in the following:

Expand All @@ -406,7 +395,7 @@ class B(A): pass

B.foo()

actual_self is the value of the type B in the expression B.foo()
original_type is the value of the type B in the expression B.foo()
"""
# TODO: verify consistency betweem Q and T
info = itype.type # type: TypeInfo
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add docstring. In particular, describe report_type.

Expand All @@ -415,11 +404,11 @@ class B(A): pass
vars = [TypeVarDef(n, i + 1, None, builtin_type('builtins.object'), tv.variance)
for (i, n), tv in zip(enumerate(info.type_vars), info.defn.type_vars)]
if is_classmethod:
t = bind_self(t, actual_self if isinstance(actual_self, TypeType) else TypeType(itype))
t = bind_self(t, original_type if isinstance(original_type, TypeType) else TypeType(itype))
return t.copy_modified(variables=vars + t.variables)
elif isinstance(t, Overloaded):
return Overloaded([cast(CallableType, add_class_tvars(i, itype, is_classmethod,
builtin_type, actual_self))
builtin_type, original_type))
for i in t.items()])
return t

Expand Down Expand Up @@ -541,15 +530,15 @@ def map_type_from_supertype(typ: Type, sub_info: TypeInfo,
F = TypeVar('F', bound=FunctionLike)


def bind_self(method: F, actual_self: Type = None) -> F:
def bind_self(method: F, original_type: Type = None) -> F:
"""Return a copy of `method`, with the type of its first parameter (usually
self or cls) bound to actual_self.
self or cls) bound to original_type.

If the type of `self` is a generic type (T, or Type[T] for classmethods),
instantiate every occurrence of type with actual_self in the rest of the
instantiate every occurrence of type with original_type in the rest of the
signature and in the return type.

actual_self is the type of E in the expression E.copy(). It is None in
original_type is the type of E in the expression E.copy(). It is None in
compatibility checks. In this case we treat it as the erasure of the
declared type of self.

Expand Down Expand Up @@ -582,13 +571,13 @@ class B(A): pass
if func.variables and (isinstance(self_param_type, TypeVarType) or
(isinstance(self_param_type, TypeType) and
isinstance(self_param_type.item, TypeVarType))):
if actual_self is None:
if original_type is None:
# Type check method override
# XXX value restriction as union?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comment about when this code pass will be hit (actual_self is None).

actual_self = erase_to_bound(self_param_type)
original_type = erase_to_bound(self_param_type)

typearg = infer_type_arguments([x.id for x in func.variables],
self_param_type, actual_self)[0]
self_param_type, original_type)[0]

def expand(target: Type) -> Type:
return expand_type(target, {func.variables[0].id: typearg}, False)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me that the erased flag should be set here. Here 'erased' means that the type is derived from substituting a type variable, and this is the case here. (This information can be quite for useful for some other tools that want to use the type information generated by mypy, but we don't use it for anything significant within mypy. The term 'erased' comes from a runtime type checker use case where is makes sense.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This erasure information (which feels like a misnomer) appears in the tests and IIRC only in actual type checking. So should I test for having A* as the result of calling a function?

BTW, what tools use it, and how?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently none that I know of, but I'm prototyping a tool that will need it for runtime type checking. They way it's used is pretty complex so you have to take my word for it as the margin isn't wide enough for a good explanation :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. But how should I test it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to test it for now -- there are some tests elsewhere for other cases where the erased flag is important, I think. expand_type should mostly do the right thing. I'll likely have to fix some edge cases if/when I actually start using the information, but that's okay. It's sufficient that we just avoid doing stuff that clearly seems to break some assumptions.

Expand Down