Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support recursive types #731

Closed
o11c opened this issue Jul 30, 2015 · 83 comments
Closed

Support recursive types #731

o11c opened this issue Jul 30, 2015 · 83 comments

Comments

@o11c
Copy link
Contributor

o11c commented Jul 30, 2015

The following in particular would be useful:

Callback = Callable[[str], 'Callback']
Foo = Union[str, List['Foo']]
@JukkaL
Copy link
Collaborator

JukkaL commented Aug 4, 2015

If (or when) mypy will have structural subtyping, recursive types would also be useful there.

@JukkaL
Copy link
Collaborator

JukkaL commented Oct 13, 2015

My current plan is to postpone recursive types until simple structural subtyping is in and reconsider them later. After thinking about them more they are going to increase complexity a lot of I haven't seen much evidence for them being needed that often.

@oconnor663
Copy link

If I'm understanding this issue right, I'm running into it for classes that want a fluent interface. So for example, if I want callers to do something like this:

myfoo.add(bar).add(baz).finish()

Then the definition of the Foo class and the add method need to look something like this:

class Foo:
    def add(self, x) -> Foo:  # Python chokes on this line!
        # do stuff
        return self

Another place where Python commonly does return self is in the __enter__ method for context managers. Is mypy able to typecheck those right now?

@refi64
Copy link
Contributor

refi64 commented Oct 16, 2015

@oconnor663 Try:

class Foo:
    def add(self, x) -> 'Foo':
        # do stuff
        return self

@oconnor663
Copy link

Ah, thank you.

@oconnor663
Copy link

@kirbyfan64, do you know if there are standard functions anywhere that understand this convention? Like, if I wanted to introspect a couple functions and compare the types of their arguments, should I handle the Foo == "Foo" case explicitly? That seems doable, but a string-aware version of say isinstance seems harder.

@refi64
Copy link
Contributor

refi64 commented Oct 16, 2015

@oconnor663 I don't think there's anything like that. If you're introspecting the functions via a decorator, you could try accessing the caller's globals and locals.

@gvanrossum
Copy link
Member

You're aware of typing.get_type_hints(obj)right? It is similar to obj.annotations` but expands forward references.
https://docs.python.org/3/library/typing.html?highlight=typing#typing.get_type_hints

There used to be an instance() implementation in typing.py but Mark Shannon
made me take it out. It's being deleted in this rev:
python/typing@ac7494f

On Fri, Oct 16, 2015 at 9:37 AM, Ryan Gonzalez notifications@github.com
wrote:

@oconnor663 https://github.com/oconnor663 I don't think there's
anything like that. If you're introspecting the functions via a decorator,
you could try accessing the caller's globals and locals.


Reply to this email directly or view it on GitHub
#731 (comment).

--Guido van Rossum (python.org/~guido)

@oconnor663
Copy link

@gvanrossum that's exactly what I was looking for, thanks. Sorry for the n00b questions today, but awesome that all this is supported.

@JukkaL
Copy link
Collaborator

JukkaL commented Oct 17, 2015

Mypy should detect missing string literal escapes (see #948).

@dmoisset
Copy link
Contributor

Going back to the original point on the issue, I found a case in the stdlib where this would be needed; the type for isinstance() is currently:

def isinstance(o: object, t: Union[type, Tuple[type, ...]]) -> bool: ...

but it should actually be:

ClassInfo = Union[type, Tuple['ClassInfo', ...]]
def isinstance(o: object, t: ClassInfo) -> bool: ...

Because according to https://docs.python.org/3/library/functions.html#isinstance the tuples can be nested. I found an actual example of this while typechecking django.http.response.HttpResponse.content

@gvanrossum gvanrossum added this to the Future milestone Aug 18, 2016
cortesi added a commit to cortesi/mitmproxy that referenced this issue Mar 20, 2017
Mypy doesn't support recursive types yet, so we can't properly express
TSerializable nested structures. For now, we just disable type checking in the
appropriate locations.

python/mypy#731
@srittau
Copy link
Contributor

srittau commented Mar 29, 2017

I have come across this while trying to define a generic JSON type:

JSON = Union[Dict[str, "JSON"], List["JSON"], str, int, float, bool, None]

So consider this a +1 for supporting this use case.

@gvanrossum gvanrossum removed this from the Future milestone Mar 29, 2017
@graingert
Copy link
Contributor

graingert commented Jul 24, 2017

@srittau JSON needs to be Any because it is recursive and you can give json.loads a custom JSONEncoder class:

_PlainJSON = Union[Dict[str, "_PlainJSON"], List["_PlainJSON"], str, int, float, bool, None]
_T = TypeVar('_T')
JSON = Union[_PlainJSON, _T, Dict[str, "JSON"], List["JSON"]]
def loads(data: str, cls: Type[JSONEncoder[_T]]) -> JSON: ...

of course recursive types and Type[JSONEncoder[_T]] types arn't supported.

@DustinWehr
Copy link

DustinWehr commented Jan 18, 2018

The following pattern seems to be good enough for my purposes. The boilerplate is tolerable for me.

class BTree(NamedTuple):
    val: int
    left_ : Any
    right_ : Any

    # typed constructor
    @staticmethod
    def c(val: int, left: Optional['BTree'] = None, right: Optional['BTree'] = None) -> 'BTree':
        return BTree(val, left, right)

    # typed accessors
    @property
    def left(self) -> Optional['BTree']:
        return cast(Optional[BTree], self.left_)
    @property
    def right(self) -> Optional['BTree']:
        return cast(Optional[BTree], self.right_)

atree = BTree.c(1, BTree.c(2, BTree.c(3), BTree.c(4)), BTree.c(5))
atree2 = BTree.c(1, BTree.c(2, BTree.c(3), BTree.c(4)), BTree.c(5))
assert atree == atree2
assert isinstance(atree,BTree) and isinstance(atree.left,BTree) and isinstance(atree.left.left,BTree)

@DustinWehr
Copy link

DustinWehr commented Feb 14, 2018

Latest version of pattern. We use this example at Legalese for interacting with SMT solvers (the SMTLIB language).

I found that I ended up forgetting to use the typed .c static method instead of the untyped constructor in the BTree example above. This version addresses that. It's only very minor deficits are:

  1. Boilerplate (which I don't care about if the datatype is significant enough for us to care about the difference between an immutable recursive datatype and Tuple[Any,...])
  2. The constructor SMTExprNonatom and the type SMTExprNonatom_ do not share the same name. But this is only aesthetic; You won't use one when you mean the other, since with this naming convention, SMTExprNonatom will come up first in autocomplete, and only SMTExprNonatom_ can be used in a type position.
# Immutable recursive datatypes pattern
SMTAtom = Union[str, int, float, bool]
SMTExpr = Union['SMTExprNonatom_',SMTAtom]
class SMTExprNonatom_(NamedTuple):  
    symb: str
    args_: Tuple[Any,...] # see `args` below
    @staticmethod  # typed constructor, which we alias to `SMTExprNonatom` after the class def
    def c(symb:str, args:Iterable[SMTExpr]) -> 'SMTExprNonatom_': return SMTExprNonatom_(symb, tuple(args))
    @property # typed accessor
    def args(self) -> Tuple[SMTExpr]: return cast(Tuple[SMTExpr], self.args_)
SMTExprNonatom = SMTExprNonatom_.c
SMTCommand = NewType('SMTCommand', SMTExprNonatom_)

@JukkaL JukkaL changed the title Support recursive types. Support recursive types May 17, 2018
@cj81499
Copy link

cj81499 commented Sep 27, 2022

@ilevkivskyi is there a way to set this config option in a pyproject.toml file? I checked the documentation, but couldn't find anything about the new cli flag (perhaps because it's still experimental, and will eventually be enabled by default?)

@ilevkivskyi
Copy link
Member

Yes, it is indeed a temporary flag (for few releases), and in 0.990 it will be on by default, you can try

$ cat pyproject.toml
[tool.mypy]
enable_recursive_aliases = true

This worked for me on Linux.

@bartfeenstra
Copy link

It appears that v0.990/v0.991 may not have addressed the entirety of the challenge that is recursive/cyclic types: #14219

@jamesbraza
Copy link
Contributor

For those reading in posterity:

  1. mypy==0.981 added --enable-recursive-alias: Enable recursive type aliases behind a flag #13297
  2. mypy==0.990 started a deprecation cycle for --enable-recursive-alias: Temporarily put back --enable-recursive-aliases (as depreceated) #13852
  3. mypy==1.7.0 completed the deprecation cycle, removing --enable-recursive-alias: Delete recursive aliases flags #16346

So with mypy>=1.7, recursive type support is built into mypy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests