How to programatically run mypy in python?

[TheDan64]

Hi, I asked this question on the gitter chat originally, but I was redirected here.

A lot of the classes in my library have metaclasses that have custom __isinstance__ and/or __issubclass__ methods so that they work in a certain way and I want to make sure that they will work with mypy. Mypy compatibility is really important to my library and I don't want to just assume they'll work because they pass my own isinstance checks and whatnot.

So my question is, is there is a way to run mypy in my unit tests so that I can assert that they work as expected? For example, I'm imagining something like:

from mypy import validator

def test_my_class():
    def foo() -> MyClass:
        # do some stuff
        return myclassinstance

    assert validator.run(foo, params={})

[gvanrossum]

There's some prior discussion in #2097 and a languishing pull request in #2114.

But the API proposed there is still meant to run on a file or set of files. (And if you had a set of files, I'd recommend just running the mypy command line as a subprocess.)

We're reluctant to make mypy importable and callable from other code, because mypy's primary usage is through the command line, targeted at whole programs, and we often make big changes to the internals that would break external invocations. Think of mypy as a linter -- would you want to invoke flake8 on a function object? I don't think so?

Perhaps what you're really looking for is runtime checking of annotations? For that there's some discussion in a different tracker: python/typing#310 -- maybe one of the libraries mentioned there is more what you're looking for?

If I am still misunderstanding you please give a more elaborate example, e.g. a function with an error and what kind of diagnostic you were expecting.

[TheDan64]

Good idea, I suppose I can write files and run mypy on those as a sub-processes in the tests. It's a little lame that I can't write them directly in the tests, but it makes sense that mypy was designed that way.

I guess I've mistakenly made the assumption that mypy works through isinstance checks since all these pass:

from typing import Tuple, List

assert isinstance((1, "str"), Tuple[int, str])
assert isinstance([1, 2], List[int])

But it seems like that's not the case. In fact, isinstance(["str"], List[int]) surprisingly passes. Could you point me in the right direction as to how this process actually works? It looks like def foo(): -> List[int]: return ["str"] rightly fails in mypy and I'd like to replicate this behavior with my own classes which do have their own type arguments.

[gvanrossum]

You seem to be confused between runtime and check time. mypy is an
offline checker and does not call isinstance() -- it analyzes your source
code. It happens to also understand isinstance(). But at runtime things
work different because of something called "type erasure".

[TheDan64]

Right - I stated that I recognize my mistake.

But mypy still knows that there's a difference between (1,) and ('foo',) when compared to Tuple[int]. I'm wondering how that works since type tuple and Tuple are different classes and serve different functions. Is there a way to take advantage of that with my own code or does mypy just know how to work with python's typing classes only (and other primitives)?

[gvanrossum]

Mypy doesn't just look at the outer type; it sees (1,) as Tuple[int] and ('',) as Tuple[str]. It interprets tuple as Tuple[Any].

[TheDan64]

So suppose I have my own classes Foo and Bar. Suppose Foo could take type arguments similarly to python's typing classes and Bar takes a value on instantiation.

Is there a way for me to make

def func() -> Foo[int]:
    return Bar(1)

validate as success in mypy?

[gvanrossum]

In our terminology, Foo is a generic class: http://mypy.readthedocs.io/en/latest/generics.html For more help please show the definitions of Foo and Bar. Please provide a complete (but small) working program -- I don't want to guess what the relationship between Foo and Bar in your examples is.

[TheDan64]

Here's a lightweight example of something I've done that I'd like to be valid in mypy (with additional configuration as needed of course): https://gist.github.com/TheDan64/69f7f9ac220e15f6b2bfdf36f93fdeda

[gvanrossum]

Given the body of either_or() in that gist, its return type is Union[Left, Right]. You can't define your own union-like classes.

Also, your Left class needs this added to the constuctor signature:

 def __init__(self, value: int) -> None:

and similar for the Right class (with str).

[gvanrossum]

And instead of if unknown.is_right(): you have to use if isinstance(unknown, Right):.

[TheDan64]

Shouldn't the constructor's value type be generic? In another function it might be a different type altogether

[gvanrossum]

Sure, you can define a generic Left e.g. like this:

T = TypeVar('T')
class Left(Generic[T]):
    def __init__(self, value: T) -> None:
        self.value = value
    # (The rest unchanged)

and similar for Right. And then you can have e.g. Union[Left[int], Right[str]].

But there's no way to have your own Either that lets you abbreviate that union type as Either[int, str].

Is this something you've seen in another language?

[TheDan64]

That sounds good, thanks. Would something like this work?

class Either:
    def __getitem__(self, types: (L, R)) -> Union[Left[L], Right[R]]:
        left_type, right_type = types
        return Union[Left[left_type], Right[right_type]]

Yeah, I'm trying to make something similar to Haskell's Either & Maybe and Rust's Result & Option (which are influenced by the former) types. I really like the control flow they provide and having them be typechecked seemed like a great combination. In a more generic form, they're similar to tagged unions in that Left and Right are just variants associated with certain type(s).

[gvanrossum]

No, that won't work. You can't dynamically create types.

I am sorry, but this topic has veered quite far from the original question, and I am going to close it and unsubscribe.

[TheDan64]

Understood. Thanks for your help!

[jeking3]

Wouldn't it make sense to allow a @dataclass to call mypy from __post_init__ to prove that all the data types in the dataclass conform to the type specifications?

[JukkaL]

@jeking3 If you are asking about calling mypy from type checked code (that defines a dataclass), then this will not be supported. You are not supposed to call mypy from code being type checked -- that's just not how mypy has been designed to be used.

[lion3ls]

For the sake of info:

The PR making it possible: #2439
The Documentation: https://github.com/python/mypy/blob/master/docs/source/extending_mypy.rst

Hope this will help someone else coming on this issue and feeling sad as I was 😉.