Add static typing to Hypothesis API

[DRMacIver]

The new typing module in 3.5 and PEP 0484 add static type annotations to Python. It would be nice for the Hypothesis API to support this, as the public API is actually pretty static in nature. stub files offer a compatibility mode, so this can be done transparently without dropping support for Python 2 (which is sadly there for the long haul).

Steps this would require:

1. Making Strategy a generic type with a single parameter, Strategy[T] that specifies the type of value generated. Note that Strategy is covariant in its type parameter.
2. Providing static types for all of the public API in hypothesis and hypothesis.strategies.
3. Running a mypy checker on Hypothesis + its test suite as part of the build

Things to note:

1. given and builds cannot be precisely typed with this type system, but they can still have types that are better than nothing by constraining their *args and **kwargs to have a strategy type (this is actually quite useful because it will catch two common gotchas when using this).
2. Settings is going to be quite hard to type correctly and is probably not worth bothering with.
3. I am not proposing to make Hypothesis internals typed. In particular templates for strategies will remain completely untyped. I've worked out what typing this accurately requires in the past and PEP 0484 can't begin to express it.

[DRMacIver]

Note: On investigation, I don't think running mypy as part of the Hypothesis build is currently feasible, and I'm not very interested in adding this without a static checking layer as part of the build. I will leave this open, but this should be considered a long-term objective rather than something viable right now.

[Zac-HD]

Another note: the comments above were written about Hypothesis v1.11, and Hypothesis 3.x is going to be much nicer to annotate. MyPy has also improved substantially.

That said, we'd still have to use stub (.pyi) files, so I'm not volunteering to add type hints before we drop Python 2 at EOL in 2020. My work on #643 also led to a terrible cynicism about the typing module, which is shockingly inconsistent between 3.5, 3.6, and even patch versions.

[Zac-HD]

It seems that this issue so far has been planning to type-annotate as much as possible. However, one of the main points of gradual typing is that you can add it to a project incrementally!

This post about Zulip has some good advice, which - summarised for Hypothesis - suggests:

• Writing a make task that runs the smallest possible set of checks, getting a clean run, adding it to .travis.yml. This is 'glorified linter' territory, and shouldn't require code changes beyond (maybe) annotating empty global variables.
• Adding the --check-untyped-defs option makes mypy try to infer variable types anywhere it can, without annotations. With luck, 'independently useful linter' territory.
• Then anyone can add annotations (in comments or stubs, for Py2 compatibility) to various parts of the codebase in future PRs. This is where type hints become really useful and mypy does substantially more than any other Python linters.

@DRMacIver, if you think steps one and two are useful in isolation I'd be happy to make an attempt, and from there it's mostly an incremental job.

The remaining question is basically whether to use comments or stub files (until we drop Python2 support, at which point we can use inline annotations). Comments are more local to the related code, but if we use stub files it would be possible to exclude them from the has_source_changes check and therefore avoid releases for hint-only PRs.

[Daenyth]

Stub files are probably preferable for now because mypy doesn't yet have an method for pip-installed packages to provide their own annotations to consumers - anything other than the local code itself needs to be in typeshed to provide annotations to consumers, and that needs stubs. Also hypothesis has a fairly stable interface so I think it wouldn't be quite so bad to have header files. Additionally it lets you use the nicer python3 syntax instead of the python2 comments

…

On Wed, Sep 13, 2017 at 3:49 AM Zac Hatfield Dodds ***@***.***> wrote: It seems that this issue so far has been planning to type-annotate as much as possible. However, one of the main points of gradual typing is that you can add it to a project incrementally! This post about Zulip <http://blog.zulip.org/2016/10/13/static-types-in-python-oh-mypy/> has some good advice, which - summarised for Hypothesis - suggests: - Writing a make task that runs the smallest possible set of checks, getting a clean run, adding it to .travis.yml. This is 'glorified linter' territory, and shouldn't require code changes beyond (maybe) annotating empty global variables. - Adding the --check-untyped-defs option makes mypy try to infer variable types anywhere it can, without annotations. With luck, 'independently useful linter' territory. - Then anyone can add annotations (in comments or stubs, for Py2 compatibility) to various parts of the codebase in future PRs. This is where type hints become really useful and mypy does substantially more than any other Python linters. @DRMacIver <https://github.com/drmaciver>, if you think steps one and two are useful in isolation I'd be happy to make an attempt, and from there it's mostly an incremental job. The remaining question is basically whether to use comments or stub files (until we drop Python2 support, at which point we can use inline annotations). Comments are more local to the related code, but if we use stub files it would be possible to exclude them from the has_source_changes check and therefore avoid releases for hint-only PRs. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#200 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAA5NNDBN6dM7UZr1uaIS3ddZCTUUmKIks5sh4kDgaJpZM4GOkyp> .

[Zac-HD]

Based on the PEP and typeshed docs, I thought that either comments or stubs in the Hypothesis package (ie in local code, not typeshed) would work - assuming that people use the --follow -imports option, obviously. I think this makes the stubs/comments distinction moot for downstream type-checking.

[Daenyth]

Currently either comments or stubs will work for local checking within the hypothesis project. It won't work for people pip installing hypothesis.
See python/typing#84

[DRMacIver]

I've gotta be honest, if the process of doing this correctly requires submitting pull requests to typeshed, my inclination is not to bother and to wait for the ecosystem around this to mature into something actually useful.

[Daenyth]

I can't argue with that. The work involved is silly, though there's a pep in the progress of being drafted to address the issue.

I think it's fairly reasonable to also just have them locally inline (with comments) and wait for the ecosystem to allow pip-install to provide them, ignoring typeshed and downstream consumers.

[DRMacIver]

I think it's fairly reasonable to also just have them locally inline (with comments) and wait for the ecosystem to allow pip-install to provide them, ignoring typeshed and downstream consumers.

That's true. I'm definitely not against that solution, and if @Zac-HD (or anyone else!) wants to put in the work to start doing that incrementally as he proposed, I'd be perfectly happy with that.

I'm personally still a bit sceptical about the typing module, but I don't have any problem with us giving it a go.

[Zac-HD]

Whew, that was a longish read. My interpretation of the various issues and PEP 561 is that putting foo.pyi next to each foo.py will work for us now, and for downstream users once tooling improves. Until then, distributing type hints is simply more trouble than it's worth.