[question] Raise minimum required Python to 3.7

[henryiii]

I found a typo in the mypy configuration; we are not actually imposing anything on mypy. If we enable what we are trying to check, we have dozens of typing errors, mostly due to missing or incomplete types. It would be nice to expand to checking some of the testing too, now that pytest supports typing and it would help with issues like #507 , where running mypy locally is easy, while running the tests locally is harder (except if CIBW_PLATFORM is set to linux). Fixing the incomplete types would be significantly easier if I can just bump the required Python to 3.7 and use from __future__ import annotations; would that be something we could do for the next version? Otherwise, I'll just add TYPE_CHECKING :/. It would also let us remove some stringified annotations.

[joerick]

I guess 3.7 is available across all the CIs we support, so I don't see why not! If we're changing the minimum, we could even skip one and go to 3.8?

[henryiii]

I don't think we get that much from 3.8 at the moment (though, admittedly, I've built a 3.7+ codebase before, but not a 3.8+ one, so not as familiar with the changes in practice). Though, admittedly, supporting 3.7 if all CI systems have 3.8 might be a little silly.

from __future__ import annotations make it possible to keep up with mypy changes instead of Python ones for annotation syntax, which is fantastic. When mypy releases 0.800, we would be able to use Python 3.9 list[str] syntax in annotations without requiring 3.9. :) Did you see there's a PR to build mypy with cibuildwheel?

[joerick]

When mypy releases 0.800, we would be able to use Python 3.9 list[str] syntax in annotations without requiring 3.9

I'd love this. Also, would we be able to use str|Path instead of Union[str, Path]? The Union thing always seemed extra verbose.

Did you see there's a PR to build mypy with cibuildwheel?

Yes :) very cool.

I don't think we get that much from 3.8 at the moment

One thing I just came across today, which would be very handy - literal types.

[henryiii]

Also, would we be able to use str|Path

I'm thinking that will take a bit longer, personally my goal would be to get a 3.9 compatible version out, then start adding the 3.10 features like this one. But not following it all that closely in MyPy (and I'd think it might be pretty easy, but could be very wrong). They might want to wait till after 3.10 is out to make sure it doesn't get pulled.

Literal types are useful if you are not using enums. I was very excited for them when designing a protocol, but then it turned out we needed to be able to allow user extension in the future, so we had to downgrade it into str. :/

[joerick]

Literal types are useful if you are not using enums.

Yeah, that was the kind of thing I was wanting them for, while working on #484 - it wasn't worth defining an enum for around 10-15 lines of logic that switched on 'x86_64/arm64', but Literal['x86_64', 'arm64'] would have been perfect for that variable. I've just gotten used to such niceties from Typescript. Actually, in Typescript, you find that it's rarely worth defining a 'real' enum, a literal union type is more convenient 95% of the time.

[joerick]

I hadn't realised that str|Path was a 3.10 thing! Oh well, gotta be patent :)

[henryiii]

Looks like MyPy master does support it: python/mypy#9647 (from the MyPy gitter). So next release of MyPy and when we use Python 3.7+, we can use them! You can track the status of the other implementations in python/typeshed#4819. Things like python/mypy#9880 don't affect type annotations, just normal code - and we can't use that until we require 3.10+, so not an issue.

[YannickJadoul]

So next release of MyPy and when we use Python 3.7+, we can use them!

How are these annotations then valid? Aren't they also evaluated, or would we need to write def foo(bar: "str | Path") ?

[henryiii]

from __future__ import annotations

def foo(bar: str | Path):
    pass

is perfectly valid Python 3.7+ code, try it if you don't believe me. :) All annotations are unevaluated strings with that future import in 3.7+ (becomes the default in 3.10+). Is nice for classes (though if you expect subclassing, you should use a bound TypeVar instead of referencing the class most of the time).

The only issue is MyPy doesn't like it yet (except if you use master, apparently, haven't built master in a little while).

[henryiii]

Here's an example (had to look for a bit to find one I was lazy on and didn't use TypeVar): https://github.com/Princeton-Penn-Vents/princeton-penn-flowmeter/blob/b72e41ec966b78b65a9e09f2bef3ded640819673/processor/rolling.py#L131

That line would not be valid in Python 3.6.

[YannickJadoul]

All annotations are unevaluated strings with that future import in 3.7+ (becomes the default in 3.10+).

Oh, OK, that's where I was wrong. I thought they were evaluated later (to solve the mutual dependency case), but they're not evaluated at all, anymore! Thanks, learned something new again, today! :-)

[henryiii]

Yep. If you ask for type annotations, you can get them evaluated. I don't ever do that though, so I don't remember how. I think Guido tweeted about it recently. Yep, so I think it's get_type_hints? (in 3.10, it evaluates them for you, I think).

[hauntsaninja]

(I also added a new documentation section to mypy that's meant to address these kinds of questions: https://mypy.readthedocs.io/en/latest/runtime_troubles.html)

[henryiii]

I propose we move as soon as we release 1.8.0.

[henryiii]

If pipx in #542 on ubuntu-18.04 picks up 3.6, I'd propose we wait on this. A little cleanup is nice, but working out of the box on 18.04 is more important for now.

[henryiii]

Next release of MyPy (build using cibuildwheel, of course) is out! I think we should hold off on this Python version bump until either we provide a way for the action to get a newer cached Python version though pipx (needs a path or a runnable expression, like python38, which is not provided. The path might be doable, though, depends.), or until 18.04 is no longer "latest" and most users have switched to 20.04.