PEP: Distributing and Packaging Type Information

[emmatyping]

I plan on making a POC implementation of the distutils extension when I have time either later today or tomorrow.

A couple points Im not sure about:

should the keyword be a boolean about whether the package is stubbed or not? Then stubbed == True inline == False and type unsafe == None.

Mixed inline/stubbed packages - I originally wrote a version with an additional option for this, but it seemed to make things more complicated than needed.

[ilevkivskyi]

Just three small comments from me.

[ilevkivskyi]

This line is too long

[ilevkivskyi]

I would move this section below the next one so that the order will match the order of three numbered options above.

[ilevkivskyi]

th the same -> with the same

[ilevkivskyi]

should the keyword be a boolean about whether the package is stubbed or not?

I think the string is OK typed="inline" is straightforward and easy to remember.

[emmatyping]

Thanks for the review @ilevkivskyi! I should have addressed all of your comments.

[gvanrossum]

Since the PEP is formatted correctly (the tests pass) I have merged it, so you officially have PEP number 461. Time to post to python-ideas!

[emmatyping]

Thanks! Though it is 561. I will write up a post for python-ideas now.

[gvanrossum]

Dang. I can't fix the commit message. :-(

[emmatyping]

Well, if you are fast, you can git commit --amend then git push --force-with-lease origin master, but yes it is unfortunate :(

[gvanrossum]

No, that didn't work. Oh well.

[Zac-HD]

I'd appreciate some clear advice for libraries that support Python 2.7 (--3.6), and want to support type hints in the same package. Our aim is to add another layer of analysis to our CI, but a low-effort and non-breaking way to distribute type hints to users would be great - HypothesisWorks/hypothesis#200 proposes to add stub files alongside the corresponding .py files and then wait for tooling to catch up 😄

In particular, the current draft PEP doesn't specify if this is sufficient, or whether we'll also need to add some metadata to setup.py. In the latter case, will this still work for users stuck on 2.7.3 with an old packaging toolchain? Unfortunately backwards-compatibility is more important to us than distributing the type hints 😕

[emmatyping]

@Zac-HD The relevant section on the latest draft https://www.python.org/dev/peps/pep-0561/#stub-only-packages indicates that an additional keyword is required in calls to setup(). Simply shipping stubs will not get picked up by type checkers under this PEP. However, I plan on changing parts of the PEP rather significantly based on the review of @ncoghlan, so I wouldn't rely on the text of the PEP as it stands. I will say that in general if this PEP is accepted, a package will need to indicate that it has opted into type checking. It will be largely up to the packaging tools to decide how that looks generally, though I expect that a new keyword to setup() is likely (also I plan on contributing to several major packaging tools as I find time). As for the ability to use this in older Python versions, I'm not sure what will be supported. I hope to be able to find a way to extend distutils on older Python versions, however I'm not certain of how that will happen at this point. Thank you for the feedback, it is exactly what I am looking for! And apologies for not having a better answer at the moment.

[ncoghlan]

For projects that want to natively offer typing information, shipping pyi files alongside their Python files is fine (and typecheckers should already support that). This PEP is aimed more at decentralising typeshed than it is at changing the way native typing support works.

As far as packaging tools go, the fact that setuptools (rather than distutils) will be the right place to add any helpers for generating the metadata is the main reason I asked @ethanhs to take the setup.py related aspects out of the PEP and replace them with a specification for how to read the relevant advisory metadata out of a PEP 376 style installation database.

[emmatyping]

@ncoghlan @Zac-HD

For projects that want to natively offer typing information, shipping pyi files alongside their Python files is fine (and typecheckers should already support that).

mypy does not currently support this, if the Python files are in site/dist-packages. I don't believe pytype nor PyCharm support it either. This is a problem the PEP is aiming to solve. Adding the package directory causes many issues for type checkers.

This PEP is aimed more at decentralising typeshed than it is at changing the way native typing support works.

Actually, that is not the main intent of this PEP, it is one of the encouraged changes for sure (and a side effect of the changes proposed in this PEP), but it is not the only change the PEP is trying to bring.

And yes, sadly distutils will not be getting the new metadata built in, so I am not sure older versions will get it unless they use setuptools or something else that will add support for it.

[Zac-HD]

And yes, sadly distutils will not be getting the new metadata built in, so I am not sure older versions will get it unless they use setuptools or something else that will add support for it.

I'm OK with requiring a newish setuptools/pip/whatever to get the type stubs - so long as old tools can still install the code!

[sproshev]

Am I right that Python-Version: 3.7 does not mean that stub packages and all the corresponding stuff should be processed with 3.7+ interpreters only?

[emmatyping]

Yes that is correct. The standard will work on any Python version back to 2.7.

[JelleZijlstra]

Perhaps that header should be changed? There's really nothing about the PEP that only applies to 3.7, because it requires no changes to Python itself, only to type checkers.

[gvanrossum]

If we change features in *typing.py* they only tend to be added to the latest Python version. Maybe we need some clarification near the top of the PEP. Can someone volunteer some text?