Generate .pyi files for all generated modules

[virtuald]

This will improve our user experience significantly.

Much of the work for extracting appropriate annotations has actually already been done in the pybind11 for robotpy-rev (which these hooks/templates were derived from). I had removed them because I mistakenly assumed we would only need them for generating sim backends.

[auscompgeek]

I presume this would eliminate the need to explicitly list everything we're importing in __init__.py to make pylint happy, right?

[virtuald]

I'm not sure, honestly.

[auscompgeek]

I've incidentally found that mypy runs fairly well on robot code now that wpilib doesn't do weird dynamic magic, with the exception of mypy not being able to type-check calls to wpilib.

We'll need to remember to throw in a py.typed into packages when creating .pyi stubs so that mypy will actually look at them.

[auscompgeek]

I just realised adding inline C++ to our bindings complicates this a fair bit, since we probably also want to list those functions in a .pyi… which means we want to be able to add stubs inline as well I guess?

[virtuald]

Yes. Also probably want to be able to generate these without needing to actually import the objects, as we want to be able to cross-compile -- but maybe you could argue they aren't needed when cross-compiling.

[virtuald]

One thing to note for @bvisness benefit, is that I believe both @auscompgeek and @TheTripleV have tried to leverage various typegen things with some success towards this. There are some pybind11 specific typegen things that are buggy because pybind11's types don't always get reliable names.

[auscompgeek]

I think last time I was looking into it for wpilib, mypy's stubgen bailed, and pybind11-stubgen generated syntactically invalid code.

[virtuald]

I think one of the biggest questions that remains to be answered is whether we should generate the annotations at runtime (eg, inspect the python modules directly -- which is how the documentation generation works) or whether we should generate the annotations at compile time. It would be really convenient to do it at compile time, but I suspect for best results we would need to do it at runtime -- maybe the build process could import it directly right after building?

[TheTripleV]

So for me, it looked like pybind11-stubgen was pretty much perfect in stubs generation. The only problem was, pybind11's docstrings would sometimes contain c++ types instead of python types, leading to invalid types in the stubs.

I guess at compile type, if we generated a map from c++ types to python types, we could just run a find/replace and get good stubs.

[auscompgeek]

find/replace

It'd probably be better to make sure any C++ type names get mangled to usable Python type names at the pybind11 level instead.

[TheTripleV]

Example of a bad type:

    def initSendable(self, builder: frc::SendableBuilder) -> None: 
        """
        Initializes this Sendable object.

        :param builder: sendable builder
        """