feat(types): Use typing.SupportsInt and typing.SupportsFloat and fix other typing based bugs.

[InvincibleRMC]

Description

Since int and floats by default can convert from __int__ and __float__ properly mark the input type. As discussed in #5158 this feature is useful since pybind already supports this and this pr updates so the type properly conveys that.

Updates the Final type to use the more narrower type hint since it always has to be assigned a value when created. Which means it would need to be assigned on declaration in C++ which is the narrower type hint.

Updates the std::function type to properly match the Callable type.

Update attr_with_type_hint to properly support io_name. This was accomplished by extracting out the parser into the new function generate_signature so it could use within attr_with_type_hint.

Closes #5158

Suggested changelog entry:

    Adds support for ``typing.SupportsInt`` and ``typing.SupportsFloat``. Update ``Final`` to be narrower type hint. Make ``std::function`` match ``Callable`` type. Fix ``io_name`` bug in ``attr_with_type_hint``.

[timohl]

Interesting addition.
You might be still working on this, but I left some comments.

Is the change in attr_with_type_hint related to the issue this PR fixes or is this a fix for another issue introduces by my changes from adding io_name?
Were there any failing tests for that or have you added some?
If not, adding tests would be great. (I have not gone through all the test_... changes yet).

[InvincibleRMC]

Is the change in attr_with_type_hint related to the issue this PR fixes or is this a fix for another issue introduces by my changes from adding io_name?

Yes this more related to the io_name changes but, wasn't uncovered since the attr_with_type_hint tests were written without any type that change based on return type. Since int was the old type being used in them making this change exposed this error.

[InvincibleRMC]

@timohl One other thing that came up is what to do about bool. Since the bool caster also works and there is no typing.SupportsBool. The common suggestions are use the object type or create a custom protocol. However pybind does not currently support protocols. I'm a little hesitant to broaden the type but maybe something like typing.Annotated[object, "bool"] might be a good middle ground for the input type.

[timohl]

Great that you could find a way to move the loop to a function. 👍
I can take a deeper second look on the weekend.

Regarding bool: I think object would do more harm than good.
It would allow any object to be passed and basically disable type checking.
I think this should stay as bool or we have to find a way to create the custom protocol in pybind11.
Creating a class should be no problem, but I am not sure if it is possible to derive from a python class in pybind11?

I also think this should go into a separate PR, since this PR contains some good additional fixes for io_name/attr_with_type_hint/std::function and should not wait on typing.SupportsBool.

[timohl]

I have added some comments.

Technically, this looks good to me, but the PR description must be improved.

Could you add a short explanation why supporting __float__/__int__ is useful?
In the end, it adds some clutter to the code and generated documentation, so there should be a good benefit for this.

If I understand correctly, this allows passing for example single element numpy arrays or pytorch tensors (See pytorch/pytorch#1129 (comment))

>>> import numpy as np
>>> zero_dim = np.array(3)
>>> scalar = np.float(3)
>>> single_elem = np.array([[[3]]])
>>> float(zero_dim), float(scalar), float(single_elem)
(3.0, 3.0, 3.0)
>>> empty = np.array([], dtype=np.float32)
>>> float(empty)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: only length-1 arrays can be converted to Python scalars

While pybind11 already supports this under the hood, this PR improves the type hints to reflect this.

[InvincibleRMC]

@timohl I have updated the pr description. Please let me know if you want more information or have any questions.

[rwgk]

Looks good to me. My suggestions are purely regarding naming.

[rwgk]

Nice, thank you!

@henryiii Do you want to glance through, too?

[gentlegiantJGC]

Thanks for working on this. I was thinking about this the other day.

[InvincibleRMC]

Thanks for working on this. I was thinking about this the other day.

No problem!