Failure when function parameter default value repr isn't valid python

[virtuald]

[2020-09-03 01:39:32,372] {__init__.py:60} ERROR - Generated stubs signature is degraded to `(*args, **kwargs) -> Any` for
[2020-09-03 01:39:32,372] {__init__.py:65} ERROR - def __init__(self: wpilib.controller._controller.trajectory.TrapezoidProfile, constraints: wpilib.controller._controller.trajectory.TrapezoidProfile.Constraints, goal: wpilib.controller._controller.trajectory.TrapezoidProfile.State, initial: wpilib.controller._controller.trajectory.TrapezoidProfile.State = <wpilib.controller._controller.trajectory.TrapezoidProfile.State object at 0x7fc9b00663b0>) -> None: ...

If I'm understanding the implementation correctly, it looks like mypy's stubgen will set it to ...?

Tried it out, looks like I'm right:

class A:
    pass


def foo(x: int = 5):
    pass


def a(a: A = A()):
    pass

Resulting generated pyi

from typing import Any

class A: ...

def foo(x: int=...) -> Any: ...
def a(a: A=...) -> Any: ...

[virtuald]

It is nice to use the pyi for documentation... so It seems to me that the easiest thing would be that if the default value were a basic type (str, int, float, None), then it can stay, otherwise replace it with .... The more complex way to go about this would be to try the default value first, and if the AST reports a failure then replace it with ....

[sizmailov]

I don't like to have many patches to generate valid pyi from invalid bindings because it makes library authors less eager about fixing their code.

This particular case is quite common, so I think it deserves attention.
I think I'll add regex replacement <\w+(\w+\.) object at 0x[0-9a-f]+> -> ... along with error message/emergency stop just as we did for invalid signature case earlier.

Note: representation of default value in pybind11 can be set via py::arg_v (probably you are aware of that).

[sizmailov]

Note that pybind allows to supply arbitrary string as default value which can break assumptions of any parser. Basically I'm saying that I'm too lazy to implement a invalid-default-values-proof parser 🦥

[sizmailov]

I've partially addressed the issue in 8016a14. Is it sufficient?

[virtuald]

I actually hadn't ran into py::arg_v previously, good to know!

Your test looks sufficient to meet my current needs, but I won't have time to try out my usecase until tonight.

[sizmailov]

Ok, I look forward for your findings.

[virtuald]

Ha, I had forgotten that you're literally parsing the docstring. Definitely a bit annoying, thank you so much for the effort, it finds my annoying default argument! I've got a few improvements in #45

... it does seem that pybind11 should have some hook to store enough data for someone to generate their own signatures so this parsing wouldn't be necessary. Oh well.

[sizmailov]

You are right, it would be nice to have finer control over generated docstrings or may be even annotations at some point :)
Although I imagine this would require quite amount of code which lowers the chances to become part of pybind in foreseeable future :(

I think this issue can be considered closed.

[orlandini]

Hello,

I am facing a similar issue but pybind11-stubgen is not being able to tell me where is the error located. here is the output I get:

[2021-02-17 11:48:09,798] {__init__.py:688} DEBUG - Parsing 'module_1' module
[2021-02-17 11:48:09,822] {__init__.py:98} ERROR - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2021-02-17 11:48:09,823] {__init__.py:98} ERROR - Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for
[2021-02-17 11:48:09,948] {__init__.py:688} DEBUG - Parsing 'module_2' module

Any clues on how to locate the issue? I am running pybind11-stubgen version 0.8.6 and pybind11 2.6.0rc3.

[orlandini]

I just added

logger.log(lvl,function_def_str)

after

logger.log(lvl, "Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for")

so I can find out which function to blame.

[sizmailov]

@orlandini I don't see how it could be. In any scenario the troubling function signature should be printed:

pybind11-stubgen/pybind11_stubgen/__init__.py

Lines 93 to 103 in f14583b

	if FunctionSignature.signature_downgrade:
	self.name = name
	self.args = "*args, **kwargs"
	self.rtype = "typing.Any"
	lvl = logging.WARNING if FunctionSignature.ignore_invalid_signature else logging.ERROR
	logger.log(lvl, "Generated stubs signature is degraded to `(*args, **kwargs) -> typing.Any` for")
	else:
	lvl = logging.WARNING
	logger.warning("Ignoring invalid signature:")
	logger.log(lvl, e.text.rstrip())
	logger.log(lvl, " " * (e.offset - 1) + "^-- Invalid syntax")

There is no problems in CI of latest master, so it's not reproduced on current master.

@orlandini I think it's better to open new issue if you think there is no glitch in your setup.

[orlandini]

Thanks @sizmailov . Since this would not occur at all times, I think it would be hard to find a minimal setup to reproduce the problem. In case I do find a consistent way to reproduce it, I will then open a new issue.