Optionally include documentation in GDExtension API dump

[rburing]

Language bindings implemented in GDExtension that generate glue code (in the future, C#?) may want to include documentation (for classes, properties, methods, signals, etc.) to improve the user experience in code editors and REPLs.

This PR adds an option to include that documentation in the extension_api.json file.

This is much more convenient than having to obtain and parse ~829 XML files from the Godot source code.

(Ideally the build system for a GDExtension should depend just on having the correct API file, and not on having to run Godot or having the Godot source code.)

• Size of extension_api.json generated using --dump-extension-api: 5559324 bytes.
• Size of extension_api.json generated using --dump-extension-api-with-docs: 9121481 bytes.
• The size increase is 65%.

Feedback welcome.

There are quite some loops that could be hidden away by adding new core methods to ClassDoc, if desired.

[Bromeon]

As a heavy user of extension_api.json, I would welcome this addition. It would mean one less data source and one less format (XML) that needs to be understood.

The XML files also contain some meta-information that is exclusive to them, for example whether a class is experimental. Do you think it makes sense to add this to the JSON as well -- maybe even the base (non-doc) version?

[dsnopek]

Thanks, this is a great idea!

I tested locally, and it seems to be working, although I only skimmed through the JSON, I didn't scrutinize it in depth.

The code looks good to me as well! I only have some minor notes about the command-line help info.

[rburing]

Thanks for the timely reviews!

@Bromeon

The XML files also contain some meta-information that is exclusive to them, for example whether a class is experimental. Do you think it makes sense to add this to the JSON as well -- maybe even the base (non-doc) version?

It makes sense to me, but I would prefer not to touch it in this PR.

We can discuss it with the GDExtension team on RocketChat.

[akien-mga]

Thanks!

[maiself]

Working with python:

Parsing the bbcode is a little awkward, but not too bad. This is really cool, thanks!

Edit: Just to add a bit of info: compressed json went from ~400kb to ~1.2mb.

[akien-mga]

Parsing the bbcode is a little awkward, but not too bad.

I assume many language bindings will use Python for codegen from the json. Maybe we could have a small python library that can be used for parsing the bbcode in each language binding?

[GeorgeS2019]

@rburing
Thanks for your contribution

We are starting to see a path to create GDExtension C# Bindings using your merge

[rburing]

For those who found this PR and want to use the feature: note that in #83318 we changed the key name from "documentation" to "description" (for internal consistency) and we added "brief_description" to classes and built-in classes.

This will allow generating even more detailed documentation.