Callback to customize the base classes used in generated bindings (merged)

[jnastarot]

It brings ability to add base classes from separate files by hand writing

    class SecondClass : CustomNS::FirstClass {
    public:
        SecondClass();
    private:
        int _value2;
    };

    class ThirdClass : SecondClass {
    public:
        ThirdClass();
    private:
        int _value3;
    };

def handle_classes_base(cls, for_python_stub):
        
        bases = []

        elem = cls.cpp_element()
        
        if elem.class_name == "SecondClass":
            bases.append("FirstClass" if for_python_stub else "CustomNS::FirstClass")

        return bases

options = litgen.LitgenOptions()
options.class_base_custom_derivation__callback = handle_classes_base

Result of using the callback

  auto pyClassSecondClass =
      py::class_<SecondClass, CustomNS::FirstClass>
          (m, "SecondClass", "")
      .def(py::init<>())
      ;


  auto pyClassThirdClass =
      py::class_<ThirdClass, SecondClass>
          (m, "ThirdClass", "")
      .def(py::init<>())
      ;

  class SecondClass(FirstClass):
      def __init__(self) -> None:
          pass

  class ThirdClass(SecondClass):
      def __init__(self) -> None:
          pass

If the callback returns an empty list it will be ignored and the process will fall back to the original logic

[jnastarot]

Deleted test call from option_custom_class_derive_test.py ff518bc

[pthom]

Hello,

Thanks a lot!!!

This PR was merged in 611d566.
I did some modifications in fe7f424 to enhance readability.

I am always pleased to see users of my library, and interested in knowing in which context they using it. Would you mind giving me more info about the context in which you are using it?

Thanks!

[jnastarot]

Hello,

Thanks a lot!!!

This PR was merged in 611d566. I did some modifications in fe7f424 to enhance readability.

I am always pleased to see users of my library, and interested in knowing in which context they using it. Would you mind giving me more info about the context in which you are using it?

Thanks!

Hello,
Thank you so much for this awesome project! I truly appreciate the effort that has gone into developing Litgen.

I would like to share my experience and provide some suggestions

Litgen is Useful

Litgen has been much more useful to me compared to tools like SWIG and Rosetta's Binder. The main reason is its flexibility in configuring the processing flow, which allows for more tailored adjustments.
I tried SWIG and Binder, but the bindings they generate are difficult to fix manually

Thoughts

However, I feel that litgen is still in its early stages and could benefit from additional features and improvements.

• Replace regex configuration with callbacks
  Using callbacks instead of regular expressions would enable more robust decision-making. Callbacks allow developers to perform additional checks and apply specific fixes dynamically, which is not always feasible with regex-based configurations.

• Add more options
  Features like those proposed in #27 are essential. Developers using ltgen often have diverse use cases, and more options would make the tool adaptable to their specific needs.

• Minimize header amalgamation
  Header amalgamation is not always desirable, especially in cases involving class derivation. Reducing this dependency or making it optional would enhance flexibility.

• My use case
  I am working on a pet project involving file format dissection. The project includes many files containing numerous classes organized under different namespaces. Header files have specific prefixes, and there are various definitions, such as struct and enum. Sometimes, structures are defined using typedef, and I believe this case should be handled natively by litgen #7.

• Since my project has a large number of classes, files, and namespaces, I expect bindings to be separated into submodules. Currently, I achieve this by post-processing generated files, using markers like <generated_from> and <submodule>. It would be fantastic if litgen could handle submodule separation out of the box.

• Performance
  I have noticed that litgen slows down significantly when processing many enums. For example, processing all my headers (~2.5 MB) takes around 40 minutes. I attempted to profile the issue but couldn’t quickly identify the bottleneck. This might be worth investigating further.

I believe litgen has the potential to become as popular and essential as tools like Pybind11 or Nanobind, as well as a valuable complement to other binding tools
Once again, thank you! Litgen has saved me a lot of time

[pthom]

I shared your suggestions here: #28