[RFC] Add Monitoring Interface for External Logging Backends

[ParagEkbote]

🚀 The feature, motivation and pitch

Liger-Kernel provides a simple way to improve training throughput, but we need to manually implement logging to see these results while training.

A lightweight monitoring interface would:

1. Make Liger's benefits visible in real experiments
2. Help debug performance regressions across release versions

The integration can feature a common liger_kernel.monitor module with a backend agnostic interface that can enable adding external logging backends such as Weights and Biases, Trackio, Comet, etc. in a seamless manner. The external backends would be decoupled such that the backends would only be imported if they are installed in the user environment. This would also ensure that no external dependencies are added to the core dependency list.

Feel free to give feedback on this RFC on the scope of the API and its design.

Alternatives

No response

Additional context

No response

[PKUWZP]

@ParagEkbote Thanks for proposing this idea. Do you have a rough idea on the scope of the work, e.g. do you want to have an explicit external logging backend such as W&B or MLFlow?

[ParagEkbote]

Thank you for reaching out. What I'm considering to do is to create a base LoggingClass and inherit it to the external logging backends such as W&B etc. In order to make them explicitly optional, we can raise theImportError in the module and add the dep to optional dependencies, ensuring that the core dependencies are not affected.

In order to keep the scope manageable, I propose to add the base LoggingClass with a popular logging backend such as W&B and Trackio and open to add the rest of backends such as MLFlow, Comet, SwanLab, etc. to the community.

cc: @PKUWZP

[ParagEkbote]

@ParagEkbote Thanks for proposing this idea. Do you have a rough idea on the scope of the work, e.g. do you want to have an explicit external logging backend such as W&B or MLFlow?

Does the above approach makes sense for you? Could we proceed with a PR to begin to add this interface?

cc: @PKUWZP