You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The objectives are to improve model management and experiments tracking, and to get closer to the standards of deep learning community.
Why
ClinicaDL is good for research experiments and works well in stand alone. All the the tools for model management and experiments tracking (MAPS) are specific to ClinicaDL. However it might make it hard to start with, which is really not the objective.
In addition, model managed by MLflow are easy to use and deploy, which is a feature that clearly lacks in ClinicaDL.
MLflow also gives a solid framework for reproducibility that is widely recognized by experts of the domain. The structure proposed seems robust and scalable.
Finally it will increase flexibility by giving the possibility to plug any tools that can be used with MLflow.
MLflow tools
There are four distinct tools in MLflow:
MLflow Models: packaging of ClinicaDL's models, thats would improve/replace MAPS. This will allow us to standardize our model management to ease for deployment and improve compatibility.
MLflow Projects: packaging of data science code, its main purpose is reproducibility. It creates project that can be easily reproduced.
MLflow Tracking: API to log parameters, hyper-parameters, outputs and code. It helps for tracking, visualization, transparency and reproducibility.
MLflow Registry: manage lifecycle by versioning and tagging models (less useful in the case of ClinicaDL). But the fact that the models would respect all the above tools will allow users to use it if needed.
The main tasks will to be to select and adapt the wanted tools to ClinicaDL and also adapt a part of ClinicaDL to these tools.
That will probably require to redefine what we call models, runs and experiences and change the MAPS.
According to the documentation, it would imply to change our core code by integrating Pytorch Lightning instead of Pytorch. This is good point since it would help us to add many features that we wanted to add.
Conclusion
The adding of MLflow to ClinicaDL is going to help us for reproducibility purposes and also for model management and flexibility.
Others tools ?
Weight&Biases is an other interesting tool, however we believe that MLflow is more suited to our case since it is not owned by a private compagny.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Integration of MLflow tools in ClinicaDL
The objectives are to improve model management and experiments tracking, and to get closer to the standards of deep learning community.
Why
ClinicaDL is good for research experiments and works well in stand alone. All the the tools for model management and experiments tracking (MAPS) are specific to ClinicaDL. However it might make it hard to start with, which is really not the objective.
In addition, model managed by MLflow are easy to use and deploy, which is a feature that clearly lacks in ClinicaDL.
MLflow also gives a solid framework for reproducibility that is widely recognized by experts of the domain. The structure proposed seems robust and scalable.
Finally it will increase flexibility by giving the possibility to plug any tools that can be used with MLflow.
MLflow tools
There are four distinct tools in MLflow:
The main tasks will to be to select and adapt the wanted tools to ClinicaDL and also adapt a part of ClinicaDL to these tools.
That will probably require to redefine what we call models, runs and experiences and change the MAPS.
According to the documentation, it would imply to change our core code by integrating Pytorch Lightning instead of Pytorch. This is good point since it would help us to add many features that we wanted to add.
Conclusion
The adding of MLflow to ClinicaDL is going to help us for reproducibility purposes and also for model management and flexibility.
Others tools ?
Weight&Biases is an other interesting tool, however we believe that MLflow is more suited to our case since it is not owned by a private compagny.
Beta Was this translation helpful? Give feedback.
All reactions