Track losses with tensorboard #11568
Conversation
This .py file has a class "TBSentenceTransformer" that is inherited from SentenceTransformer. This class is to be used instead of SentenceTransformer if we want to log our training progress using TensorBoard. I have been waiting for the SentenceTransformer team to integrate tensorboard into their repo but it seems like they are taking some time doing it since it is ground up. This is an alternative to that. Code has been written to minimize the impact on any of the repos using SentenceTransformers i.e. if at any time, SentenceTransformers do come up with a reporting framework, it will be easy to revert to that if required with minimal changes to the code
A couple of lines added to the creation of the SentenceTransformersFinetuneEngine class: 1. Add a parameter log_path that specifies where the tensorboard logs should be stored 2. Update the constructor: self.model = = TBSentenceTransformer() if the log_path is specified else self.model = SentenceTransformer()
Updated class inherited from SentenceTransformers to include tensorboard reporting
Contains an inherited class from SentenceTransformer called TBSentenceTransformer that allows llamaindex's SentenceTransformersFinetuneEngine to track and record loss while finetuning using tensorboard. Credit to this PR: UKPLab/sentence-transformers#1532 , by Ahmed Magdy https://github.com/Mogady
Updated SentenceTransformersFinetuneEngine's self.model to accept a TBSentenceTransformer object if a log_path is specified or revert to accepting a SentenceTransformer object. Initial commits had some mistakes..
dosubot
bot
added
the
size:XL
| from llama_index.finetuning.types import BaseEmbeddingFinetuneEngine | ||
|
|
||
|
|
||
| class SentenceTransformersFinetuneEngine(BaseEmbeddingFinetuneEngine): |
There was a problem hiding this comment.
what is the change here for this file?
There was a problem hiding this comment.
So if a log_path is provided,
self.model will be a TBSentenceTransformer object
else if no log_path is provided,
self.model will be a standard SentenceTransformer object
Lines 40-43 in sentence_transformer.py
if log_path:
self.model = TBSentenceTransformer(model_id, writer_path = log_path)
else:
self.model = SentenceTransformer(model_id)
sentence_transformer.py
Outdated
There was a problem hiding this comment.
did you mean to commit these two files in the root?
There was a problem hiding this comment.
Yes, there were two files to be committed.
These were to be in the same folder as sentence_transformer.py i.e. in finetuning\embeddings\
There was a problem hiding this comment.
Do correct me if I have missed something?
There was a problem hiding this comment.
I don't think these files belong in the root of the entire repo 👀
There was a problem hiding this comment.
oh god.... i'm so sorry... that was a mistake. Resolved on my repo so I am assuming it is replicated in this PR?
Apologies. This is the first time I am committing to such a large and useful repo so I may be a little lost
dosubot
bot
added
size:L
Description
This PR allows us to track progress of the loss for each step and for each epoch with Tensorboard. It is specifically written to minimize changes to the existing repo. Hence, if the SentenceTransformer repo finally updates with a reporting framework, these changes can be reverted with just two lines and the deletion of one file.
Added screenshot of tensorboard report running with this script
Type of Change
Please delete options that are not relevant.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Suggested Checklist:
make format; make lintto appease the lint godsAdded screenshot of my test run