-
Notifications
You must be signed in to change notification settings - Fork 5.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Track losses with tensorboard #11568
Open
SwamiKannan
wants to merge
8
commits into
run-llama:main
Choose a base branch
from
SwamiKannan:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Commits on Mar 2, 2024
-
This .py file has a class "TBSentenceTransformer" that is inherited from SentenceTransformer. This class is to be used instead of SentenceTransformer if we want to log our training progress using TensorBoard. I have been waiting for the SentenceTransformer team to integrate tensorboard into their repo but it seems like they are taking some time doing it since it is ground up. This is an alternative to that. Code has been written to minimize the impact on any of the repos using SentenceTransformers i.e. if at any time, SentenceTransformers do come up with a reporting framework, it will be easy to revert to that if required with minimal changes to the code
Configuration menu - View commit details
-
Copy full SHA for 6ca6652 - Browse repository at this point
Copy the full SHA 6ca6652View commit details -
A couple of lines added to the creation of the SentenceTransformersFinetuneEngine class: 1. Add a parameter log_path that specifies where the tensorboard logs should be stored 2. Update the constructor: self.model = = TBSentenceTransformer() if the log_path is specified else self.model = SentenceTransformer()
Configuration menu - View commit details
-
Copy full SHA for afa4ab7 - Browse repository at this point
Copy the full SHA afa4ab7View commit details -
Updated class inherited from SentenceTransformers to include tensorboard reporting
Configuration menu - View commit details
-
Copy full SHA for 73f1435 - Browse repository at this point
Copy the full SHA 73f1435View commit details -
Contains an inherited class from SentenceTransformer called TBSentenceTransformer that allows llamaindex's SentenceTransformersFinetuneEngine to track and record loss while finetuning using tensorboard. Credit to this PR: UKPLab/sentence-transformers#1532 , by Ahmed Magdy https://github.com/Mogady
Configuration menu - View commit details
-
Copy full SHA for 5bc5f1f - Browse repository at this point
Copy the full SHA 5bc5f1fView commit details -
Updated SentenceTransformersFinetuneEngine's self.model to accept a TBSentenceTransformer object if a log_path is specified or revert to accepting a SentenceTransformer object. Initial commits had some mistakes..
Configuration menu - View commit details
-
Copy full SHA for d006816 - Browse repository at this point
Copy the full SHA d006816View commit details
Commits on Mar 3, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 114f580 - Browse repository at this point
Copy the full SHA 114f580View commit details -
Configuration menu - View commit details
-
Copy full SHA for 28e62dc - Browse repository at this point
Copy the full SHA 28e62dcView commit details
Commits on Mar 4, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 37129b3 - Browse repository at this point
Copy the full SHA 37129b3View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.