Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ClearML experiment tracking integration #8620

Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
ec7986a
Add titles to matplotlib plots
thepycoder Jul 6, 2022
11da722
Add ClearML Experiment Tracking integration.
thepycoder Jul 6, 2022
d395a28
Add ClearML Data Version Management automatic download when requested
thepycoder Jul 6, 2022
a160dfc
Add ClearML Hyperparameter Optimization
thepycoder Jul 6, 2022
16a1a48
ClearML save period integration
thepycoder Jul 12, 2022
8e957c9
Fix wandb breaking when used with ClearML dataset
thepycoder Jul 12, 2022
027ca12
Fix wandb breaking when used with ClearML resume and dataset
thepycoder Jul 12, 2022
a5ae4bb
Add ClearML documentation
thepycoder Jul 7, 2022
29a2686
fixed small bug in clearml integration that misreports epoch number
thepycoder Jul 11, 2022
be45d1b
Final ClearMl additions before refactor
thepycoder Jul 12, 2022
bd20628
Add correct epoch reporting
thepycoder Jul 12, 2022
c69d56f
Add remote execution and autoscaling docs for ClearML integration
thepycoder Jul 12, 2022
358354d
Added images to clearml integration docs
thepycoder Jul 12, 2022
fd0b10d
fixed logo alignment bug and added hpo screenshot clearml
thepycoder Jul 12, 2022
1cbe74b
Fixed small epoch number bug in clearml integration
thepycoder Jul 12, 2022
51f051d
Remove saved model flush clearml
thepycoder Jul 12, 2022
2557ace
Cleanup clearml readme section
thepycoder Jul 12, 2022
3c9403b
Cleaned up clearml logger docstring
thepycoder Jul 12, 2022
85ac912
Remove resume readme section clearml
thepycoder Jul 12, 2022
a9bb3be
Clearml integration cleanup
thepycoder Jul 12, 2022
a29eb1c
Updated ClearML documentation
thepycoder Jul 14, 2022
806c22a
Added dark vs light icons ClearML Readme
thepycoder Jul 14, 2022
194cf62
Clearml Readme styling
thepycoder Jul 14, 2022
421eb19
Add better gifs
thepycoder Jul 14, 2022
36ce901
Fixed gif file size
thepycoder Jul 14, 2022
aa36080
Add better images in tutorial notebook
thepycoder Jul 14, 2022
ba99667
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 18, 2022
c77575f
Merge branch 'master' into feature/clearml_integration_no_resume
glenn-jocher Jul 30, 2022
359d4fa
Merge branch 'master' into feature/clearml_integration_no_resume
glenn-jocher Jul 30, 2022
02bada8
Addressed comments in PR #8620
thepycoder Aug 1, 2022
eda35ea
Merge branch 'feature/clearml_integration_no_resume' of github.com:th…
thepycoder Aug 1, 2022
bae0c2f
Fixed circular import
thepycoder Aug 1, 2022
c3c394f
Fixed circular import
thepycoder Aug 1, 2022
c370935
Update tutorial.ipynb
glenn-jocher Aug 1, 2022
a770823
Update tutorial.ipynb
glenn-jocher Aug 1, 2022
8561281
Merge branch 'master' into feature/clearml_integration_no_resume
glenn-jocher Aug 1, 2022
d4bbc92
Inline comment
glenn-jocher Aug 1, 2022
5d31d2d
Merge branch 'master' into feature/clearml_integration_no_resume
glenn-jocher Aug 1, 2022
40a9616
Merge branch 'master' into feature/clearml_integration_no_resume
glenn-jocher Aug 2, 2022
0bf8969
Restructured tutorial notebook
thepycoder Aug 3, 2022
c49f827
Add correct ClearML link to README
thepycoder Aug 3, 2022
efa3d09
Merge branch 'master' into feature/clearml_integration_no_resume
glenn-jocher Aug 3, 2022
28fedbd
Merge branch 'master' into feature/clearml_integration_no_resume
glenn-jocher Aug 4, 2022
0c10d66
Update tutorial.ipynb
glenn-jocher Aug 4, 2022
424b524
Merge branch 'master' into feature/clearml_integration_no_resume
glenn-jocher Aug 5, 2022
7bde104
Update general.py
glenn-jocher Aug 5, 2022
92657cb
Update __init__.py
glenn-jocher Aug 5, 2022
a98dc0f
Update __init__.py
glenn-jocher Aug 5, 2022
9c0eab3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 5, 2022
fd0cff6
Update __init__.py
glenn-jocher Aug 5, 2022
c97a2ad
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 5, 2022
8bbf04e
Update __init__.py
glenn-jocher Aug 5, 2022
0930d59
Update README.md
glenn-jocher Aug 5, 2022
68f9a9c
Update __init__.py
glenn-jocher Aug 5, 2022
b618614
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 5, 2022
918d861
spelling
glenn-jocher Aug 5, 2022
092fe34
Update tutorial.ipynb
glenn-jocher Aug 5, 2022
e6eca5f
notebook cutt.ly links
glenn-jocher Aug 5, 2022
b45cac4
Update README.md
glenn-jocher Aug 5, 2022
40d3b95
Update README.md
glenn-jocher Aug 5, 2022
5ed6620
cutt.ly links in tutorial
glenn-jocher Aug 5, 2022
00eda91
Removed labels as they show up on last subplot only
glenn-jocher Aug 5, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Clearml integration cleanup
  • Loading branch information
thepycoder committed Jul 18, 2022
commit a9bb3be572f2c18901fbe11a03de8591854e7123
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ Get started in seconds with our verified environments. Click each icon below for

|Weights and Biases|Roboflow|ClearML ⭐ NEW|
|:-:|:-:|:-:|
|Automatically track and visualize all your YOLOv5 training runs in the cloud with [Weights & Biases](https://wandb.ai/site?utm_campaign=repo_yolo_readme)|Label and export your custom datasets directly to YOLOv5 for training with [Roboflow](https://roboflow.com/?ref=ultralytics) |Track experiments, vizusalise them and remotely train YOLOv5 in the cloud or locally using [ClearML](https://clear.ml/?ref=ultralytics)
|Automatically track and visualize all your YOLOv5 training runs in the cloud with [Weights & Biases](https://wandb.ai/site?utm_campaign=repo_yolo_readme)|Label and export your custom datasets directly to YOLOv5 for training with [Roboflow](https://roboflow.com/?ref=ultralytics) |Track experiments, vizusalise them and **remotely train YOLOv5** in the cloud or locally using [ClearML](https://clear.ml/?ref=ultralytics)

<!-- ## <div align="center">Compete and Win</div>

Expand Down
2 changes: 1 addition & 1 deletion utils/general.py
Original file line number Diff line number Diff line change
Expand Up @@ -436,7 +436,7 @@ def check_file(file, suffix=''):
torch.hub.download_url_to_file(url, file)
assert Path(file).exists() and Path(file).stat().st_size > 0, f'File download failed: {url}' # check
return file
elif file.startswith('clearml:'):
elif file.startswith('clearml://'):
# ClearML Dataset ID
return file
else: # search
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably prepare for the edge case where user passes clearml file but clearml is not installed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're 100% correct, thanks for catching it!

Expand Down
5 changes: 3 additions & 2 deletions utils/loggers/clearml/clearml_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@


def construct_dataset(clearml_info_string):
dataset_id = clearml_info_string.replace('clearml:', '')
dataset_id = clearml_info_string.replace('clearml://', '')
dataset = Dataset.get(dataset_id=dataset_id)
dataset_root_path = Path(dataset.get_local_copy())

Expand Down Expand Up @@ -82,14 +82,15 @@ def __init__(self, opt, hyp):
tags=['YOLOv5'],
output_uri=True,
auto_connect_frameworks={'pytorch': False}
# We disconnect pytorch auto-detection, because we added manual model save points in the code
)
# ClearML's hooks will already grab all general parameters
# Only the hyperparameters coming from the yaml config file
# will have to be added manually!
self.task.connect(hyp, name='Hyperparameters')

# Get ClearML Dataset Version if requested
if opt.data.startswith('clearml:'):
if opt.data.startswith('clearml://'):
# data_dict should have the following keys:
# names, nc (number of classes), test, train, val (all three relative paths to ../datasets)
self.data_dict = construct_dataset(opt.data)
Expand Down