Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Torchmetrics usage improvements with classes instead of functionals #517

Merged
merged 71 commits into from
Sep 10, 2024

Conversation

DomInvivo
Copy link
Collaborator

@DomInvivo DomInvivo commented Jun 27, 2024

Changelogs

  • Relaxing the constraints on Torchmetrics version
  • Changing the code to use the update and compute to avoid memory issues with large validation set
  • Major changes and cleanup to the predictor_summaries.py to simply handle metrics updates and compute + new unit-tests
  • Fixing all unit-tests, including those testing the training and finetuning of the models
  • Adding more unit-tests to cover the changes in MetricWrapper
  • Adding more unit-tests to cover SingleTaskSummary and MultiTaskSummary
  • Adding more unittests in test_training.py and test_finetuning.py to make sure that the full pipeline runs
  • Moving to a ProgressBar callback to fix the progress bar logging
  • Running on a real task and ensuring that we recover the same performance, and that all metrics are reported correctly on CPU
  • Running again on single GPU
  • Running again with DDP multi-gpus, and that they give the expected values (all predictions are synced before computing the metrics)
  • Update all the config files to be compatible with new dataloader, and some changes in torchmetrics (example, f1score and accuracy now require the task parameter.

Checklist:

  • _Was this PR discussed in an issue? Issue Update MetricWrapper to work with the update/compute from torchmetrics #466
  • Add tests to cover the fixed bug(s) or the new introduced feature(s) (if appropriate).
  • Update the API documentation is a new function is added, or an existing one is deleted.
  • Write concise and explanatory changelogs above.
  • If possible, assign one of the following labels to the PR: feature, fix or test (or ask a maintainer to do it for you).

discussion related to that PR

@DomInvivo DomInvivo requested a review from WenkelF July 10, 2024 21:00
@DomInvivo DomInvivo self-assigned this Jul 10, 2024
@DomInvivo DomInvivo linked an issue Jul 10, 2024 that may be closed by this pull request
4 tasks
@DomInvivo DomInvivo added the fix PR that fixes an issue label Jul 12, 2024
@DomInvivo DomInvivo changed the title Torchmetrics usage improvements Torchmetrics usage improvements with classes instead of functionals Jul 16, 2024
DomInvivo and others added 28 commits July 17, 2024 15:54
@AnujaSomthankar AnujaSomthankar merged commit f723632 into graphium_3.0 Sep 10, 2024
0 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request fix PR that fixes an issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update MetricWrapper to work with the update/compute from torchmetrics
2 participants