Fix zero_grad place resulting in zero logs #2555
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Also fix bug in min_lr check
…#2459) I have not changed the behavior of AMP, APEX, TPU as I don't have the means to test it.
Closes pytorch#2459 with help of PR pytorch#2470
vfdev-5
reviewed
Apr 19, 2022
There was a problem hiding this comment.
Thanks for the PR @sadra-barikbin
Few comments to make it better
sadra-barikbin
force-pushed
the
fix_2459
branch
from
a6d8ae3 to
48ea731
Compare
vfdev-5
reviewed
Apr 19, 2022
vfdev-5
reviewed
Apr 19, 2022
vfdev-5
reviewed
Apr 19, 2022
sadra-barikbin
force-pushed
the
fix_2459
branch
from
7b7d88e to
ee21110
Compare
sadra-barikbin
force-pushed
the
fix_2459
branch
from
7eb5063 to
3cde27f
Compare
vfdev-5
approved these changes
May 1, 2022
|
@sadra-barikbin your PR is failing on CUDA:
Please send a fixing PR that we'll test directly on CUDA |
vfdev-5
added a commit
that referenced
this pull request
May 1, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #2459
Description:
Based on PR #2470 . Tests are added.
Check list: