Add Clipping schedulers #556
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D42644261 |
Darktex
added a commit
to Darktex/opacus
that referenced
this pull request
Jan 20, 2023
Summary: Pull Request resolved: pytorch#556 This diff introduces gradient clipping schedulers that can be used to vary gradient clipping throughout training. Addresses pytorch#375 in OSS. Differential Revision: D42644261 fbshipit-source-id: 7480e2f81432bd4a05d58af420ee4a783bdd63f1
Darktex
force-pushed
the
export-D42644261
branch
from
15d5d51 to
6358c20
Compare
|
This pull request was exported from Phabricator. Differential Revision: D42644261 |
Darktex
added a commit
to Darktex/opacus
that referenced
this pull request
Jan 20, 2023
Summary: Pull Request resolved: pytorch#556 This diff introduces gradient clipping schedulers that can be used to vary gradient clipping throughout training. Addresses pytorch#375 in OSS. Differential Revision: D42644261 fbshipit-source-id: 0af54b977d41b164531c18d4804a2506a359ce28
Darktex
force-pushed
the
export-D42644261
branch
from
6358c20 to
ec71f8b
Compare
|
This pull request was exported from Phabricator. Differential Revision: D42644261 |
karthikprasad
approved these changes
Jan 23, 2023
| ) | ||
| @settings(deadline=None) | ||
| def test_checkpoints(self, noise_scheduler: Optional[Type[StepNoise]]): | ||
| def test_checkpoints( |
There was a problem hiding this comment.
Could you break this function into multiple smaller functions to make flake8 happy?
There was a problem hiding this comment.
Yeah, looks like that function was right at the limit for complexity and I crossed it :D Will refactor
Darktex
added a commit
to Darktex/opacus
that referenced
this pull request
Jan 24, 2023
Summary: Pull Request resolved: pytorch#556 This diff introduces gradient clipping schedulers that can be used to vary gradient clipping throughout training. Addresses pytorch#375 in OSS. Reviewed By: karthikprasad Differential Revision: D42644261 fbshipit-source-id: 91bedb4c3dd68f336917d16cec42f939ace02406
Darktex
force-pushed
the
export-D42644261
branch
from
ec71f8b to
28cfe8c
Compare
|
This pull request was exported from Phabricator. Differential Revision: D42644261 |
Darktex
added a commit
to Darktex/opacus
that referenced
this pull request
Jan 24, 2023
Summary: Pull Request resolved: pytorch#556 This diff introduces gradient clipping schedulers that can be used to vary gradient clipping throughout training. Addresses pytorch#375 in OSS. Reviewed By: karthikprasad Differential Revision: D42644261 fbshipit-source-id: 94f5fb756dce8ec576cce2ff003c2054eb926e27
|
This pull request was exported from Phabricator. Differential Revision: D42644261 |
Darktex
force-pushed
the
export-D42644261
branch
from
28cfe8c to
3412d26
Compare
Darktex
added a commit
to Darktex/opacus
that referenced
this pull request
Jan 24, 2023
Summary: Pull Request resolved: pytorch#556 This diff introduces gradient clipping schedulers that can be used to vary gradient clipping throughout training. Addresses pytorch#375 in OSS. Reviewed By: karthikprasad Differential Revision: D42644261 fbshipit-source-id: a571569785697b369c4c6f9709d35a5d3ff7e73c
|
This pull request was exported from Phabricator. Differential Revision: D42644261 |
Darktex
force-pushed
the
export-D42644261
branch
from
3412d26 to
5d3ed10
Compare
Summary: Pull Request resolved: pytorch#556 This diff introduces gradient clipping schedulers that can be used to vary gradient clipping throughout training. Addresses pytorch#375 in OSS. Reviewed By: karthikprasad Differential Revision: D42644261 fbshipit-source-id: 57c87ba5f0b012359761fa015d046edc5c13da88
|
This pull request was exported from Phabricator. Differential Revision: D42644261 |
Darktex
force-pushed
the
export-D42644261
branch
from
5d3ed10 to
8c8e540
Compare
|
This pull request has been merged in d888fd0. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This diff introduces gradient clipping schedulers that can be used to vary gradient clipping throughout training.
Addresses #375 in OSS.
Differential Revision: D42644261