Repository navigation
Properly handle parent modules w/ parameters in BaseFinetuning callback - #7931
Merged
awaelchli merged 7 commits intoJun 14, 2021
Merged
Conversation
speediedan
marked this pull request as ready for review
June 10, 2021 22:04
speediedan
requested review from
Borda,
SeanNaren,
awaelchli,
carmocca,
justusschock,
kaushikb11,
tchaton and
williamFalcon
as code owners
June 10, 2021 22:04
Codecov Report
@@ Coverage Diff @@
## master #7931 +/- ##
=======================================
- Coverage 92% 87% -5%
=======================================
Files 203 203
Lines 13127 13127
=======================================
- Hits 12013 11395 -618
- Misses 1114 1732 +618 |
carmocca
reviewed
Jun 11, 2021
carmocca
left a comment
Contributor
There was a problem hiding this comment.
Nice catch!
Remember to update the CHANGELOG.md file
BaseFinetuning callback
speediedan
force-pushed
the
bugfix/7930_parent_module_w_param
branch
from
June 11, 2021 19:09
e477e51 to
d36525d
Compare
added 2 commits
June 11, 2021 14:49
…was a superset, updated changelog
speediedan
force-pushed
the
bugfix/7930_parent_module_w_param
branch
from
June 11, 2021 21:50
d36525d to
8b8a8cf
Compare
carmocca
approved these changes
Jun 12, 2021
awaelchli
approved these changes
Jun 13, 2021
kaushikb11
approved these changes
Jun 13, 2021
kaushikb11
enabled auto-merge (squash)
June 13, 2021 04:33
carmocca
disabled auto-merge
June 13, 2021 15:41
carmocca
reviewed
Jun 13, 2021
tchaton
approved these changes
Jun 14, 2021
tchaton
left a comment
Contributor
There was a problem hiding this comment.
Good catch ! Thanks for using the BaseFinetuning Callback and catching this bug.
justusschock
approved these changes
Jun 14, 2021
awaelchli
enabled auto-merge (squash)
June 14, 2021 13:59
Closed
carmocca
added a commit
that referenced
this pull request
Jun 15, 2021
…back (#7931) Co-authored-by: Daniel Dale <dan@distributedinsight.com> Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com> Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
carmocca
added a commit
that referenced
this pull request
Jun 15, 2021
…back (#7931) Co-authored-by: Daniel Dale <dan@distributedinsight.com> Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com> Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
lexierule
pushed a commit
that referenced
this pull request
Jun 17, 2021
…back (#7931) Co-authored-by: Daniel Dale <dan@distributedinsight.com> Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com> Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #7930
This PR fixes issue #7930 and provides relevant test coverage: test_finetuning_callback.py::test_parent_module_w_param_model
Models w/ parent modules that directly contained parameters themselves (as opposed to parameters exclusively in their constituent submodules) had their parameters ignored by the BaseFinetuning callback. The primary issue was addressed with an additional filtering condition in BaseFinetuning.flatten_modules(). To avoid passing duplicate parameters in methods that depended upon BaseFinetuning.flatten_modules, I also added recurse=False to the parameter loops in BaseFinetuning.[freeze, make_trainable, filter_params]. The new test I've added could potentially be consolidated with /tests/callbacks/test_finetuning_callback.py::test_deep_nested_model but I at least initially have broken the test out separately for clarity. As a practical example of the issue, I initially encountered this bug when using BaseFinetuning w/ deberta, specifically, the DisentangledSelfAttention parent module
I've updated the docstring of flatten_modules() to reflect the changes. All test_finetuning_callback.py tests are passing on my end.
PR review
Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:
Did you have fun?
Make sure you had fun coding 🙃
YES! This is my first contribution to PL but I'm hoping to be more involved in the future. Thanks for the awesome product/work!