Fix RR-ClArC to freeze layers before CAV layer during fine-tuning #24
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
RR-ClArC's fine-tuning procedure was incorrectly optimizing all model parameters, when it should only update layers from the CAV layer onwards. According to the original paper and as spotted by @kasia284, the method should freeze all layers before the layer where gradient reprojection is applied during fine-tuning.
Solution
This PR modifies the
apply_model_correction()method inRRCLARCto:cav_layerin the model's execution orderrequires_grad=Falsefor all parameters belonging to modules beforecav_layercav_layeronwards)requires_gradstate after fine-tuning completesExample
For a model with layers
[layer0, layer1, layer2, layer3]andcav_layer='layer1':layer1,layer2, andlayer3are updated;layer0is frozenImplementation Details
The freezing logic properly handles:
layer0.conv.weightare correctly frozen whenlayer0should be frozenrequires_gradvalues are restored after training to avoid side effectsTesting
Comprehensive tests verify:
Impact
Fixes the issue reported by @kasia284 regarding incorrect layer freezing in RR-ClArC fine-tuning procedure.
Original prompt
Fixes #23
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.