Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ModulesToSaveWrapper __getattr__ #1238

Merged
merged 3 commits into from
Dec 13, 2023

Conversation

zhangsheng377
Copy link
Contributor

#1225

We are recently using peft=0.4.0+Megatron to run GPTModel. There is a code of the model self.language_model.output_layer.weight, which directly takes the weight of the output_layer, but the output_layer has already been modules_to_save, so it reported "ModulesToSaveWrapper has no 'weight' attribute" error.

So, I returns the weight of the currently active adapter or of the original module of the adapter is deactivated.

And __getattr__ will be called only when its own class does not have this attribute, so this should not destroy the original scene.

Copy link
Member

@BenjaminBossan BenjaminBossan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Thinking a bit more about it, I think the best course of action would be to actually implement this as a @property, which allows us to avoid fiddling with __getattr__.

This should still not affect stuff like state_dict, so would not break anything.

On top of that, would you be up to implement a unit test?

@zhangsheng377
Copy link
Contributor Author

Thanks for the PR. Thinking a bit more about it, I think the best course of action would be to actually implement this as a @property, which allows us to avoid fiddling with __getattr__.

This should still not affect stuff like state_dict, so would not break anything.

On top of that, would you be up to implement a unit test?

ok~

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link
Member

@BenjaminBossan BenjaminBossan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM.

Can one of you @pacman100 or @younesbelkada give this a quick look to ensure it doesn't break anything?

Copy link
Contributor

@pacman100 pacman100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @zhangsheng377 for adding the weight property when using ModulesToSaveWrapper! This makes sense and I don't see any issues with this approach.

Copy link
Contributor

@younesbelkada younesbelkada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! let's run integration tests after merging to see if this didn't break anything

@younesbelkada younesbelkada merged commit 0f1e909 into huggingface:main Dec 13, 2023
14 checks passed
@younesbelkada
Copy link
Contributor

BenjaminBossan pushed a commit to BenjaminBossan/peft that referenced this pull request Dec 13, 2023
* Update other.py

* Update other.py

* Update test_low_level_api.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants