Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove _module prefix from GradSampleModule parameter and buffer names #435

Open
karthikprasad opened this issue May 31, 2022 · 1 comment
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@karthikprasad
Copy link
Contributor

🚀 Feature

Motivation

See discussion in #429 for full context. Relevant snippets form that discussion below:

In my experience "_module" prefix is always an annoyance and never helps.
In particular, getting rid of the prefix makes it much easier to load weights from publicly pre-trained models (currently you have to do it strictly before you wrap a model with GSM).
...today you have to be careful and load_state_dict before you wrap the model with GradSampleModule. With same state_dicts you don't have to worry about it.

@karthikprasad karthikprasad added the enhancement New feature or request label May 31, 2022
@karthikprasad karthikprasad added this to the 1.3.0 milestone May 31, 2022
@pierrestock
Copy link
Contributor

pierrestock commented May 31, 2022

Also loosely related to this discussion.

@ffuuugor ffuuugor self-assigned this Jun 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants