[Fix] remove setting lr for T5 text encoder when using prodigy in flux dreambooth lora script #9473
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| @@ -1372,7 +1372,6 @@ def load_model_hook(models, input_dir): | |||
| # changes the learning rate of text_encoder_parameters_one and text_encoder_parameters_two to be | |||
| # --learning_rate | |||
| params_to_optimize[1]["lr"] = args.learning_rate | |||
| params_to_optimize[2]["lr"] = args.learning_rate | |||
There was a problem hiding this comment.
I am not sure if this is the right fix because what happens when we train the text encoder, too? Cc @linoytsaban
There was a problem hiding this comment.
when we enable the option for training text encoders too, we only target the clip model and not the T5 for flux, hence this line was unnecessary from what I noticed
There was a problem hiding this comment.
hence also the reason for the indexerror^
There was a problem hiding this comment.
indeed for now we support full fine-tuning of the CLIP encoder only when --train_text_encoder is enabled, so simply removing this line for now should be good!
|
should be good to merge I believe, are we awaiting further changes? @sayakpaul |
|
@linoytsaban feel free to merge once the CI is green. |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
|
@linoytsaban okay to merge? |
|
@sayakpaul ok to merge, it's just the test that's failing for some reason |
…x dreambooth lora script (#9473) * fix: removed setting of text encoder lr for T5 as it's not being tuned * fix: removed setting of text encoder lr for T5 as it's not being tuned --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: Linoy Tsaban <57615435+linoytsaban@users.noreply.github.com>
What does this PR do?
Fixes #9464
Partially Fixes #9393 (comment)
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
cc @sayakpaul @linoytsaban