[Advanced LoRA v1.5] fix: gradient unscaling problem

[sayakpaul]

What does this PR do?

Part of #6552.

@linoytsaban I couldn't find instructions to run this script in the README. Would appreciate some help in verifying it.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

@linoytsaban a gentle ping here.

[sayakpaul]

@linoytsaban sorry about the repeated pings here.

[linoytsaban]

sorry missed this!

[linoytsaban]

LGTM!

[sayakpaul]

Would you be able to test it as well? I don;t have the training commands for v1.5 to test.

[linoytsaban]

Apologies for the delay, tested with ⬇️ and it works good! we can ship :) Thanks a lot!

#!/usr/bin/env bash
!accelerate launch train_dreambooth_lora_sd15_advanced.py \
  --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" \
  --dataset_name="./linoy_face" \
  --instance_prompt="a TOK woman"\
  --validation_prompt="watercolor painting of a TOK woman with pink hair in New York"\
  --train_text_encoder_ti\
  --output_dir="linoy_lora_v1_sd15" \
  --caption_column="prompt" \
  --mixed_precision="fp16" \
  --resolution=1024 \
  --train_batch_size=3 \
  --repeats=1\
  --report_to="wandb"\
  --gradient_accumulation_steps=1 \
  --gradient_checkpointing \
  --learning_rate=1.0 \
  --text_encoder_lr=1.0 \
  --optimizer="prodigy"\
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --rank=32 \
  --max_train_steps=1000 \
  --checkpointing_steps=2000 \
  --seed="0" \
  --push_to_hub

[sayakpaul]

@linoytsaban thanks but we need to check if it can successfully resumed as well. Refer to #6514 (comment) for more details.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[sayakpaul]

Friendly ping here @linoytsaban

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.