Sd3 freeze x_block

[sdbds]

use
--num_last_block_to_freeze
to set freeze blocks num,reverse freeze.
(default none,you can set 0~240)

SD3 has 240 x_blocks, i test if freeze half of it will get better effect.

Comparison：
Before↓

After freeze 120↓

[kohya-ss]

Thank you for this! This seems to work like block_lr for SDXL. I will merge when I can confirm the SD3 training is fine :)

[sdbds]

Thank you for this! This seems to work like block_lr for SDXL. I will merge when I can confirm the SD3 training is fine :)

Yes, the functionality is very similar with block_lr.
After freezing the blocks, I only input the parameters of the blocks that need to be trained to the optimizer.
This can reduce additional overhead.

[FurkanGozukara]

how do you know which layers you should freeze or it is purely experimental?

[sdbds]

how do you know which layers you should freeze or it is purely experimental?

I received advice from SD3 trainer.

[FurkanGozukara]

how do you know which layers you should freeze or it is purely experimental?

I received advice from SD3 trainer.

makes sense. so what parameter they suggest ? like --num_last_block_to_freeze 80 ?

[sdbds]

how do you know which layers you should freeze or it is purely experimental?

I received advice from SD3 trainer.

makes sense. so what parameter they suggest ? like --num_last_block_to_freeze 80 ?

120~240，i think magic number between these.

[kohya-ss]

Thank you for this! Sorry it took so long.