-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
During my training, I accidently found if I use the default shift
timestep sampling method, even after thousands of steps, the model still learned nothing, the sample images are identical. Which is very strange. Because when using nextdit_shift
it only needs ~200 steps to show the noticeable changes in sample images
Lumina uses t=0 as the noise and t=1 as the image. Which is reversed, unlike other models (t=1 as the noise and t=0 as image).
I found this causes some funcs not working as expected, since they expects the usual timesteps.
In
sd-scripts/library/lumina_train_util.py
Line 806 in f5d44fd
def get_noisy_model_input_and_timesteps( |
nextdit_shift
reversed the timesteps in time_shift()
. Other timestep sampling methods don't work properly. They still shift toward t=1.
--min_snr_gamma
is also affected, maybe more.
I also found that diffusers still uses the usual timesteps in its api, and only reverse it before passing to the network.
Maybe we should do this as well?
sd-scripts/lumina_train_network.py
Line 271 in f5d44fd
t=timesteps / 1000, # timesteps需要除以1000来匹配模型预期 |