Training a PonyXL LoRa advice needed

[warpathic]

I've been training on the free colab Trainer which have a simple beginner friendly interface. I wanted to train locally so I installed OneTrainer on my Legion 4090 16 GB laptop. So many settings and no Pony presets, so I loaded the SDXL preset and searched online for PonyXL settings. Couldn't find any, but found several SDXL advice, but my first few tries didn't give me the results I wanted. My dataset consists of 100 images and I train for 2500 steps. I used Prodigy with cosine, and I toggled DoRa on as I read it improves details. 32/32 Rank/alpha and text encoder training off. My previous colab trains were AdamW 8-bit, cosine restart 3, 8/4 R/A, text encoder trained.

Any advice would be greatly appreciated.

[carlo0000]

just load an existing checkpoint from civitai.com then i advice you too train your lora in high resolution i use 2000x2600
my picture are all in the 4096x something
you will not able to use that resolution but 1280x2000 1512x2000 even 2000x2000 work pretty good
some checkpoint there already trained at 1280 but still very poor if you train at 1280

use the default setting of sdxl lora, and train in weight data float32 + train data type float32 , output 32 also, the other stuff you can reduce to safe memory
then i use the adamw_adv with bias correction + factored optimizer (reduce vram) , i only have 11 of vram
learning rate 0.0005 but need lot more steps than 2500
don't disable text encoder embeding

the quality is good , only 1 big problem i have is that if you want to train an other character in the same lora (continue from backup), that is not working, it will completely forget the previous trained stuff

also after 2-3000 steps safe a file each 500 steps , so you don't need to load back a backup if you got to far

i test at high steps like 36 if it start to get to much trained then it's good, and instead of going back , i use that one but it need a lot less step 15-20 max or you take a previous saved one, but then you probably need more steps to get a good image