SDXL LoRA training - 8GB VRAM

[gabrielottj]

Guys,

I have a RTX 4060 Ti 8GB, and I would like to know if it's possible to train a SDXL LoRA with only 8GB of VRAM, with the Lora > Training > Parameters > Basic > Max Resolution set to 1024x1024.

I've watched this video: https://www.youtube.com/watch?v=ovuO8bT9Nzw and now I have a basic notion to start the training, I would just need the help regarding VRAM.

Any help is much appreciated.

Thanks

[5KilosOfCheese]

It is, quite easy be default. There are full settings tutorials for this*. Lowest I have seen people do is 6gb, but some have managed to pull 4gb (by having lots of ram to fall back on). However you don't need to worry too much, since with Nvidia you can easily use some of the RAM also, the system works. There is also the dynamo backends which help with this. Just give it a try. I run 4060 TI 18gb and depending on what I am doing, I can be low as 6 gb and high as 15 gb. I run fp8 training by default regardless.

Also you don't need high resolutions images to train with. My dataset is a mess of all sort of dimensions for a reason, it helps with the training. My smallest dimension can be low as 384. But generally I don't go above 1024 because I haven't really found any actual benefit for it in my training runs - just slows the process down.

*Keep in mind that all settings are just suggestion. There are no optimal settings for training, every thing you want to do will require some adjusting along with trial and error to get the best output. I don't really do LoRAs to generate with or publish, I just enjoy the puzzle of trying to get a thing to happen.

Also don't be scared if the training overflows to use RAM side, most of the time it doesn't affect speeds at all and even if it does the difference not really anything to worry about. When you are training with your own setup, it calls for certain amount of patience to begin with. I run training when I am at work, or I do other things.

E: Just to add... Just try stuff. Look guides only enough to get started and then just try stuff. Make notes on what works and doesn't work. Just... Have fun with the process. You will very quickly find things which work, and things that save Vram and wasn't mentioned in some guide. Go forth and have fun!

[gabrielottj]

Thank you for your very generous and kind reply, brother. Cheers!

[Xephier102]

I run 4060 TI 18gb

I'd like to know how you managed to attain that, lol. Mine only has 16gb..

generally I don't go above 1024 because I haven't really found any actual benefit for it in my training runs - just slows the process down.

Well, details are a good thing. Especially when training a dataset with people. Cuz SDXL does OK with close-up, but the people in the background (even if only by 10 feet or so) look like some form of inbred abominations. Also, below a certain res, it's difficult to get stuff like eyelashes and individual hairs to display correctly, and that's upward in the range of 2-3k+. And while that may just be splitting hairs, so to speak, lol, there's other considerations like the vast detail and depth of the human iris.

Plus, not every image is going to fit in a literal box shape. Honestly, I'm surprised there aren't more gripes about the 1024x1024 standard. It's almost like SD was training the model using CRT screens or something.. Heck, even those were 640x480. Nothing is box shaped in the PC/screen world.

don't be scared if the training overflows to use RAM side

Yea, lol. Cuz the alternative is an OOM crash of the program.

That said, I've got a 4060TI 16gb, and it still took me 11 hrs to train one epoch of an SDXL Lora. Mind you, I was using 300 images, varying in size from 256x256 to 3k x 4k, and just allowing onetrainer to 'bucket' the images in training. I'm putting more thought into my next run by junking any images below a certain size/quality, and resizing before training, to a box res of 1256x1256 or 1512x1512, while padding the space for images that aren't box shaped (IE most of them). I'm currently down to 125 images. Combine that with a little more hands-on captioning, and the second run of it should go much better.

My dataset is a mess of all sort of dimensions for a reason, it helps with the training. My smallest dimension can be low as 384.

In my limited experience (and much more research than experience thus far), I don't think that's great advice, if I'm being honest. For one, if your training program resizes images during training, it can increase training times. But, additionally, the more detail that the training has to work with, the better the Lora will be produced. Cuz, for example, the Lora model will understand that eyelashes are individual hairs that grow outward from the upper/lower borders of the eye; whereas if it was trained on a mix of images with some as low as 384 res, it may know that the above description is 'eyelashes', but it will also think that spiky clumped together smudges in the general region of the eye, are 'eyelashes'.