Add Validation loss for LoRA training

[hinablue]

Base on #1165 @gesen2egee and #914 "Add validation loss" by @rockerBOO works.

And I only test on Lora training, DreamBooth and ControlNet training need more test.

[rockerBOO]

instead of taking the code and making a commit of your own you should follow the full commit process so the credit and copyright is property accounted for. It's very rude to all those before it. The PR you took from also did it to my original code and it's really annoying as I get no credit

[hinablue]

instead of taking the code and making a commit of your own you should follow the full commit process so the credit and copyright is property accounted for. It's very rude to all those before it. The PR you took from also did it to my original code and it's really annoying as I get no credit

Sorry for the late reply. I've included the original PR link with your information. If you think there is anything that needs to be corrected, please let me know. In fact, I have been following this PR for a long time, and I apologize for the trouble it has caused you.

[kohya-ss]

I apologize for not being able to merge the previous validation loss PR. I would like to take advantage of the current holiday season to merge.

@hinablue What is the update from the previous PR?

@rockerBOO Is this PR already okay with you now?

[hinablue]

I apologize for not being able to merge the previous validation loss PR. I would like to take advantage of the current holiday season to merge.

@hinablue What is the update from the previous PR?

@rockerBOO Is this PR already okay with you now?

I ref the #1165 @gesen2egee and rewrite it to work in the sd3 branch. Modified the calculation method of val loss after discussing with @gesen2egee (62164e5).

[hinablue]

Sorry to disturb everyone, I decided to close this PR. I will find another time to re-make the PR from @rockerBOO 's repo and send it to his sd3 branch.

[rockerBOO]

@hinablue i'm working to merge these together with the proper commits. Should be ready to review in a couple hours.

[rockerBOO]

@hinablue Also want to apologize as I was a little tough with my statement previously.

I hope my combination of PR's will be adequate and I appreciate the work you did to fix any conflicts and update the code for the latest. Made the process pretty straight forward for me. I will make sure you get the appropriate credit on the official release, as well as your commits are preserved.