Flux2 Dev+Klein support

[dxqb]

git fetch origin pull/1261/head:pr-1261
git switch pr-1261

update.sh or update.bat

• requires testing
• requires experimentation with quantization and training parameters - the preset is very preliminary
• might break Qwen-Image: this PR updates diffusers to a commit that includes Flux2-Klein - but that commit also fixes some bugs in their Qwen-Image pipeline. The workarounds for these bugs have to be removed in OneTrainer.
  They're still making changes on this part of the code: [Qwen] avoid creating attention masks when there is no padding huggingface/diffusers#12987
• there is an issue with masked training in this branch
• rename "Safetensors" output format
• test GGUF loading
• Support loading of Comfy LoRA files?
• more presets
• full tinetuning: Flux2: Tensor tuples can cause issues for checkpointing huggingface/diffusers#12777
• parameters for dynamic timestep shifting
• GGUF loading: Shape mismatch when loading a Flux2.Klein GGUF huggingface/diffusers#13001
• benchmark

see e50970f for code changes only for Klein

[dxqb]

masked training fixed @Calamdor

[dxqb]

removed Comfy LoRA format, because Comfy now supports diffusers format
you might have to update Comfy

[Paul315d]

I received the same screen yesterday after attempting to train Flux Klein. The model will not load even though it is available as a drop down selection. Please Fix.

[Calamdor]

It is a soft locked model, you need your hf key and to ask for access

[dxqb]

GGUF loading is broken for Flux2 because something in diffusers: huggingface/diffusers#13001
loading safetensors is NOT broken @Frostedbyte404 . You are trying to load a pre-quantized safetensors, which isn't supported for any model. OneTrainer quantizes during loading. If you want to use fp8, set fp8 but don't "Override" the transformer.

[Paul315d]

It would not load the Klein 9b non quantized base model downloaded straight from black forest labs. A hugging face token was included. I tried every setting possible This was while using Massed Compute and their Ubuntu 22.04 os. Maybe it works on windows?

Update: I think I found my problem.

When selecting Flux 1 from the drop down menu, the Base Model field is populated with the exact location of the model. When selecting Flux 2 from the drop down menu, no base model link is selected.

I copied the syntax for the Flux 1 and found the corresponding syntax on the hugging face webpage. Inserting this into the base model path and the model began working correctly. I have not trained with it yet, but this seems to fix my issue.

You may want to insert the correct base model path for the Flux 2 Klein drop down like the Flux 1 so others don't have this minor mistake.

[dxqb]

GGUF loading works now, using the workaround suggested in huggingface/diffusers#13001

[Paul315d]

Thank you for making the changes.

Don't know if it's a one trainer thing or a Klein thing, but the model does not train properly on .jpg data. Only Png. This is probably a good thing, but if you can't figure out why your loss rate is stuck very high, try training in .Png and watch a proper slope curve form.