feature: diffusers-models

[kalle07]

Is it possible to convert pure image models, or is that not possible or too time-consuming?

like
https://huggingface.co/Qwen/Qwen-Image
or
https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct
or
https://huggingface.co/black-forest-labs/FLUX.1-dev

ok, qwen_vl seems to work but how i use
auto_round_mllm
in python code, or must i call directly the exe with parameters?

from auto_round import AutoRound
autoround = AutoRound(model, tokenizer, bits=bits, group_size=group_size, sym=sym, batch_size=4, low_gpu_mem_usage=True, disable_opt_rtn=True)

[kalle07]

i work on the QwenVL model 7B
https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct

while a "normal" LLM model 7B only need 10min to be ready,
this model with more or less same parameter after 1hour, it runs my GPU works 16GB overflow my VRAM and RAM 64GB full ... i think all as much to improve ;)

i know from joycaption-model 16bit(17GB) can be load instantly as 8bit in 30sec and after it runs for image recognition, need never more 6-7GB vram

cmd = [
str(exe_path),
"--model", str(model_dir),
"--output", str(output_dir),
"--bits", "4",
"--group_size", "128",
"--format", "auto_round",
"--disable_opt_rtn",
"--batch_size", "2",
"--low_gpu_mem_usage",
"--iters", "100",
"--nsample", "512",
"--seqlen", "2048",
]

2025-10-13 20:33:26 WARNING template.py L174: Unable to recognize qwen2_5_vl, using default template instead.
2025-10-13 20:33:41 INFO autoround.py L481: model.visual.blocks.0.mlp.gate_proj will not be quantized due to its shape not being divisible by 32, resulting in an exporting issue to autogptq
2025-10-13 20:33:41 INFO autoround.py L481: model.visual.blocks.0.mlp.up_proj will not be quantized due to its shape not being divisible by 32, resulting in an exporting issue to autogptq
2025-10-13 20:33:41 INFO autoround.py L481: model.visual.blocks.0.mlp.down_proj will not be quantized due to its shape not being divisible by 32, resulting in an exporting issue to autogptq
...
...

2025-10-13 20:54:32 INFO autoround.py L1294: start to cache block inputs
cache block inputs: 100%|██████████████████████████████████████████████████████████████████████| 512/512 [00:05<00:00, 97.83it/s]
2025-10-13 20:54:46 INFO autoround.py L1308: caching done
Quantizing model.language_model.layers.0: 0%| | 0/28 [00:00<?, ?it/s]2025-10-13 20:56:03 INFO autoround.py L2269: quantized 7/7 layers in the block, loss iter 0: 0.000777 -> iter 94: 0.000337
Quantizing model.language_model.layers.1: 4%|█▊ | 1/28 [01:48<48:51, 108.57s/it]

at least its runs with low parameters batch 2 and 100 iters ...

why i cant save the safetensors file in one big file?

ok, now i use the file, it is not refused but works bad maybe because...

2025-10-13 21:55:12.468 [Debug] [ComfyUI-0/STDERR] clip missing: ['model.layers.0.self_attn.q_proj.weight', 'model.layers.0.self_attn.k_proj.weight', 'model.layers.0.self_attn.v_proj.weight', 'model.layers.0.self_attn.o_proj.weight', 'model.layers.0.mlp.gate_proj.weight', 'model.layers.0.mlp.up_proj.weight', 'model.layers.0.mlp.down_proj.weight',...
like above

how i can fix that?

[wenhuach21]

the slow issue has been fixed in #871.

We currently support several VLM models and recently added support for Flux. Other diffusion models are not yet supported.

[kalle07]

ok, when will be the pip update available, so no need to clone the git repository?

[wenhuach21]

If no big issues are found during the pre-release test, we plan to release the new version this Friday.