You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Please add gguf‑quantized support for "Ideogram-4" #452
changed the title [-]Please support qwen3vl_8b (Ideogram-4) gguf text encoder[/-][+]Please support qwen3vl_8b gguf (Ideogram-4) text encoder[/+]on Jun 4, 2026
I got a gguf working, but I think there is a problem with the mmproj file (which isn't needed for ideogram 4)
I'm using this one https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-GGUF With these names:
Qwen-3-VL-8B-Instruct-heretic.Q8_0.gguf
Qwen-3-VL-8B-Instruct-heretic.Q8_0.mmproj-f16.gguf
If I try to use comfyui's native "Generate text" node, vision doesn't work (but it does in llama.cpp). OTOH, that may be an issue with comfyui, as the vision on the model comfyui team uploaded to huggingface doesn't work either
For anyone arriving here trying to run Ideogram-4 via GGUF: the diffusion-model side, plus an MPS-specific dtype problem that breaks generation, are addressed in #454 (PRs #455 and #456). A public fork with both branches is available for immediate use: https://github.com/fxd0h/ComfyUI-GGUF
That covers the UNet. For the text encoder this issue is about, the official qwen3vl_8b_fp8_scaled safetensors (Comfy-Org) loads through the standard CLIPLoader and is enough to run Ideogram-4 end to end, so a GGUF text encoder is a convenience here rather than a hard requirement. The GGUF text-encoder path this issue requests remains open.
Verified end to end on Apple Silicon (M-series, MPS, torch 2.12).
Developed with AI assistance; verified on-device before posting.
@frankyifei Add to the very bottom of loader.py.
Although vision functionality won't work, it will at least properly display as BooguTEModel and run without issues. m8rr@a963418
# add qwen3vl part
if arch == "qwen2vl":
vsd = gguf_mmproj_loader(path)
sd.update(vsd)
if arch == "qwen3vl":
sd["model.visual.deepstack_merger_list.0.norm.weight"] = torch.zeros(4608)
sd["model.visual.merger.linear_fc2.weight"] = torch.zeros(4096)
changed the title [-]Please support qwen3vl_8b gguf (Ideogram-4) text encoder[/-][+]Please add gguf‑quantized support for Ideogram-4[/+]on Jun 24, 2026
From my repeated testing, a machine with 12GB VRAM and 16GB RAM can run the FP8 diffusion model provided by Comfy Org together with the Q4_K_M GGUF text‑encoder model. However, if the diffusion model is replaced with a GGUF‑quantized version, the image quality drops no matter what.
https://huggingface.co/rectangleworm/ideogram-4-gguf/blob/main/diffusion/cond/ideogram4-Q4_K.gguf
I used Q4K for this. The results are in order: INT8, FP8, Q4K. They’re slightly different, and the GGUF version feels a bit darker, but there isn't a huge difference overall. Q4K actually got the gloves right in PIC1. As for FP8 in PIC3, the text is a mess. I haven't used ideogram4 much, so I don't know well...
https://huggingface.co/rectangleworm/ideogram-4-gguf/blob/main/diffusion/cond/ideogram4-Q4_K.gguf I used Q4K for this. The results are in order: INT8, FP8, Q4K. They’re slightly different, and the GGUF version feels a bit darker, but there isn't a huge difference overall. Q4K actually got the gloves right in PIC1. As for FP8 in PIC3, the text is a mess. I haven't used ideogram4 much, so I don't know well...
INT8 / FP8 / Q4K
Sorry — my conclusion was referring specifically to the unconditional model. During testing I was always using the ideogram4_fp8_scaled main model and did not switch to any other quantized models. So I need to clarify that point.
So it's like this:
fp8 cond + fp8 uncond + GGUF TE = fine
fp8 cond + GGUF uncond + GGUF TE = problem
Is that right? I don't use 'uncond' at all, so I'm downloading it right now. I'll test it out later. But even if there's an issue, I probably won't be able to fix it...
So it's like this:
fp8 cond + fp8 uncond + GGUF TE = fine
fp8 cond + GGUF uncond + GGUF TE = problem
Is that right? I don't use 'uncond' at all, so I'm downloading it right now. I'll test it out later. But even if there's an issue, I probably won't be able to fix it...
Running the model itself doesn’t cause any issues, but the output image results vary depending on the quantization of the unconditional model. This difference is much greater compared to the different quantizations of the conditional model you tested above, both in terms of visual details and overall image quality. That’s why I recommend using the unquantized unconditional model.
Basically, Q4(leejet) is reddish and over-sharpened, while Q4K(rectangleworm) is slightly yellowish and blurry. As a test, I tried merging the Q4 and Q4K, and the result seems better.
We might just have to wait for a better GGUF release, though. Of course, it could also be a calculation issue with the gguf node.
Basically, Q4(leejet) is reddish and over-sharpened, while Q4K(rectangleworm) is slightly yellowish and blurry. As a test, I tried merging the Q4 and Q4K, and the result seems better. We might just have to wait for a better GGUF release, though. Of course, it could also be a calculation issue with the gguf node.
Thanks for providing such a detailed comparison test. I still think the FP8 uncond model from Comfy Org is the most recommended option. I don’t fully understand the underlying reason, but the quantization of the uncond model affects the results far more than the quantization of the main model.
Yeah, I agree. It feels like 'uncond' has a major impact. But since it’s way too slow, I think it might be better to just use 'cond' and handle it with something like seedvr or pid. I haven't actually tried it though.
Activity
[-]Please support qwen3vl_8b gguf text encoder[/-][+]Please support qwen3vl_8b (Ideogram-4) gguf text encoder[/+][-]Please support qwen3vl_8b (Ideogram-4) gguf text encoder[/-][+]Please support qwen3vl_8b gguf (Ideogram-4) text encoder[/+]juanml82 commentedon Jun 6, 2026
I got a gguf working, but I think there is a problem with the mmproj file (which isn't needed for ideogram 4)
I'm using this one
https://huggingface.co/mradermacher/Qwen-3-VL-8B-Instruct-heretic-GGUF With these names:
Qwen-3-VL-8B-Instruct-heretic.Q8_0.gguf
Qwen-3-VL-8B-Instruct-heretic.Q8_0.mmproj-f16.gguf
If I try to use comfyui's native "Generate text" node, vision doesn't work (but it does in llama.cpp). OTOH, that may be an issue with comfyui, as the vision on the model comfyui team uploaded to huggingface doesn't work either
fxd0h commentedon Jun 7, 2026
For anyone arriving here trying to run Ideogram-4 via GGUF: the diffusion-model side, plus an MPS-specific dtype problem that breaks generation, are addressed in #454 (PRs #455 and #456). A public fork with both branches is available for immediate use: https://github.com/fxd0h/ComfyUI-GGUF
That covers the UNet. For the text encoder this issue is about, the official
qwen3vl_8b_fp8_scaledsafetensors (Comfy-Org) loads through the standardCLIPLoaderand is enough to run Ideogram-4 end to end, so a GGUF text encoder is a convenience here rather than a hard requirement. The GGUF text-encoder path this issue requests remains open.Verified end to end on Apple Silicon (M-series, MPS, torch 2.12).
Developed with AI assistance; verified on-device before posting.
m8rr commentedon Jun 20, 2026
@frankyifei Add to the very bottom of loader.py.
Although vision functionality won't work, it will at least properly display as BooguTEModel and run without issues.
m8rr@a963418
m8rr commentedon Jun 20, 2026
m8rr@249624c
Add Qwen3-VL vision support; use with mmproj file.
Tested using the GGUF files below.
https://huggingface.co/mradermacher/Qwen3-VL-8B-Instruct-GGUF/blob/main/Qwen3-VL-8B-Instruct.mmproj-Q8_0.gguf
https://huggingface.co/unsloth/Qwen3-VL-8B-Instruct-GGUF/blob/main/Qwen3-VL-8B-Instruct-Q5_K_M.gguf
m8rr commentedon Jun 23, 2026
m8rr@c7fb703
Added Qwen3VL-4B
[-]Please support qwen3vl_8b gguf (Ideogram-4) text encoder[/-][+]Please add gguf‑quantized support for Ideogram-4[/+][-]Please add gguf‑quantized support for Ideogram-4[/-][+]Please add gguf‑quantized support for "Ideogram-4"[/+]makisekurisu-jp commentedon Jun 26, 2026
From my repeated testing, a machine with 12GB VRAM and 16GB RAM can run the FP8 diffusion model provided by Comfy Org together with the Q4_K_M GGUF text‑encoder model. However, if the diffusion model is replaced with a GGUF‑quantized version, the image quality drops no matter what.
m8rr commentedon Jun 26, 2026
https://huggingface.co/rectangleworm/ideogram-4-gguf/blob/main/diffusion/cond/ideogram4-Q4_K.gguf
I used Q4K for this. The results are in order: INT8, FP8, Q4K. They’re slightly different, and the GGUF version feels a bit darker, but there isn't a huge difference overall. Q4K actually got the gloves right in PIC1. As for FP8 in PIC3, the text is a mess. I haven't used ideogram4 much, so I don't know well...
INT8 / FP8 / Q4K
makisekurisu-jp commentedon Jun 26, 2026
Sorry — my conclusion was referring specifically to the unconditional model. During testing I was always using the
ideogram4_fp8_scaledmain model and did not switch to any other quantized models. So I need to clarify that point.m8rr commentedon Jun 26, 2026
So it's like this:
fp8 cond + fp8 uncond + GGUF TE = fine
fp8 cond + GGUF uncond + GGUF TE = problem
Is that right? I don't use 'uncond' at all, so I'm downloading it right now. I'll test it out later. But even if there's an issue, I probably won't be able to fix it...
makisekurisu-jp commentedon Jun 26, 2026
Running the model itself doesn’t cause any issues, but the output image results vary depending on the quantization of the unconditional model. This difference is much greater compared to the different quantizations of the conditional model you tested above, both in terms of visual details and overall image quality. That’s why I recommend using the unquantized unconditional model.
m8rr commentedon Jun 26, 2026
https://huggingface.co/leejet/ideogram-4-GGUF/blob/main/ideogram4_uncond-Q4_0.gguf
Use this one. rectangleworm's uncond is blurry and lacks detail, but leejet's is close to FP8. Looks like it's an issue with rectangleworm's GGUF.
leejet / rectangleworm / FP8
There definitely seems to be an issue with rectangleworm. leejet's image composition also changes more than the cond comparison.
m8rr commentedon Jun 27, 2026
Basically, Q4(leejet) is reddish and over-sharpened, while Q4K(rectangleworm) is slightly yellowish and blurry. As a test, I tried merging the Q4 and Q4K, and the result seems better.
We might just have to wait for a better GGUF release, though. Of course, it could also be a calculation issue with the gguf node.
makisekurisu-jp commentedon Jun 27, 2026
Thanks for providing such a detailed comparison test. I still think the FP8 uncond model from Comfy Org is the most recommended option. I don’t fully understand the underlying reason, but the quantization of the uncond model affects the results far more than the quantization of the main model.
m8rr commentedon Jun 27, 2026
Yeah, I agree. It feels like 'uncond' has a major impact. But since it’s way too slow, I think it might be better to just use 'cond' and handle it with something like seedvr or pid. I haven't actually tried it though.