Feature Requests

[KintCark]

Can u add the ability to save the qauntized models to storage that way if we want to use it again we don't have to keep reconverting every time we start an generation.

[grauho]

You should already be able to do that by setting the "-M, --mode" argument to "convert" to save the model as a quantized gguf file at the location specified with "-o, --output".

More information in the docs:
docs/quantization_and_gguf.md

[KintCark]

It auto saves to the main folder but I tried qauntized aura flow buy it killed when saving output

[grauho]

Interesting, please recompile with --config Debug, re-run with --verbose, and post the output

[KintCark]

Interesting, please recompile with --config Debug, re-run with --verbose, and post the output

I can't post output termux crashes immediately I got 7gb ram to spare but it loads the tensors but as soon as it trys to save output it crashes. Could someone else try converting aura flow 0.3 please I need q5 or q4 and q8

[grauho]

I'm not familiar with aura flow but I wonder if it's based on a model type that sdcpp doesn't currently support. You could pipe the output to a file to avoid losing it when termux crashes, eg: "./sd etc etc &> foo.txt" to pipe both stdout and stderr to a file.

[KintCark]

I'm not familiar with aura flow but I wonder if it's based on a model type that sdcpp doesn't currently support. You could pipe the output to a file to avoid losing it when termux crashes, eg: "./sd etc etc &> foo.txt" to pipe both stdout and stderr to a file.

You're right that has to be what it is it's not supported so it don't work so I can quantize sd3 I bet. Will flux and aura flow be added soon?

[KintCark]

Gguf used less memory I can run flux q8_0 and t5xxlfp16 in comfyui on my phone so using gguf is better than safetensors

[KintCark]

Sd3 uses flow so how come flux and aura not auto support?

[grauho]

Sd3 uses flow so how come flux and aura not auto support?

From my understanding of the similarities between Flux and SD3 is that while they share some components, they are not identical in their architecture. So, that's why despite SD3 being available in sdcpp there is no "auto support" for Flux.