SDXL model load error: All available backends failed to load the model... are you sure it has an archtecture ID set properly?

[xmicro]

Expected Behavior

Model to load correctly without any reported errors.

Actual Behavior

When I try to load SDXL models into Swarm, I get the following error:

[Error] [BackendHandler] Backend request #[1,2,3,etc.] failed: All available backends failed to load the model '/swarm/SwarmUI/Models/diffusion_models/[model name].safetensors'.

Possible reason: Model loader for [model name].safetensors didn't work - are you sure it has an architecture ID set properly? (Currently set to: 'stable-diffusion-xl-v1-base')

The model does not load.

Steps to Reproduce

Either select an SDXL model and try to generate with it, or click hamburger menu and select "Load Now" on an SDXL model.

Debug Logs

[Warning] [BackendHandler] backend #0 failed to load model [model name]
[Warning] [BackendHandler] All backends failed to load the model '/swarm/SwarmUI/Models/diffusion_models/[model name].safetensors'! Cannot generate anything.
[Error] [BackendHandler] Backend request #[1,2,3,etc.] failed: All available backends failed to load the model '/swarm/SwarmUI/Models/diffusion_models/[model name].safetensors'.

Possible reason: Model loader for [model name].safetensors didn't work - are you sure it has an architecture ID set properly? (Currently set to: 'stable-diffusion-xl-v1-base')

Other

I can use Flux models just fine for instance; this issue only appears to pop up with SDXL models. I've tried researching this problem online, but it seems like an extremely rare bug becasue there's very few reports of it, and even where I can find it mentioned, no relevant fix is listed. I also tried changing the architecture ID metadata to all the SDXL 1.0 variants, but this didn't work. VAE selection between automatic, none, or an actual VAE does not influence the error either.

[mcmonkey4eva]

Someone coincidentally asked the same on the discord earlier today, but didn't censor all the relevant debug data -- their issue was they put the file in diffusion_models, when SDXL models need to be in Stable-Diffusion

If your issue is different, please post full logs

[xmicro]

Someone coincidentally asked the same on the discord earlier today, but didn't censor all the relevant debug data

Lmao what? I'm not "censoring" the debug data; I was writing it for a generic model since the model itself is not relevant for the issue and for other people trying to search this issue that don't have the same exact model. The number increases from 1 to n, which is why I expressed it like that with a series. Part of the directory was cut off (now fixed). What's written is the entirety of the debug log for any given attempt, and there's nothing before or after this. Anything else is just the logs when I start the program without any errors.

their issue was they put the file in diffusion_models, when SDXL models need to be in Stable-Diffusion
If your issue is different, please post full logs

This solved the issue. I'm not sure why this is required, as I've never had any issue loading models from diffusion_models before instead of Stable-diffusion with other programs, which work seamlessly regardless of which directory I use. I'd still consider this an unfixed bug in the program.

To anyone else experiencing this issue, make sure you use Stable_diffusion instead of diffusion_models. This is still a bug, because flux models for example work in diffusion_models, but SDXL ones don't...

[xmicro]

@mcmonkey4eva Can't reopen this issue for some reason, but you should since the behavior itself has not been fixed. (And if you're going to say that there is no issue and this is intended, then flux models working in diffusion_models is asymmetric. It makes no sense why both directories don't work though outside of this just being a bug.)

ISSUE REMAINS UNSOLVED

[mcmonkey4eva]

Part of the directory was cut off (now fixed).
Yeah you had initially cut off the information that showed the cause of the error.

As a broad rule: if you knew what part of the logs was the source of the issue, you'd be fixing the issue yourself. Because you don't know what the root of the issue is, you should not cut off any log data.

Anything else is just the logs when I start the program without any errors.
Which matters! Sometimes the error is related to the fact that you're running a Mac, or an AMD graphics card, or whatever, and that's up near the top of the logs.
Sometimes the error is you're running an outdated install, and that's near the top of the logs.
Sometimes the error only happens if you ran x y z things beforehand, and those are up in the log above where the part that things went wrong at.
Sometimes the error sources from that one little random warning early in the logs you've ignored because it hasn't broken anything before.

Thus, again: never cut the logs. Just hit the pastebin button on the Server->Logs tab and give the link it generates. If you have eg previous gens you don't want to be showing the prompts for or whatever like that privacy concerns, just restart Swarm, replicate just the bug, and send the log pastebin after, that way nothing but what's relevant is included.

---

It makes no sense why both directories don't work though outside of this just being a bug.
There is an actual reason. I don't love it, I wish they weren't separated, but you're going to have to argue that case to https://github.com/comfyanonymous/ComfyUI as that split requirement is enforced by the comfy backend, not by Swarm.

The technical reason is that the Stable-Diffusion folder ("checkpoints" as Comfy calls it normally) is for all-in-one models. SDXL for example has the VAE and TextEncoders included in the one file.
"diffusion_models" is for backbone-only files, ones where the VAE and TextEnc are loaded separately.
Comfy treats them as different folders, and uses different nodes to load them.
Comfy generally provides all-in-one copies of models, for example here's a Comfy Org official Flux1-Dev all-in-one: https://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensors
But it's increasingly common to distribute backbone-only models, since approx the SD3/Flux era, wherein textencoder used is often the giant 5/10 gig T5-XXL model, so it doesn't really make sense for storage space to combo that data as much anymore (vs for SDXL, the model is ~6 gigs and adding the vae/textencs only brings it up to ~7 gigs, so everyone leaves them in).

[xmicro]

It makes no sense why both directories don't work though outside of this just being a bug. There is an actual reason. I don't love it, I wish they weren't separated, but you're going to have to argue that case to https://github.com/comfyanonymous/ComfyUI as that split requirement is enforced by the comfy backend, not by Swarm.

The technical reason is that the Stable-Diffusion folder ("checkpoints" as Comfy calls it normally) is for all-in-one models. SDXL for example has the VAE and TextEncoders included in the one file. "diffusion_models" is for backbone-only files, ones where the VAE and TextEnc are loaded separately. Comfy treats them as different folders, and uses different nodes to load them. Comfy generally provides all-in-one copies of models, for example here's a Comfy Org official Flux1-Dev all-in-one: https://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensors But it's increasingly common to distribute backbone-only models, since approx the SD3/Flux era, wherein textencoder used is often the giant 5/10 gig T5-XXL model, so it doesn't really make sense for storage space to combo that data as much anymore (vs for SDXL, the model is ~6 gigs and adding the vae/textencs only brings it up to ~7 gigs, so everyone leaves them in).

Ah I see now. Yeah that is kind of dumb and arbitrary.
For Swarm's end for now, maybe a helpful alternative to Possible reason: Model loader for [model name].safetensors didn't work - are you sure it has an architecture ID set properly? (Currently set to: 'stable-diffusion-xl-v1-base') could be Possible reason: Model loader for [model name].safetensors didn't work - ensure that models with VAEs and text encoders built in like SDXL are in the "Stable-diffusion" directory, whereas models that require separate VAEs and text encoders like Flux are in the "diffusion_models" directory or something like that?