Unable to run Inference "coreml stable diffusion xl base original compiled"

[vincyb]

Error:

StableDiffusion/TextEncoder.swift:89: Fatal error: Unexpectedly found nil while unwrapping an Optional value
zsh: trace trap  swift run StableDiffusionSample "An Apple" --resource-pat

[atiorh]

Are you using --xl as an argument as described here?

[vincyb]

Are you using --xl as an argument as described here?

yes Here my command

swift run StableDiffusionSample "An Apple" --resource-path coreml-stable-diffusion-xl-base_original_compiled/Resources --output-path /Result --compute-units cpuAndGPU --xl

[atiorh]

Could you please share what ls -al coreml-stable-diffusion-xl-base_original_compiled/Resources prints?

[vincyb]

ls -al coreml-stable-diffusion-xl-base_original_compiled/Resources

total 2744
drwxr-xr-x  10 vincyb  staff     320 Aug  6 11:58 .
drwxr-xr-x   8 vincyb  staff     256 Aug  6 11:58 ..
-rw-r--r--@  1 vincyb  staff    8196 Aug  6 11:58 .DS_Store
drwxr-xr-x   7 vincyb  staff     224 Aug  6 08:50 TextEncoder.mlmodelc
drwxr-xr-x   7 vincyb  staff     224 Aug  6 08:50 TextEncoder2.mlmodelc
drwxr-xr-x   7 vincyb  staff     224 Aug  6 08:50 Unet.mlmodelc
drwxr-xr-x   7 vincyb  staff     224 Aug  6 08:50 VAEDecoder.mlmodelc
-rw-r--r--   1 vincyb  staff  524657 Aug  6 08:50 merges.txt
-rw-r--r--   1 vincyb  staff  862328 Aug  6 08:50 vocab.json

[ZelnickB]

Same issue here (albeit with a more intricate prompt). I ran the command twice:

Run 1

user@MacBook-Pro-M2-Max ml-stable-diffusion % swift run StableDiffusionSample "An enormous sad poodle walking through New York City" --resource-path /Users/user/sdxl/Resources --output-path ./tmp --compute-units cpuAndGPU --xl
Fetching https://github.com/apple/swift-argument-parser.git
Fetched https://github.com/apple/swift-argument-parser.git (1.37s)
Computing version for https://github.com/apple/swift-argument-parser.git
Computed https://github.com/apple/swift-argument-parser.git at 1.2.0 (1.89s)
Computed https://github.com/apple/swift-argument-parser.git at 1.2.0 (0.00s)
Creating working copy for https://github.com/apple/swift-argument-parser.git
Working copy of https://github.com/apple/swift-argument-parser.git resolved at 1.2.0
Building for debugging...
[72/72] Linking StableDiffusionSample
Build complete! (5.77s)
Loading resources and creating pipeline
(Note: This can take a while the first time using these resources)
Sampling ...
StableDiffusion/TextEncoder.swift:89: Fatal error: Unexpectedly found nil while unwrapping an Optional value
zsh: trace trap  swift run StableDiffusionSample  --resource-path  --output-path ./tmp   --xl

Run 2

user@MacBook-Pro-M2-Max ml-stable-diffusion $ swift run StableDiffusionSample "An enormous sad poodle walking through New York City" --resource-path /Users/user/sdxl/Resources --output-path /Users/user/tmp/ --compute-units cpuAndGPU --xl
Building for debugging...
Build complete! (0.14s)
Loading resources and creating pipeline
(Note: This can take a while the first time using these resources)
Sampling ...
StableDiffusion/TextEncoder.swift:89: Fatal error: Unexpectedly found nil while unwrapping an Optional value
zsh: trace trap  swift run StableDiffusionSample  --resource-path  --output-path   cpuAndGPU

[ryanto]

Seeing the same error:

> python -m python_coreml_stable_diffusion.torch2coreml --convert-unet --convert-vae-decoder --convert-text-encoder --xl-version --model-version stabilityai/stable-diffusion-xl-base-1.0 --bundle-resources-for-swift-cli --attention-implementation ORIGINAL -o ./mlpackages

> swift run StableDiffusionSample "Astronaut on Mars During sunset" --resource-path mlpackages/Resources --output-path images/ --compute-units cpuAndGPU --xl

Building for debugging...
Build complete! (0.14s)
Loading resources and creating pipeline
(Note: This can take a while the first time using these resources)
Sampling ...
StableDiffusion/TextEncoder.swift:89: Fatal error: Unexpectedly found nil while unwrapping an Optional value
fish: Job 1, 'swift run StableDiffusionSample…' terminated by signal SIGTRAP (Trace or breakpoint trap)

> ls -al mlpackages/Resources

total 2720
drwxr-xr-x   8 ryan  staff     256 Aug 10 23:49 ./
drwxr-xr-x  26 ryan  staff     832 Aug 10 23:49 ../
drwxr-xr-x   7 ryan  staff     224 Aug 10 23:49 TextEncoder.mlmodelc/
drwxr-xr-x   7 ryan  staff     224 Aug 10 23:49 TextEncoder2.mlmodelc/
drwxr-xr-x   7 ryan  staff     224 Aug 10 23:49 Unet.mlmodelc/
drwxr-xr-x   7 ryan  staff     224 Aug 10 23:49 VAEDecoder.mlmodelc/
-rw-r--r--   1 ryan  staff  524657 Aug 10 23:49 merges.txt
-rw-r--r--   1 ryan  staff  862328 Aug 10 23:49 vocab.json

I'm not sure if this matters, but I noticed that the StableDiffusionXLPipeline is only available on macOS 14.0. I'm on 13.5.

[atiorh]

@vincyb @ZelnickB @ryanto Thanks for the additional reports! Could you all please share your respective macOS versions ($ sw_vers gives the most detailed info) and Mac HW specs (chip and RAM)?

[ryanto]

> sw_vers
ProductName:		macOS
ProductVersion:		13.5
BuildVersion:		22G74

16-inch MBP 2021
Apple M1 Max
32 GB

[vincyb]

@vincyb @ZelnickB @ryanto Thanks for the additional reports! Could you all please share your respective macOS versions ($ sw_vers gives the most detailed info) and Mac HW specs (chip and RAM)?

> sw_vers
ProductName:		macOS
ProductVersion:		13.5
BuildVersion:		22G74

Model Name: MacBook Pro
Model Identifier: MacBookPro18,1
Chip: Apple M1 Pro
Total Number of Cores: 10 (8 performance and 2 efficiency)
Memory: 16 GB

[vincyb]

@vincyb @ZelnickB @ryanto Thanks for the additional reports! Could you all please share your respective macOS versions ($ sw_vers gives the most detailed info) and Mac HW specs (chip and RAM)?

Stable diffusion Automatic 1111, comfyui, Mochi Diffusion.app, Diffusers.app, Draw Things.app all works fine
Thanks

[ZelnickB]

@vincyb @ZelnickB @ryanto Thanks for the additional reports! Could you all please share your respective macOS versions ($ sw_vers gives the most detailed info) and Mac HW specs (chip and RAM)?

I'm running macOS version 13.5 (build 22G74) on an M2 Max with 96 GB RAM.

[atiorh]

Thanks all! M1 and newer is good enough for SDXL as far as HW goes but the SW support is tested with (and is made available for) macOS 14+ only due to major improvements to the underlying SW from 13.5.

[JurijsNazarovs]

@atiorh Does it mean that we have to wait until relaease of macos 14 to run xl model with apple acceleration? Is that for both, swift run and python_coreml_stable_diffusion.pipeline?

[atiorh]

@JurijsNazarovs Yes (or you may enroll in the Public Beta program). Note that the Python pipeline does not support SDXL, only the Swift pipeline does. I will add a clarifying note in the README section about this.