Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline reduceMemory = true Crash on iOS #219

Open
TimYao18 opened this issue Jul 28, 2023 · 4 comments
Open

Pipeline reduceMemory = true Crash on iOS #219

TimYao18 opened this issue Jul 28, 2023 · 4 comments

Comments

@TimYao18
Copy link

TimYao18 commented Jul 28, 2023

I test the StableDiffusionPipeline API with Diffusers app on both macOS and iOS and try to let users select their own custom models into the app.

I put custom Stable-Diffusion-v1-5_split-einsum_cn models into the path below and set it as the url: StableDiffusionPipeline(resourcesAt: url)

macOS:
file:///Users/kyd6/MochiDiffusion/models/stable-diffusion-1-5-split_einsum_cn/

iOS:
// Custom model put under "Download" folder
file:///private/var/mobile/Containers/Shared/AppGroup/C73F1304-A597-4223-9538-0EBF0AF8800F/File%20Provider%20Storage/Downloads/Stable-Diffusion-v1-5_split-einsum_cn/

// Default Hub Downloaded model put under Application folder
file:///var/mobile/Containers/Data/Application/8C1DE5AA-76FC-4037-87DD-FCDE210BA66E/Library/Application%20Support/hf-diffusion-models/coreml-stable-diffusion-2-1-base-palettized_split_einsum_v2_compiled/

On macOS, it works well and has no problem;
On iOS:

  • it will crash at TextEncoder when generating images using Download folder
  • It works find when using hub downloaded models in application folder.

The iOS throw Error below:

tableDiffusion/TextEncoder.swift:93: Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=com.apple.CoreML Code=0 "Unable to load model: file:///private/var/mobile/Containers/Shared/AppGroup/C73F1304-A597-4223-9538-0EBF0AF8800F/File%20Provider%20Storage/Downloads/Stable-Diffusion-v1-5_split-einsum_cn/TextEncoder.mlmodelc/. Compile the model with Xcode or MLModel.compileModel(at:). " UserInfo={NSLocalizedDescription=Unable to load model: file:///private/var/mobile/Containers/Shared/AppGroup/C73F1304-A597-4223-9538-0EBF0AF8800F/File%20Provider%20Storage/Downloads/Stable-Diffusion-v1-5_split-einsum_cn/TextEncoder.mlmodelc/. Compile the model with Xcode or MLModel.compileModel(at:). , NSUnderlyingError=0x281a754d0 {Error Domain=com.apple.CoreML Code=3 "Failed to open file: /private/var/mobile/Containers/Shared/AppGroup/C73F1304-A597-4223-9538-0EBF0AF8800F/File Provider Storage/Downloads/Stable-Diffusion-v1-5_split-einsum_cn/TextEncoder.mlmodelc/coremldata.bin. It is not a valid .mlmodelc file. : unspecified iostream_category error" UserInfo={NSLocalizedDescription=Failed to open file: /private/var/mobile/Containers/Shared/AppGroup/C73F1304-A597-4223-9538-0EBF0AF8800F/File Provider Storage/Downloads/Stable-Diffusion-v1-5_split-einsum_cn/TextEncoder.mlmodelc/coremldata.bin. It is not a valid .mlmodelc file. : unspecified iostream_category error}}}

I don't know why it uses a folder located inside mlmodelc file in iOS, but when it comes to macOS it is OK.

How can I fix this issue?

@TimYao18
Copy link
Author

I found if StableDiffusionPipeline(..., reduceMemory: true) will make this happen.
Set reduceMemory to false, this will not happen.

But the app might be crashed due to out of memory.

@TimYao18 TimYao18 closed this as completed Aug 1, 2023
@TimYao18
Copy link
Author

TimYao18 commented Aug 1, 2023

In Image Generation with Swift:
On iOS, the reduceMemory option should be set to false when constructing StableDiffusionPipeline

I think there might be something wrong?

@TimYao18 TimYao18 reopened this Aug 1, 2023
@TimYao18
Copy link
Author

Close issue due to not the same question about title.

@TimYao18 TimYao18 changed the title Using Custom model on iOS will crash at model path Pipeline reduceMemory = true Crash on iOS Aug 16, 2023
@TimYao18
Copy link
Author

Change the title and reopen this issue.

@TimYao18 TimYao18 reopened this Aug 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant