Integration of `from_pretrained` and `from_single_file` #10208

suzukimain · 2024-12-13T09:55:32Z

What does this PR do?

This PR integrates the from_single_file function into from_pretrained

pip install -q git+https://github.com/suzukimain/diffusers.git@loading_method_integration

from diffusers import StableDiffusionPipeline

# Load a single file checkpoint
pipe = StableDiffusionPipeline.from_pretrained("https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5/blob/main/v1-5-pruned-emaonly.safetensors")

# Load a multi-folder Diffusers format checkpoint
pipe = StableDiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5")

Additionally, since AutoPipeline doesn't have a method to load single file checkpoints, I've adapted a part of the model_search functionality to enable this capability.

from diffusers import AutoPipelineForText2Image

# Load a single file checkpoint
pipe = AutoPipelineForText2Image.from_pretrained("https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5/blob/main/v1-5-pruned-emaonly.safetensors")

# Load a multi-folder Diffusers format checkpoint
pipe = AutoPipelineForText2Image.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5")

Edit:
Reflected changes

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline?
Did you read our philosophy doc (important for complex PRs)?
Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review?

@yiyixuxu ,@asomoza

suzukimain · 2024-12-14T03:56:03Z

This PR allows these issues to be corrected.
#6986 #7496 #7979

…loading_method_integration

suzukimain · 2024-12-17T03:05:48Z

Updated dictionary used when loading the single file checkpoint in AutoPipeline

suzukimain · 2024-12-19T09:29:03Z

Can anyone please review this PR?

…loading_method_integration

…e map for loading single file checkpoints.

suzukimain · 2024-12-23T00:52:12Z

Hi, Could you please review this PR if possible?
I hope to not have caused any inconvenience. Thank you for your cooperation.
cc: @hlky @DN6

bghira · 2025-01-03T16:10:00Z

cc @yiyixuxu @a-r-r-o-w

…tegration

…Integrate-load-method-into-from_pretrained

suzukimain · 2025-01-10T08:20:38Z

src/diffusers/loaders/single_file.py

-            local_files_only=local_files_only,
-            revision=revision,
-        )
+        pipeline_class = _get_pipeline_class(cls, class_name=cls.__name__, config=None)


If you don't pass the class_name when loading a single file checkpoint in DiffusionPipeline, it will result in a TypeError.

suzukimain · 2025-01-10T08:31:59Z

src/diffusers/pipelines/pipeline_utils.py

+        # Load the pipeline from a single file
+        elif load_method_name == "from_single_file":
+            # The arguments for the __init__ method of `DiffusionPipeline` are keyword arguments, so they cannot be loaded from `from_single_file`
+            if cls.__name__ == "DiffusionPipeline":
+                # import it here to avoid circular import
+                from .stable_diffusion.convert_from_ckpt import download_from_original_stable_diffusion_ckpt
+
+                checkpoint = load_single_file_checkpoint(
+                    pretrained_model_name_or_path,
+                    force_download=force_download,
+                    proxies=proxies,
+                    token=token,
+                    cache_dir=cache_dir,
+                    local_files_only=local_files_only,
+                    revision=revision,
+                )
+
+                return download_from_original_stable_diffusion_ckpt(checkpoint, **kwargs)
+            else:
+                return cls.from_single_file(pretrained_model_name_or_path, **kwargs)


I'm separating the process because executing DiffusionPipeline.from_single_file results in the following error:

ValueError: Pipeline <class 'diffusers.pipelines.pipeline_utils.DiffusionPipeline'> expected {'kwargs', 'args'}, but only set() were passed.

suzukimain · 2025-01-10T08:38:42Z

Hello, I have made corrections based on your advice.
cc @DN6 @yiyixuxu

…loading_method_integration

github-actions · 2025-02-03T15:03:19Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

…loading_method_integration

suzukimain · 2025-03-01T09:08:47Z

Hello, Could you please review this PR?
@yiyixuxu @DN6

yiyixuxu · 2025-03-04T00:27:59Z

hi @suzukimain

thanks for the PR, like @DN6 mentioned here #10208 (comment) but we are not 100% about if we are going to unify these two methods yet. We will not be able to merge PR at the moment.

sorry if we wasted your time. In the future, would you be abe to open an issue and start a discussion first?

suzukimain · 2025-03-05T07:25:06Z

I understand. I appreciate your kind response during your busy time.

suzukimain and others added 16 commits December 13, 2024 10:15

update

d8dc461

update

b239d4f

update

abf20e1

update

67b1f40

update

4132129

Added pipeline mappings for loading a single file checkpoint

ef77efb

update

349726f

fix

5e9d485

update

2475b86

Merge branch 'huggingface:main' into load_Method_Integration

e791d62

fix

4788002

fix

fdf5020

fix

b9449df

Fixed duplicate checkpoint loading

8c25f96

Merge branch 'main' into loading_method_integration

cb525df

Merge branch 'main' into loading_method_integration

f545353

suzukimain and others added 5 commits December 15, 2024 11:51

Merge branch 'main' into loading_method_integration

ef96367

Code formatting

891b11c

Merge branch 'main' of https://github.com/huggingface/diffusers into …

e072f90

…loading_method_integration

fix

7b22b18

Add keys and sort alphabetically

eb09cf0

suzukimain added 2 commits December 18, 2024 20:49

Merge branch 'main' into loading_method_integration

b9b62ea

Merge branch 'main' into loading_method_integration

ca764d4

suzukimain added 2 commits December 23, 2024 00:03

Merge branch 'main' of https://github.com/huggingface/diffusers into …

0874ab6

…loading_method_integration

Added flux-depth, flux-fill, and mochi-1-preview to the pipelin…

6488693

…e map for loading single file checkpoints.

suzukimain and others added 17 commits January 10, 2025 09:47

update

e9a48fb

update

0ec85e0

update

41f960f

update

8c10847

fix

6a79268

make quality

8e845c2

update

6b252aa

update

d10e3f1

update

ceae059

update

18fce79

update

a5604da

update

69ba240

update

d82b9a9

make quality

b0f18ae

Merge https://github.com/huggingface/diffusers into loading_method_in…

8e58877

…tegration

Merge branch 'main' of https://github.com/huggingface/diffusers into …

32ce3a6

…Integrate-load-method-into-from_pretrained

Integrate from_pretrained with from_single_file functionality

bf0f979

suzukimain commented Jan 10, 2025

View reviewed changes

Merge branch 'main' of https://github.com/huggingface/diffusers into …

0a7aa0e

…loading_method_integration

github-actions bot added the stale Issues that haven't received updates label Feb 3, 2025

suzukimain added 3 commits March 1, 2025 16:24

Merge branch 'main' of https://github.com/huggingface/diffusers into …

6b41c16

…loading_method_integration

Update pipeline mapping

b480850

Fix and make quality

2f7a245

github-actions bot removed the stale Issues that haven't received updates label Mar 1, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Integration of `from_pretrained` and `from_single_file` #10208

Integration of `from_pretrained` and `from_single_file` #10208

suzukimain commented Dec 13, 2024 •

edited

Loading

Uh oh!

suzukimain commented Dec 14, 2024

Uh oh!

suzukimain commented Dec 17, 2024

Uh oh!

suzukimain commented Dec 19, 2024

Uh oh!

suzukimain commented Dec 23, 2024 •

edited

Loading

Uh oh!

bghira commented Jan 3, 2025

Uh oh!

suzukimain Jan 10, 2025

Uh oh!

suzukimain Jan 10, 2025

Uh oh!

suzukimain commented Jan 10, 2025

Uh oh!

github-actions bot commented Feb 3, 2025

Uh oh!

suzukimain commented Mar 1, 2025

Uh oh!

yiyixuxu commented Mar 4, 2025

Uh oh!

suzukimain commented Mar 5, 2025

Uh oh!

Uh oh!

Integration of from_pretrained and from_single_file #10208

Are you sure you want to change the base?

Integration of from_pretrained and from_single_file #10208

Conversation

suzukimain commented Dec 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

Who can review?

Uh oh!

suzukimain commented Dec 14, 2024

Uh oh!

suzukimain commented Dec 17, 2024

Uh oh!

suzukimain commented Dec 19, 2024

Uh oh!

suzukimain commented Dec 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bghira commented Jan 3, 2025

Uh oh!

suzukimain Jan 10, 2025

Choose a reason for hiding this comment

Uh oh!

suzukimain Jan 10, 2025

Choose a reason for hiding this comment

Uh oh!

suzukimain commented Jan 10, 2025

Uh oh!

github-actions bot commented Feb 3, 2025

Uh oh!

suzukimain commented Mar 1, 2025

Uh oh!

yiyixuxu commented Mar 4, 2025

Uh oh!

suzukimain commented Mar 5, 2025

Uh oh!

Uh oh!

Integration of `from_pretrained` and `from_single_file` #10208

Integration of `from_pretrained` and `from_single_file` #10208

suzukimain commented Dec 13, 2024 •

edited

Loading

suzukimain commented Dec 23, 2024 •

edited

Loading