Merge changes by Skquark · Pull Request #195 · Skquark/diffusers

Skquark · 2024-12-20T13:02:10Z

No description provided.

add rest of the lora loader mixins to the docs.

* Add `dynamic_shifting` to SD3 * calculate_shift * FlowMatchHeunDiscreteScheduler doesn't support mu * Inpaint/img2img

use_flow_sigmas copy

…10214) * Use non-human subject in StableDiffusion3ControlNetPipeline example * make style

Fix repaint scheduler

* torchao quantizer --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* attnprocessors * lora * make style * fix * fix * sana * typo

add contribution note for lawrence.

update

* add lora support for ltx * add tests * fix copied from comments * update --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

update Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * Update src/diffusers/quantizers/gguf/utils.py Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * update * update * update * update * update * update * update * update * update * update * Update docs/source/en/quantization/gguf.md Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * update * update * update * update --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * update * Update src/diffusers/models/transformers/transformer_mochi.py Co-authored-by: Aryan <aryan@huggingface.co> --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: Aryan <aryan@huggingface.co>

update

delete_adapters Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

…pipeline (#10251) fix: fix typo that cause error

* feat: lora support for SANA. * make fix-copies * rename test class. * attention_kwargs -> cross_attention_kwargs. * Revert "attention_kwargs -> cross_attention_kwargs." This reverts commit 23433bf. * exhaust 119 max line limit * sana lora fine-tuning script. * readme * add a note about the supported models. * Apply suggestions from code review Co-authored-by: Aryan <aryan@huggingface.co> * style * docs for attention_kwargs. * remove lora_scale from pag pipeline. * copy fix --------- Co-authored-by: Aryan <aryan@huggingface.co>

fix: licensing header.

* Use `torch` in `get_2d_rotary_pos_embed` * Add deprecation

fix: reamde -> readme

* Support pass kwargs to sd3 custom attention processor --------- Co-authored-by: hlky <hlky@hlky.ac> Co-authored-by: YiYi Xu <yixu310@gmail.com>

* flux_control_inpaint - failing test_flux_different_prompts * removing test_flux_different_prompts? * fix style * fix from PR comments * fix style * reducing guidance_scale in demo * Update src/diffusers/pipelines/flux/pipeline_flux_control_inpaint.py Co-authored-by: hlky <hlky@hlky.ac> * make * prepare_latents is not copied from * update docs * typos --------- Co-authored-by: affromero <ubuntu@ip-172-31-17-146.ec2.internal> Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: hlky <hlky@hlky.ac>

* update * Update docs/source/en/quantization/gguf.md Co-authored-by: Aryan <aryan@huggingface.co> --------- Co-authored-by: Aryan <aryan@huggingface.co>

update

…10288) update

…#10270) fix joint pos embedding device

rename integration test

remove nullop imports

Update README_sana.md to update the default model

* update * udpate * fix test

x-flux single-blocks lora load Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Check correct model type is passed to `from_pretrained` * Flax, skip scheduler * test_wrong_model * Fix for scheduler * Update tests/pipelines/test_pipelines.py Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * EnumMeta * Flax * scheduler in expected types * make * type object 'CLIPTokenizer' has no attribute '_PipelineFastTests__name' * support union * fix typing in kandinsky * make * add LCMScheduler * 'LCMScheduler' object has no attribute 'sigmas' * tests for wrong scheduler * make * update * warning * tests * Update src/diffusers/pipelines/pipeline_utils.py Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com> * import FlaxSchedulerMixin * skip scheduler --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* 1217 * 1217 * 1217 * update * reverse * add test * update test * make style * update * make style --------- Co-authored-by: Aryan <aryan@huggingface.co>

update

fix

…fusionPipeline` (#10181)

* Port UNet2DModel gradient checkpointing code from #6718. --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: Vincent Neemie <92559302+VincentNeemie@users.noreply.github.com> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com> Co-authored-by: hlky <hlky@hlky.ac>

* Added support for single IPAdapter on SD3.5 pipeline --------- Co-authored-by: hlky <hlky@hlky.ac> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> Co-authored-by: YiYi Xu <yixu310@gmail.com>

…10256) * add sharded + device_map check

contiguous tensors in resnet Co-authored-by: YiYi Xu <yixu310@gmail.com>

* update * add docs. --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

…d Fill (#10259) * lora expansion with dummy zeros. * updates * fix working 🥳 * working. * use torch.device meta for state dict expansion. * tests Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com> * fixes * fixes * switch to debug * fix * Apply suggestions from code review Co-authored-by: Aryan <aryan@huggingface.co> * fix stuff * docs --------- Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com> Co-authored-by: Aryan <aryan@huggingface.co>

add integration tests for lora expansion stuff in Flux.

* update * update * update * update * update --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

…)` (#10316) Update ltx_video.md to remove generator from `from_pretrained()`

sayakpaul and others added 30 commits December 16, 2024 08:50

[Docs] add rest of the lora loader mixins to the docs. (#10230)

ea893a9

add rest of the lora loader mixins to the docs.

Use t instead of timestep in _apply_perturbed_attention_guidance (

672bd49

#10243)

Add dynamic_shifting to SD3 (#10236)

a7d5052

* Add `dynamic_shifting` to SD3 * calculate_shift * FlowMatchHeunDiscreteScheduler doesn't support mu * Inpaint/img2img

Fix use_flow_sigmas (#10242)

3f421fe

use_flow_sigmas copy

Fix ControlNetUnion _callback_tensor_inputs (#10218)

87e8157

Use non-human subject in StableDiffusion3ControlNetPipeline example (#…

438bd60

…10214) * Use non-human subject in StableDiffusion3ControlNetPipeline example * make style

Add enable_vae_tiling to AllegroPipeline, fix example (#10212)

7186bb4

Fix checkpoint in CogView3PlusPipeline example (#10211)

e9a3911

Fix RePaint Scheduler (#10185)

2f023d7

Fix repaint scheduler

Add ControlNetUnion to AutoPipeline from_pretrained (#10219)

5ed761a

fix downsample bug in MidResTemporalBlock1D (#10250)

aafed3f

[core] TorchAO Quantizer (#10009)

9f00c61

* torchao quantizer --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

[docs] Add missing AttnProcessors (#10246)

7667cfc

* attnprocessors * lora * make style * fix * fix * sana * typo

[chore] add contribution note for lawrence. (#10253)

6fb94d5

add contribution note for lawrence.

Fix copied from comment in Mochi lora loader (#10255)

0d96a89

update

[LoRA] Support LTX Video (#10228)

ac86393

* add lora support for ltx * add tests * fix copied from comments * update --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

[docs] Clarify dtypes for Sana (#10248)

f9d5a93

update Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

[tests] Remove/rename unsupported quantization torchao type (#10263)

1524781

update

[docs] delete_adapters() (#10245)

2739241

delete_adapters Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

[Community Pipeline] Fix typo that cause error on regional prompting …

9c68c94

…pipeline (#10251) fix: fix typo that cause error

Add set_shift to FlowMatchEulerDiscreteScheduler (#10269)

ec1c7a7

[chore] fix: licensing headers in mochi and ltx (#10275)

ba6fd6e

fix: licensing header.

Use torch in get_2d_rotary_pos_embed (#10155)

0ac52d6

* Use `torch` in `get_2d_rotary_pos_embed` * Add deprecation

[chore] fix: reamde -> readme (#10276)

63cdf9c

fix: reamde -> readme

Make time_embed_dim of UNet2DModel changeable (#10262)

88b015d

Support pass kwargs to sd3 custom attention processor (#9818)

8eb73c8

* Support pass kwargs to sd3 custom attention processor --------- Co-authored-by: hlky <hlky@hlky.ac> Co-authored-by: YiYi Xu <yixu310@gmail.com>

hlky and others added 27 commits December 18, 2024 12:22

Fix sigma_last with use_flow_sigmas (#10267)

e222246

Fix Doc links in GGUF and Quantization overview docs (#10279)

b389f33

* update * Update docs/source/en/quantization/gguf.md Co-authored-by: Aryan <aryan@huggingface.co> --------- Co-authored-by: Aryan <aryan@huggingface.co>

Make zeroing prompt embeds for Mochi Pipeline configurable (#10284)

8304adc

update

[Single File] Add single file support for Flux Canny, Depth and Fill (#…

862a7d5

…10288) update

[tests] Fix broken cuda, nightly and lora tests on main for CogVideoX (…

c4c99c3

…#10270) fix joint pos embedding device

Rename Mochi integration test correctly (#10220)

f66bd32

rename integration test

[tests] remove nullop import checks from lora tests (#10273)

f35a387

remove nullop imports

[chore] Update README_sana.md to update the default model (#10285)

9c0e20d

Update README_sana.md to update the default model

Hunyuan VAE tiling fixes and transformer docs (#10295)

f781b8c

* update * udpate * fix test

Add Flux Control to AutoPipeline (#10292)

4450d26

Update lora_conversion_utils.py (#9980)

2f7a417

x-flux single-blocks lora load Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: YiYi Xu <yixu310@gmail.com>

[LoRA] Support HunyuanVideo (#10254)

1826a1e

* 1217 * 1217 * 1217 * update * reverse * add test * update test * make style * update * make style --------- Co-authored-by: Aryan <aryan@huggingface.co>

[Single File] Add single file support for Mochi Transformer (#10268)

9764f22

update

Allow Mochi Transformer to be split across multiple GPUs (#10300)

3ee9669

update

Fix local_files_only for checkpoints with shards (#10294)

074798b

Fix failing lora tests after HunyuanVideo lora (#10307)

d8825e7

fix

unet's sample_size attribute is to accept tuple(h, w) in `StableDif…

b756ec6

…fusionPipeline` (#10181)

[WIP] SD3.5 IP-Adapter Pipeline Integration (#9987)

3191248

* Added support for single IPAdapter on SD3.5 pipeline --------- Co-authored-by: hlky <hlky@hlky.ac> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> Co-authored-by: YiYi Xu <yixu310@gmail.com>

Add support for sharded models when TorchAO quantization is enabled (#…

41ba8c0

…10256) * add sharded + device_map check

Make tensors in ResNet contiguous for Hunyuan VAE (#10309)

151b74c

contiguous tensors in resnet Co-authored-by: YiYi Xu <yixu310@gmail.com>

[Single File] Add GGUF support for LTX (#10298)

dbc1d50

* update * add docs. --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

[Tests] add integration tests for lora expansion stuff in Flux. (#10318)

bf6eaa8

add integration tests for lora expansion stuff in Flux.

Mochi docs (#9934)

e12d610

* update * update * update * update * update --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

[Docs] Update ltx_video.md to remove generator from `from_pretrained(…

b64ca6c

…)` (#10316) Update ltx_video.md to remove generator from `from_pretrained()`

Skquark merged commit d18e6ae into Skquark:main Dec 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge changes#195

Merge changes#195
Skquark merged 57 commits intoSkquark:mainfrom
huggingface:main

Skquark commented Dec 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

16 participants

Conversation

Skquark commented Dec 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

16 participants