Skip to content

Commit

Permalink
update timestepping to the new comfyui code
Browse files Browse the repository at this point in the history
  • Loading branch information
matt3o committed Apr 7, 2024
1 parent 5044599 commit 6b2ed91
Show file tree
Hide file tree
Showing 3 changed files with 54 additions and 5 deletions.
5 changes: 5 additions & 0 deletions CrossAttentionPatch.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,11 @@ def __call__(self, q, k, v, extra_options):
cond = cond_alt[t_idx]
del cond_alt

if cond_alt == 11 and (t_idx == 0 or t_idx == 3 or t_idx == 7):
weight = weight * 2
else:
weight = weight * .1

if weight == 0:
continue

Expand Down
12 changes: 7 additions & 5 deletions InstantID.py
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ def INPUT_TYPES(s):
FUNCTION = "apply_instantid"
CATEGORY = "InstantID"

def apply_instantid(self, instantid, insightface, control_net, image, model, positive, negative, start_at, end_at, weight=.8, ip_weight=None, cn_strength=None, noise=0.35, image_kps=None, mask=None, combine_embeds='average'):
def apply_instantid(self, instantid, insightface, control_net, image, model, positive, negative, start_at, end_at, weight=.8, ip_weight=None, cn_strength=None, noise=0.35, image_kps=None, mask=None, combine_embeds='average', layer=0):
self.dtype = torch.float16 if comfy.model_management.should_use_fp16() else torch.float32
self.device = comfy.model_management.get_torch_device()

Expand Down Expand Up @@ -322,8 +322,8 @@ def apply_instantid(self, instantid, insightface, control_net, image, model, pos

work_model = model.clone()

sigma_start = work_model.model.model_sampling.percent_to_sigma(start_at)
sigma_end = work_model.model.model_sampling.percent_to_sigma(end_at)
sigma_start = model.get_model_object("model_sampling").percent_to_sigma(start_at)
sigma_end = model.get_model_object("model_sampling").percent_to_sigma(end_at)

if mask is not None:
mask = mask.to(self.device)
Expand All @@ -337,6 +337,7 @@ def apply_instantid(self, instantid, insightface, control_net, image, model, pos
"mask": mask,
"sigma_start": sigma_start,
"sigma_end": sigma_end,
"cond_alt": layer,
}

if not is_sdxl:
Expand Down Expand Up @@ -416,6 +417,7 @@ def INPUT_TYPES(s):
"end_at": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001, }),
"noise": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.1, }),
"combine_embeds": (['average', 'norm average', 'concat'], {"default": 'average'}),
"layer": ("INT", {"default": 0, "min": 0, "max": 11, "step": 1, }),
},
"optional": {
"image_kps": ("IMAGE",),
Expand Down Expand Up @@ -494,8 +496,8 @@ def patch_attention(self, instantid, insightface, image, model, weight, start_at

work_model = model.clone()

sigma_start = work_model.model.model_sampling.percent_to_sigma(start_at)
sigma_end = work_model.model.model_sampling.percent_to_sigma(end_at)
sigma_start = model.get_model_object("model_sampling").percent_to_sigma(start_at)
sigma_end = model.get_model_object("model_sampling").percent_to_sigma(end_at)

if mask is not None:
mask = mask.to(self.device)
Expand Down
42 changes: 42 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,28 @@ Native [InstantID](https://github.com/InstantID/InstantID) support for [ComfyUI]

This extension differs from the many already available as it doesn't use *diffusers* but instead implements InstantID natively and it fully integrates with ComfyUI.

# Sponsorship

<div align="center">

**[:heart: Github Sponsor](https://github.com/sponsors/cubiq) | [:coin: Paypal](https://paypal.me/matt3o)**

</div>

If you like my work and wish to see updates and new features please consider sponsoring my projects.

- [ComfyUI IPAdapter Plus](https://github.com/cubiq/ComfyUI_IPAdapter_plus)
- [ComfyUI InstantID (Native)](https://github.com/cubiq/ComfyUI_InstantID)
- [ComfyUI Essentials](https://github.com/cubiq/ComfyUI_essentials)
- [ComfyUI FaceAnalysis](https://github.com/cubiq/ComfyUI_FaceAnalysis)
- [Comfy Dungeon](https://github.com/cubiq/Comfy_Dungeon)

Not to mention the documentation and videos tutorials. Check my **ComfyUI Advanced Understanding** videos on YouTube for example, [part 1](https://www.youtube.com/watch?v=_C7kR2TFIX0) and [part 2](https://www.youtube.com/watch?v=ijqXnW_9gzc)

The only way to keep the code open and free is by sponsoring its development. The more sponsorships the more time I can dedicate to my open source projects.

Please consider a [Github Sponsorship](https://github.com/sponsors/cubiq) or [PayPal donation](https://paypal.me/matt3o) (Matteo "matt3o" Spinelli). For sponsorships of $50+, let me know if you'd like to be mentioned in this readme file, you can find me on [Discord](https://latent.vision/discord) or _matt3o :snail: gmail.com_.

## Important updates

- **2024/02/27:** Added [noise injection](#noise-injection) in the negative embeds.
Expand Down Expand Up @@ -90,3 +112,23 @@ The noise helps reducing the "burn" effect.

It works very well with SDXL Turbo/Lighting. Best results with community's checkpoints.

## Current sponsors

It's only thanks to generous sponsors that **the whole community** can enjoy open and free software. Please join me in thanking the following companies and individuals!

### Gold sponsors

[![Kaiber.ai](https://f.latent.vision/imgs/kaiber.png)](https://kaiber.ai/)

### Esteemed individuals

- [Jack Gane](https://github.com/ganeJackS)
- [Nathan Shipley](https://www.nathanshipley.com/)

### One-time Extraordinaire

- [Eric Rollei](https://github.com/EricRollei)
- [francaleu](https://github.com/francaleu)
- [Neta.art](https://github.com/talesofai)
- [Samwise Wang](https://github.com/tzwm)
- _And all private sponsors, you know who you are!_

0 comments on commit 6b2ed91

Please sign in to comment.