Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optional noise_seed to make augmentation deterministic #3275

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
+12 −4
Open

Add optional noise_seed to make augmentation deterministic #3275

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
43cf3f6
Add optional noise_seed to make augmentation deterministic
harelc Apr 17, 2024
2d3c1d6
Move to using generator for the noise, making seed required
harelc Apr 18, 2024
4b56c32
Make noise_seed optional after all
harelc Apr 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Make noise_seed optional after all
  • Loading branch information
@harelc
harelc committed Apr 21, 2024
commit 4b56c3278f35ad8cd0b69401d0c1fbfea786ee9d
12 changes: 8 additions & 4 deletions comfy_extras/nodes_video_model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,9 @@ def INPUT_TYPES(s):
"motion_bucket_id": ("INT", {"default": 127, "min": 1, "max": 1023}),
"fps": ("INT", {"default": 6, "min": 1, "max": 1024}),
"augmentation_level": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 10.0, "step": 0.01}),
"noise_seed": ("INT", {"default": 0, "min": 0, "max": 2**32-1}),
}
},
"optional": { "noise_seed": ("INT", {"default": 0, "min": 0, "max": 2**32-1})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the seed max value as seen in eg the KSampler node should be set as 0xffffffffffffffff

}
}
RETURN_TYPES = ("CONDITIONING", "CONDITIONING", "LATENT")
RETURN_NAMES = ("positive", "negative", "latent")
Expand All @@ -45,13 +46,16 @@ def INPUT_TYPES(s):
CATEGORY = "conditioning/video_models"

def encode(self, clip_vision, init_image, vae, width, height, video_frames, motion_bucket_id,
fps, augmentation_level, noise_seed):
fps, augmentation_level, noise_seed=None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] this shouldn't have the line break here

output = clip_vision.encode_image(init_image)
pooled = output.image_embeds.unsqueeze(0)
pixels = comfy.utils.common_upscale(init_image.movedim(-1,1), width, height, "bilinear", "center").movedim(1,-1)
encode_pixels = pixels[:,:,:,:3]
if augmentation_level > 0:
generator = torch.manual_seed(noise_seed)
if noise_seed is not None:
generator = torch.manual_seed(noise_seed)
else:
generator = None
encode_pixels += torch.randn(pixels.shape, generator=generator) * augmentation_level
t = vae.encode(encode_pixels)
positive = [[pooled, {"motion_bucket_id": motion_bucket_id, "fps": fps, "augmentation_level": augmentation_level, "concat_latent_image": t}]]
Expand Down