Skip to content

Commit

Permalink
Update DreamPose.md
Browse files Browse the repository at this point in the history
  • Loading branch information
garg-aayush authored Sep 13, 2023
1 parent b8b009a commit 25844d7
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion Summaries/Diffusion/DreamPose.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,9 @@ The Dreampose takes an input image $x_o$ and poses ${p_1,p_2...p_N}$
details in the conditioning image.
- So, instead VAE encoder latent embeddings are added along with CLIP image embeddings as image conditioning
- These embeddings are combined and added using an Adapter Module. This adapter blends both the signals together and transforms the output into the typical shape expected by the cross-attention modules of the denoising U-Net.
$$ c_I = A(c_{CLIP}, c_{VAE})$$
$c_I = A(c_{CLIP}, c_{VAE})$
- The weights corresponding to the VAE embeddings are set to zero, so as to mitigate network “shock” in training.

![img](./images/dreampose/method_1.png)

### Pose-conditioning using a modified UNet
Expand Down

0 comments on commit 25844d7

Please sign in to comment.