Is this compatible with IP- Adapter faceID, can you explain how to use this code with pre trained IP-Adapter FaceID

[unboxdisease]

I have a pretrained IP-Adapter that uses adaface embeddings, i am trying to stylize the outputs, i think this method can help, but i dont think it is compatible with faceID is it?

[tanghengjian]

FYI
tanghengjian/instantid_with_ipa@0a3b733

[haofanwang]

Our team is working on integrating InstantStyle into native diffusers API (check this PR), once it merged, we will officially support multiple IPA, InstantID+InstantStyle, InstantStyle+IP-Adapter.

[plienhar]

@haofanwang Do you have a code example for InstantID+InstantStyle? Loading of InstantID IP adapter is pretty custom, it is hard to figure out how that interacts with a second adapter loaded separately when you don't know the inner workings of diffusers. Many thanks in advance!

[elismasilva]

@haofanwang Do you have a code example for InstantID+InstantStyle? Loading of InstantID IP adapter is pretty custom, it is hard to figure out how that interacts with a second adapter loaded separately when you don't know the inner workings of diffusers. Many thanks in advance!

even multiple ipa on diffusers is not working properly, maybe shape problem of models. I am trying do my own pipeline to load instant id and instant style together but i am stucking in image emb problem because instant id use controlidentity with 3d shape tensors, diffusers does 4d tensors, and other problem is attn processors.

[xiankgx]

Is this an issue related to IP-Adapter vs IP-Adapter Plus?

IP Adapter plus works not with image embeddings, but image "fine-grained features".
For example, IPAdapterXL vs IPAdapterPlusXL image embeddings shape is (1, 1280) vs (1, 257, 1280) respectively. This suggests the Plus SDXL models learned from more fine grained features (features from image patches).

This means embedding arithmetic like "king" - "man" + "woman" as demonstrated with this repo will likely not work.