Support Multi/InfiniteTalk

[kijai]

This is somewhat work in progress PR that adds initial basic Multi/InfiniteTalk support:

• Functionality to load Multi or InfiniteTalk models as model_patch objects
• cross_attn patching function to Wan base model
• InfiniteTalk Image to video node that supports both long generation within a loop and extension with separate samplers

I've chosen not to do the MultiTalk long generation method as I find it redundant due to InfiniteTalk, MultiTalk model itself will load, but further functionality need to be added to add the audio control to other models such as VACE.

As I'm not fully versed in how everything works yet, especially memory management wise, I'll be relying on feedback from @comfyanonymous and @Kosinkadink to finish this PR.

[ptmaster]

Y'all, this project's bout to make Comfyui famous — they been too quiet on voice tech, sheesh :)

[Kosinkadink]

Hey, thanks for the PR and sorry for the wait! I reviewed this PR with comfy just now. I left individual comments, but here is the summary:

1. Make block_idx go int transformer_options instead in the for loops of the forward function. This will be something we want for attention tricks in the near future so will keep from block_idx being implemented in multiple ways.
2. Even though it removes a lot of the useability right now, remove the looping components from this PR and instead make it work wiht just the first 'window' without the looping. I will consult with you and Jacob on implementing the standard for looping stuff in ComfyUI as a separate PR, that we can then use to bring back this functionality for this + self forcing models.
3. q and k should not be returned as it is an unknown whether it will cause higher memory peaks for all Wan stuff. Instead, the optimized_attention call should be overridable so that the extra attention call thingy can be done here, with the q and k that you need stored in transformer_options for the cross_attn patch to use.

More details are included in the comments. Let me know if you have any questions!

[armynew]

I hope this PR can be completed as soon as possible. I want to experience it in the native workflow. Thank you.

[kijai]

Test result with latest commit that includes workflow:

native_MultiTalk_testing_00012-audio.mp4

The inputs used can be found here:

https://github.com/MeiGen-AI/InfiniteTalk/tree/main/examples/multi

[snomiao]

+label:notify:jk

[armynew]

This PR took too long time

[zwukong]

yes ,i agree. Result looks good @comfyanonymous

[zwukong]

This is the first time kj tries to apply his wrapper to native workflow. It meas a lot, i hope this merge will contribute a lot to the community. He has supported a lot of wan projects, however almost none of them can be used in native workflow

[armynew]

Please quick review this PR , it took too long time

[Kosinkadink]

@kijai DynamicCombo will be officially supported once #11345 gets merged after next stable.

Even before getting merged, DynamicCombo is still 'secretly' available and works with sufficiently new frontend if you import from _io.py. If you change the node to have a 'speakers' (or something like that) dynamic combo that has a 'one' and 'two', then you can make the two masks + optional speaker 2 audio track only appear when 'two' is selected.