Add support for HuMo

[wsbagnsv1]

Feature Idea

A similar model to wan s2v
https://huggingface.co/bytedance-research/HuMo
GGUFs are already ready to go (;

Existing Solutions

No response

Other

No response

[comfyanonymous]

#9903

[comfyanonymous]

Reopening because I implemented the 1.7B and looks like the 14B is slightly different.

[juntaosun]

I updated to v0.3.59, but I can't find the HuMo node.When will it be released?

[krisheetu]

Hey @comfyanonymous may you please check if it is possible to implement the eligen Lora and its entity functionality to comfyUI?

[zwukong]

Not Compatible with sageAttention2

[juntaosun]

@comfyanonymous I tried HuMo 1.7B and the quality was terrible. The faces weren't consistent, and the audio and lip movements didn't seem to align. What's going on?

[wsbagnsv1]

@comfyanonymous I tried HuMo 1.7B and the quality was terrible. The faces weren't consistent, and the audio and lip movements didn't seem to align. What's going on?

Its based on the old wan 1.3b, which wanst very good to begin with

[wsbagnsv1]

The 17b one should be a lot better (;

[patientx]

is there a workflow yet ?

[juntaosun]

This is a single image reference workflow. Does anyone know how to build a multi-image reference? Feel free to let me know.

​The model is trained on 97-frame videos at 25 FPS.
Generating video longer than 97 frames may degrade the performance. We will provide a new checkpoint for longer generation.

@comfyanonymous

[patientx]

This is a single image reference workflow. Does anyone know how to build a multi-image reference? Feel free to let me know.

Where is the workflow ?

[kurttu4]

The decoder and lora aren't loading there. All the lores are downloaded from links in the workflow.

missing audio encoder: ['feature_extractor.mel_spectrogram.spectrogram.window', 'feature_extractor.mel_spectrogram.mel_scale.fb']
unexpected audio encoder: ['decoder.embed_positions.weight', 'decoder.embed_tokens.weight', 'decoder.layer_norm.bias', 'decoder.layer_norm.weight', 'decoder.layers.0.encoder_attn.k_proj.weight', 'decoder.layers.0.encoder_attn.out_proj.bias', 'decoder.layers.0.encoder_attn.............

.......

lora key not loaded: diffusion_model.blocks.0.cross_attn.k_img.diff_b
lora key not loaded: diffusion_model.blocks.0.cross_attn.k_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.0.cross_attn.k_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.0.cross_attn.norm_k_img.diff
lora key not loaded: diffusion_model.blocks.0.cross_attn.v_img.diff_b
lora key not loaded: diffusion_model.blocks.0.cross_attn.v_img.lora_down.weight
lora key not loaded: diffusion_model.blocks.0.cross_attn.v_img.lora_up.weight
lora key not loaded: diffusion_model.blocks.1.cross_attn.k_img.diff_b............

kijai/ComfyUI-WanVideoWrapper - works perfectly with the same models.

humo_lora key not loaded.txt