WanVaceToVideo node reference_image does not work ！

[juntaosun]

Expected Behavior

The WanVaceToVideo node provided by the official template does not work as expected.
It has a WanVaceToVideo node input, and reference_image does not work at all.

Actual Behavior

Although the WanVaceToVideo node has been released for a long time and is now in the official template, there seems to be no reported issues with this node.

It has a reference_image input for reference to VACE generation. But in fact, no matter how you change the reference image, reference_image does not work at all.

The result it generates does not refer to your image input at all. This is not a problem with VACE, but there may be a bug in the WanVaceToVideo node.

Steps to Reproduce

For the same model, I used KJ's node (ComfyUI-WanVideoWrapper), which also contains a ref_images reference image input, and ran the workflow test, which showed that it worked well and the results were generated according to the reference image.

So it proves that the reference_image in the WanVaceToVideo node should be a bug, please fix it. Thank you~!

Debug Logs

Vace 1.3B /  14B

ComfyUI 0.3.39
ComfyUI_frontend v1.20.7
Pytorch Version 2.6.0+cu124

Other

No response

[comfyui-wiki]

Please share your workflow here. Since some basic nodes are missing in the core nodes, there are many parts in the template that currently need to be set manually. I need your workflow to understand the corresponding situation.

[juntaosun]

@comfyui-wiki You can use the official workflow to test. Change the model of 1.3B or 14B.
workflows/video_wan_vace_14B_i2v.json (templates)

ComfyUI-WanVideoWrapper workflows (VACE):
https://github.com/kijai/ComfyUI-WanVideoWrapper/tree/main/example_workflows

[comfyui-wiki]

Testing. I will try to correct it if I can.

[comfyui-wiki]

Setting the lora strength_model to between 0.3 and 0.5 seems better.

Try the original workflow by comfy here

It seems to be because of the lora, and I need to test the rest of the template. The CausVid lora is used to speed up the VACE video generation, but the parameter needs to be adjusted to the right value.
I'm still testing now.

Thanks for the feedback. I will update here if I have fixed it.

[juntaosun]

I just tried turning down the Lora intensity, but the results were not satisfactory.
You can try a reference image of a person (a man or a woman) so it will be easier to spot the problem.

Reference image:

Lora intensity =0.8, result: （Does not match input reference）

Lora intensity =0.3, result:（Does not match input reference）

[comfyui-wiki]

Sorry for replying so late. I have updated the template and submitted a PR now. And there is still one error setting in the WanVaceToVideo node. The strength should be 1.0 but it was wrongly set to 0.5. This caused the renference_image to not work properly.

Already tested all the new settings,

Will be fixed once the PR is merged.

[wsbagnsv1]

Just for your info, there are two new causvid lora versions, at least V1.5 works a lot better with phantom, didnt test vace yet though

[comfyui-wiki]

Just for your info, there are two new causvid lora versions, at least V1.5 works a lot better with phantom, didnt test vace yet though

Thanks for the feedback. I've been busy with other model support recently. There are a lot of templates that should be added. You can test it with the native workflow. I still need to create inpainting and mask editing templates for VACE. I'm just too busy these days.

[wsbagnsv1]

Same with all those new releases, my internet connection is maxed out with uploads of ggufs 😅

[juntaosun]

After today's update, K sampler doesn't work, others have the same problem. WAN21_Vace

ComfyUI 0.3.39, ComfyUI_frontend v1.21.3
#8371 (comment)

\ComfyUI\comfy\model_base.py", line 174, in _apply_model
    ex.append(ext.to(dtype))
              ^^^^^^
AttributeError: 'float' object has no attribute 'to'

[comfyui-wiki]

Still the same template, right? Haven't changed any settings? If so, I'll check again later.

[psinsf]

Same problem as OP, WanVaceToVideo is ignoring the reference_image. (ComfyUI 0.3.39). I have strength set to 1.00, not 0.5. I've also tried with and without CausVid, same error.