Question about app.py

[beautifulchoi]

Hello,

First of all, thanks for your great work and for coming up with open source !

While playing with this (app.py), I have one question.

Since Flux itself needs quite a lot of VRAM memory, is it essential to have one for this task?

Is it just for comparison or any other reason to use both video inpainting and image inpainting?

Thanks for your assistance.

[yxbian23]

Hello,

First of all, thanks for your great work and for coming up with open source !

While playing with this (app.py), I have one question.

Since Flux itself needs quite a lot of VRAM memory, is it essential to have one for this task?

Is it just for comparison or any other reason to use both video inpainting and image inpainting?

Thanks for your assistance.

Thanks for the question!
In our work, we actually support two settings: image-to-video and text-to-video.

• Image-to-Video Setting:
  For this setting, we can take advantage of Flux’s strong image inpainting ability to first inpaint the first frame. Then we use VideoPainter to propagate the inpainted result to the subsequent frames based on the moving masks.

• Text-to-Video Setting:
  In this setting, VideoPainter is used to inpaint all frames directly from scratch, without relying on a given first frame.
  However, due to some internal release constraints, we are currently only able to open-source the image-to-video version. The text-to-video pipeline is implemented internally but not publicly released at this time.