Official pytorch implementation of "FramePainter: Endowing Interactive Image Editing with Video Diffusion Priors"
default.mp4
- [01/25/2025] Inference demo and pre-trained weights are available now!
- [01/15/2025] Paper of FramePainter released!
FramePainter allows users to manipulate images through intuitive sketches. Benefiting from powerful video diffusion priors, it not only enables intuitive and plausible edits in common scenarios, but also exhibits exceptional generalization in out-of-domain cases, e.g., transform the fish into shark-like shape.
Download pre-trained weights of FramePainter into checkpoints/ directory, including finetuned U-Net and sparse control encoder. The app.py will automatically download stabilityai/stable-video-diffusion-img2vid-xt-1-1 during inference.
conda create -n framepainter python=3.10
conda activate framepainter
pip install -r requirements.txtDirectly run python app.py.
This repository borrows code from Diffusers and ControlNext. Thanks for their contributions!