Skip to content

Add Start and End Frame control,works great! #167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

TTPlanetPig
Copy link

@TTPlanetPig TTPlanetPig commented Apr 20, 2025

by simply replace the last predicted image by inserted end image. now we can easily control the video to what we want. the Pincer attack from TeneT now is complete.
Thanks my friend "hhy" helped me on the coding as I only have the idea but he is much better on coding!

4.20.-1.mp4

image

250420_204753_497_5473_10.mp4

@FragmentY
Copy link

nice work!

@luze1011
Copy link

NICE

@FurkanGozukara
Copy link

Amazing work

I have implemented into my app and working amazing

My app tutorial : https://youtu.be/HwMngohRmHg?si=ImYvFey-R030fbiM

screencapture-localhost-7860-2025-04-20-20_19_26

250420_194342_833_7054_seed269740392_37.mp4

screencapture-localhost-7860-2025-04-20-20_25_31

@kabachuha
Copy link

kabachuha commented Apr 20, 2025

Interpolating between the start and the end conditionings depending on the frame number can be cool as well (actually, such interpolation is a unique feature of next frame prediction models), it will require some caching though, but it's for the future maybe :)

Anyway, amazing work, works like a charm! 🚀

@rkfg
Copy link

rkfg commented Apr 20, 2025

I tried doing the same and if the images are different enough the effect is bad. All segments but the first would animate the last frame, and the first segment would quickly turn from the first frame to last. Maybe I did something wrong, can you try with a longer video like 10-15 seconds?

@zadokov
Copy link

zadokov commented Apr 20, 2025

It could have been great but for now this only works for the first 1 second (all the transformation is done on 1st second and all the other seconds are just static).

@lllyasviel
Copy link
Owner

I think maybe I should create a repo like FramePackSoftware or FramePackAdvanced or some similar names as a independent repo to merge and implement ideas. This main repo is a research repo and need to be made simple. I will probably move this PR. Let me think how to process it

@zadokov
Copy link

zadokov commented Apr 20, 2025

I think maybe I should create a repo like FramePackSoftware or FramePackAdvanced or some similar names as a independent repo to merge and implement ideas. This main repo is a research repo and need to be made simple. I will probably move this PR. Let me think how to process it

@lllyasviel - All Issues and PRs here seem related to the software/studio - Maybe create a new repo for the Research?

@rkfg
Copy link

rkfg commented Apr 20, 2025

It could have been great but for now this only works for the first 1 second (all the transformation is done on 1st second and all the other seconds are just static).

That was my experience as well, I described my findings here: #32 (comment)

@spawner1145
Copy link

Amazing work

I have implemented into my app and working amazing

My app tutorial : https://youtu.be/HwMngohRmHg?si=ImYvFey-R030fbiM

screencapture-localhost-7860-2025-04-20-20_19_26

250420_194342_833_7054_seed269740392_37.mp4
screencapture-localhost-7860-2025-04-20-20_25_31

I have seen you in many issues and you are always trying to copy others' open source ideas to make money,don't you feel ashamed?

@TTPlanetPig
Copy link
Author

I tried doing the same and if the images are different enough the effect is bad. All segments but the first would animate the last frame, and the first segment would quickly turn from the first frame to last. Maybe I did something wrong, can you try with a longer video like 10-15 seconds?

for long video, you will need more middle frame or change the schedule method. still under study...

@GJohGJ
Copy link

GJohGJ commented Apr 21, 2025

Nice work!! If we could use controlnet also, this make a difference in guiding from a existing video

@@ -184,7 +200,8 @@ def worker(input_image, prompt, n_prompt, seed, total_second_length, latent_wind
history_pixels = None
total_generated_latent_frames = 0

latent_paddings = reversed(range(total_latent_sections))
# 将迭代器转换为列表
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

English comment would be great

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

中文评论才是精华

@andorxornot
Copy link

it would be cool to set keyframes for any frame, not just start/end.

@Eikwang
Copy link

Eikwang commented Apr 22, 2025

效果非常好!

@deniss-muhla
Copy link

it would be cool to set keyframes for any frame, not just start/end.

If we have start and end frames when we can just split video generation in sections...
Actually it's even better to have a separate generation for each part as it will be easier to describe motion.

IMHO the next step will be batch generation and joining.
For now I join them in the video editor.

@poprice29
Copy link

Amazing work

I have implemented into my app and working amazing

My app tutorial : https://youtu.be/HwMngohRmHg?si=ImYvFey-R030fbiM

screencapture-localhost-7860-2025-04-20-20_19_26
250420_194342_833_7054_seed269740392_37.mp4

screencapture-localhost-7860-2025-04-20-20_25_31

You are literally saying you implemented someone else's work and are essentially selling it. Have you no shame?

@mocat15
Copy link

mocat15 commented Apr 25, 2025

Great job, @TTPlanetPig and hhy! It worked great for me. One issue that I had was that the size of the output videos was smaller than the i2v default output files with the same settings. I turned down the MPEG compression to zero, which made me feel a bit better.

Thank you, @lllyasviel, for Framepack!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.