Speed and suitability for 1-sec everyday-style videos

[umaar]

Hey, would love to get this working with https://github.com/umaar/video-everyday

Here are a few issues I had from doing this manually with ffmpeg, I'm wondering if Editly gets around these? Browsing the readme, sounds like it might do!

1. Slow concatenation

Doing both video resizing and concatenation is pretty slow, e.g. scaling:

ffmpeg  -i video.mp4 -filter:v "scale=iw*min(1920/iw\,1080/ih):ih*min(1920/iw\,1080/ih), pad=1920:1080:(1920-iw*min(1920/iw\,1080/ih))/2:(1080-ih*min(1920/iw\,1080/ih))/2" -c:a copy video-out.mp4

Before a concat, I would try a scale videos appropriately. This seemed to be one way of scaling different sized videos into one common resolution. This command is a little daunting! Think I copied that from a forum. Also this is pretty slow!

2. Slow subtitles

In my project, I tried to overlay subtitles - it works, but again it's super slow. It was actually faster to export an SRT file from my node script, and then import everything into a commercial video editing tool, and then export again!

3. Slow trimming

This is the trickiest issue. When ingesting videos, I would grab a small segment around the half way mark. Using lossless cut or ffmpeg fast mode was fast - but because it didn't align with a keyframe, there'd be a black screen at the start and funny audio issues. In the end I had to do it the "slow" way in ffmpeg (I think it reencodes), the problem: extracting a 2 second clip from a 20min 4k video takes a long time! It's strangely faster for me to open the video in a commercial video editing tool, trim the video, and export!

I think what I'm trying to get across in this long comment is that in ffmpeg, there seems to be two ways of doing things:

• fast way: but is inaccurate and can produce strange artefacts in the output video
• slow way: very accurate, does exactly what you ask, but can take a long time to execute

I was hoping for a fast and accurate way! I'm wondering, does editly get around these challenges?

[chapmanjacobd]

Slow trimming

I think it's definitely possible to use -ss -i -ss to quickly seek to let's say 4 seconds "before" (I guess ffmpeg seeks backwards and doesn't reverse if go too far) the calculated target duration (lossless), then slowly seek with keyframe accuracy (re-encode). but I'm not sure. maybe we would need to do a fast cut for the bulk of the clip and then do a re-encode for the start and end of the clip, (then concat those parts together...) It depends on the video codec and I'm not sure how ffmpeg handles this

related to: mifi/lossless-cut#126 which is related to mifi/lossless-cut#13

[mifi]

1. The reason why editly does scale all videos to the same size is because it has to be 100% sure of the output size (width x height x channels, rgba raw video out) when processing frames, or else it will lose sync in the data stream. It is also a technical impossibility to have multiple different resolutions in one video (afaik).

2. how big is the output resolution of your project? and how slow is it? there are probably some performance optimizations that can be done in the way I handle raw frames, and in fabric.js (which is used to overlay and draw text), but I'm not sure where is the bottleneck

3. Seeking should not be very slow, because it is using a fast seek (-ss ... -i ...) which AFAIK is now also accurate and fast at the same time. How long does it take to seek before it starts producing frames? (--verbose)

Out of curiosity, are you specifying duration of your video clips? if you are, that possibly means the clips are being sped up, which can slow things down.

[chapmanjacobd]

I think most of these comments aren't about editly specifically but about how we are handling ffmpeg which would help the video-everyday project.

Fast and accurate should be possible

https://superuser.com/a/704118/76968

Edit: hmmmm

https://www.reddit.com/r/ffmpeg/comments/aq98eu/help_frame_accurate_keyframe_seek_with_stream_copy/

Looks like the behavior changed in the last few years:

When used as an input option (before -i), seeks in this input file to position. Note that in most formats it is not possible to seek exactly, so ffmpeg will seek to the closest seek point before position. When transcoding and -accurate_seek is enabled (the default), this extra segment between the seek point and position will be decoded and discarded. When doing stream copy or when -noaccurate_seek is used, it will be preserved.
When used as an output option (before an output filename), decodes but discards input until the timestamps reach position.

So moving the -ss shouldn't make any difference

Maybe accurate seeking is just not possible in actuality. "in most formats it is not possible to seek exactly"

[mifi]

My experience is also that -ss both before and after -i is fast, so I don't understand why it's so slow for @umaar

[umaar]

Thanks for chiming in anyway! After some research it seems that a big reason is just the videos are so large, they are 4K videos for example. Other professional video editing tools use hardware acceleration which is why it appears faster when using them.