😍 Implement "smart cut" #126
Comments
|
Hi guys, Interesting that it is a limitation of the codecs. Reason I say this is that I can do a frame specific "lossless" cut using Apple's (very old) Quicktime Player 7 in Pro mode. For the cut to take at the specific frame it has to export to a new file (rather than edit the existing file). But you can do this just passing through the source data with out any form of recoding (both video and audio tracks). Not sure exactly how it does it, but seems that som,ehting could be possible. Save time isn't too bad either as it just seems to shuttle the data into a new file. Happy to send you a before and after file of a minute or so duration showing what it can do, if you'd like that. This is about the only thing that I would need to replace the ancient Quicktime Player 7 with lossless-cut as my mainstay tool. Other enhancemnets wouldbe nice, but this one is pretty essential for frame level edits. Hope this helps. Cheers, Ian |
|
Have you tried quicktime pro with a large file? Like 1gb, does it still take only a couple of seconds? |
|
This would be awesome really :) |
|
I more or less grew up with SvenOver's DVBcut which did implement "keyhole surgery" editing. He never updated it for HD (H.264 and etc) and, in the end, I shifted to VideoReDo which handles top-and-tailing and advert removal very efficiently and reliably using "smart cut" techniques and FFmpeg as the internal engine. Alas, it's Windows-only, not open-source and non-free (but relatively cheap) but it's a good benchmark to follow. So ... I'm absolutely enthralled if you're planning to add the facility to your LosslessCut ... all power to your elbow! In the meantime though, I wonder if you might consider the following: There has been some discussion here already on "cut at the natural frame-break before or after the desired cut-point " (so apologies if the matter has already been settled). How about a pair of extra buttons adjacent to the < and > buttons that skip to the natural break respectively before or after? That way the user can choose (to taste) whether to exclude the last (intrusive) advert frame or include the first (treasured) movie frame. At the moment, it's not obvious which sacrifice one will make :-). I'm enjoying getting to know your creation - it's a valuable addition to the linux toolkit. Thank you! David |
|
I'd like to point out that if you use It's a quirk to keep in mind when implementing smart cut. |
Does this ever lose frames? If there are key frames at frames Do you lose the first 2 frames of video? Or does it look for the first keyframe before/after your in/out points? |
|
Seeing as @ilaw138 didn't reply, I can. I've been using QTP7 and MPEG Streamclip for the past 10 years to manage the media files of a weekly show at work. It's possible to do a perfect cut on any size file, here are the steps:
|
|
Hi. Thanks for the info. I'm not really sure how QuickTime Pro does this, and I don't really understand why you need to use Streamclip in addition to quicktime pro? |
|
Apologies. Missed your e-mail about large files.
In short, no I haven’t tried to edit out segments frame exact on a large file of 1GB.
QTP7 didn’t really like large files in any event - at least in my experience - so never tried anything too challenging for the software. Have found if I save file from QTP7 (as opposed to export) it didn’t cut the frames exactly - so assume using key-frame - but export does.
Sorry I can’t be of more help.
Regards,
Ian
… On 14 Feb 2020, at 12:55, Daniel Mee ***@***.***> wrote:
Seeing as @ilaw138 <https://github.com/ilaw138> didn't reply, I can. I've been using QTP7 and MPEG Streamclip for the past 10 years to manage the media files of a weekly show at work. It's possible to do a perfect cut on any size file, here are the steps:
Trim to the exact position you want using the in/out markers in QTP7
Save it as a separate file (you can use QTP7's reference file option though so it will save a new file however fast your computer can write to disk and the size of the file)
Open the file in MPEG Streamclip and select File > Save As
In the save dialogue box change the dropdown to be mp4 and Save.
Done, no reencoding, exact to the frame and plays as expected. Feel free to ask me to provide you anything you need that might help you to solve this issue.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#126?email_source=notifications&email_token=ADIUWJYFHDTB5JX5QPTQ2MDRC2IFLA5CNFSM4GWMKI2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELY54EY#issuecomment-586276371>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADIUWJ2G4QRDHZRPB6U6CJLRC2IFLANCNFSM4GWMKI2A>.
|
Quicktime doesn't let you save a clipped file as a .mp4, only a .mov, Streamclip allowed me to get it back to .mp4 without recompressing. The only other way to get it out of QTP7 as a .mp4 was to export it but then it would compress it to those export settings. |
Not meaning to contradict you Ian but my experience working with QTP7 for 10 years pretty much every day was totally different. It regularly handled the 5, 10 or even 50G files I threw at it and its accuracy was spot on to the frame each time. If it wasn't I wouldn't have been able to use it as the transitions between segments in the program had to be edited out to the frame and QTP7 worked flawlessly each time. I also used the info panels to find out the duration to the millisecond (see the screenshot of a video's details, note the Current Time)
If it's helpful I can do a screencast of myself using it and how accurate it was... if it's going to help. |
|
Hi Daniel,
I am sure that we all have different experiences of the same software, I too have been using QTP7 for many years. I use export and set the compress options for audio and video to pass through, so no change to compression, but as I recall (I haven't used them for some time since I started to use Handbrake to compress my file before editing) was that large files and export was a hassle and quite slow to complete.
Yes you can save them as you mentioned, but I found that the saved output was not always "clean" in that there was sometimes a residual frame or two that I had cut out in the output file. From memory reference files did the same thing, but I didn't use StreamClip and may be that "cleans" up the file to the exact cut point.
So certainly for me only using QTP7 I found large files more cumbersome to export than already compressed files and Handbrake handles files of all sizes (again my experience) elegantly.
So Mikael, thinking about things I now presume that QTP7 when exporting did recode the entire file and set new key frames based upon the content of the edited file (sorry not that literate in exactly how things work, so am guessing here). As I used relatively small files - normally < 1GB it didn't take long, even on my MacBook Air (Core 2 Duo!)
Hope this helps the conversation. As I said originally with 32 bit now history from Apples perspective, it would be great if a solution could be found using Lossless Cut. As this is a great Application and handles files very well.
Cheers,
Ian
On 14 February 2020 at 15:56, Daniel Mee <notifications@github.com> wrote:
QTP7 didn’t really like large files in any event - at least in my experience
Have found if I save file from QTP7 (as opposed to export) it didn’t cut the frames exactly - so assume using key-frame - but export does.
Not meaning to contradict you Ian but my experience working with QTP7 for 10 years pretty much every day was totally different. It regularly handled the 5, 10 or even 50G files I threw at it and its accuracy was spot on to the frame each time. If it wasn't I wouldn't have been able to use it as the transitions between segments in the program had to be edited out to the frame and QTP7 worked flawlessly each time. I also used the info panels to find out the duration to the millisecond (see the screenshot of a video's details, note the Current Time)
If it's helpful I can do a screencast of myself using it and how accurate it was... if it's going to help.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
By the way I think normal quicktime that comes with mac os (not pro) can aslo trim losslessly. I just tried a relatively big file and it was instant and accurate, so im thinking it must be lossless. However it only works on mov/mp4 |
|
In my opinion this feature would be a key feature for lossless-cut. Maybe 'avcut' would be a good solution. I have been using it for some time now for smart rendering in my project. Here you can find the repository, and here is a blog about it. If you like, you can test a static build (Linux 64Bit) of mine. Even if only a few frames (before and after the keyframes) have to be rendered, this editing variant still needs much more CPU resources. If one could build avcut/ffmpeg in combination with hardware acceleration (vaapi / qsv), it would probably almost eliminate this disadvantage. |
|
I tried starting to implement a keyframe accurate cutting with re-rendering the part outside the keyframes, similar to avcut, but I was not able to make it work consistently. When I cut exactly on keyframe timestamp, sometimes the beginning of the output file gets corrupted, sometimes not. depending on the video file I test on. I managed to figure out how to render key frames effectively on the timeline however, so in the newest version you will have keyframes show up on the timeline, and you can zoom in to seek closely. Also now seeks should align with frames based on frame rate. |
|
This issue along with #372 should be a pinned issue. |
|
Sorry I didn't read the whole thread. But can't the out point cut be precise, as long as it's NOT on an i-frame? The next frame (beginning of the next segment) would be an I-frame? |
|
I just used handbrake it is precisely cutting but does re-encoding as well. Maybe you can implement like them? |
|
@mattack I think for most files, the out cut is indeed precise. |
|
I'm using the latest development version of SmartCut and it works well, but I notice that the ffmpeg command to make the |
|
I'm also using smart cut with av1 and it's very slow. It should not take multiple minutes to encode (on a Ryzen 5700X) what adds up to a few seconds at most. Is there a way to manually change the encoder used in smart cut? |
Even if you were to import the source file into Handbrake and select those frames/seconds to encode, you might find it takes a similarly long time. It doesn't just encode the selected portion; it moves through the file to that portion to find it. That is, it doesn't just jump to it and start encoding, but rather works its way to it. As for changing the encoder, I'd imagine it just automatically uses the same one the source file is encoded in, as using another would cause incompatibility, though I have no idea how it chooses the various settings for the encode beyond that. Given the chance to allow the user to customize this, that would provide the ability for using incompatible settings (not saying this shouldn't be done, just that it requires caution, and I've discussed this in another issue). I'd suggest importing your source file into Handbrake, selecting the same codec and as similar as possible quality settings as the source file, and having it encode the same portion of the file that you're trying to do with LC (i.e. the bit(s) at the beginning/end of the clip(s) you're editing. I'd wager that if you do this for the beginning and end of each clip, it's going to take a while, as for each one it's going to seek through the video up to that point before actually starting to encode. And while I can't say for sure, as obviously LC and HB are two different programs, I'm guessing they both use similar mechanisms to perform these operations, and that's likely why encoding a few pieces that are only a second or two, which seems like it should be very quick, are taking a while, because it's not the actual encoding that's taking so much time, but the encoder moving through the file to get to that point (or those points). |
|
Well, I can see through task manager that a lot of the time of the total video processing is just disk operations which makes sense as LC is making a copy of a large video file, but I can also see when the disk operations go down to 0% and the CPU usage goes up which is where the smart cut takes place. When I say it takes too long I mean just the smart cut portion, as the rest is justified. I also tested it with shorter video files and the smart cut takes many times longer than running the entire video through handbrake AV1-SVT encoding (although I don't know if the handbrake settings are comparable, but still). I tested the same 10 second file converted with Handbrake from AV1 to 264/265 too. Smart cut goes through 264/265 files in like a second or two but takes minutes on AV1. |
This is an issue I have brought up as well, Lossless cut uses whatever the default av1 implementation of ffmpeg is, this happens to be the original See my previous comment here for some numbers. Hopefully this can be implemented some time soon as it would increase the speed of these encodings by a good amount especially for much slower machines |
And this is with just one clip? Even with that, it's encoding two pieces, and for each one it has to run through the file up to that point and then encode it, so for each clip it has to do this twice, and that will take more time. But if you're saying it takes that much longer to do just one clip/segment vs encoding the entire video in HB, then that does sound wrong. I wonder if it's LC or ffmpeg (or an external encoder?) taking that long. I suppose you could watch the output files and see how long each one takes to complete and see what part of the process is so slow. |
Yeah, it would be nice if it allowed users to customize parameters. While this could be done via a GUI, that could get quite complicated, but even providing an option to show the command(s) that will be used and allowing the user to modify them would be nice. Granted, this would be a feature only for advanced users, but if you could just manually change part of the ffmpeg command to use whatever codec or settings you prefer, that would allow for full flexibility. And it could allow for automatic replacement of words in the commands, so you could set it to always replace e.g. codec_name_1 with codec_name_2. |
the request to manually modify commands before running them has come up before, but i'm not sure how useful it is, because it involves a lot of manual work, if you have to modify a command line every time a cut is done, and imagine if cutting 100 segments, then you'd have to do it 100 times 😵 - see #750
@BuyMyMojo Thanks for researching. The way I understand it from ffmpeg docs, if one wants to set
it also says:
So I would think that setting it to 8 would yield the lowest quality? Is that quality even acceptable? for libstav1 I would have to update the ffmpeg build script, probably with this: But how does it compare to the other codecs rav1e or aomedia av1 codec? |
That is only for libaom which I am still suggesting to move away from when detected as libsvtav1 doesn't require a separate usage flag
more than acceptable, it is still av1 so it still has the many benefits of that codec. I have seen more than decent encoding at usage 10 with svtav1 (which is the default). in my original post I mention svtav1 getting a VMAF score of 96.0291 which can very basically be seen as a % of quality compared to the original. so svtav1 at it's default preset of 10 can reach a really good quality. so compared to aom's encoder it is a tiny bit lower quality but only by a percent or two but with a massive speed boost. rav1e is slow by default but can also get a decent speed improvement but from my experience is more annoying to get to encode, needing more flags to encode. overall they are very similar quality but I would probably personally prefer svtav1's speed and quality combo for the smaller smart cut sections |
|
next nightly build will use libstvav1 for encoding as well as dav1d for decoding (ffmpeg built in decoder seems to be very broken). Also next nightly build will have a new Export Option called "Shift all start times", it can be used to automatically shift all segment start times forward by one or more frames before cutting (up to 10 frames). This can be useful if the output video starts from the wrong (preceding) keyframe. For Smart Cut, cutting from the wrong keyframe would sometimes cause the part around the cutpoint to be repeated (duplicated). When "shift start times" is set to +1 or +2 frames, this seems to resolve the issue for smart cut too for some files. |
|
Hello! |
|
AFAICT that answer is basically saying to do smart encoding, which is what LC attempts to do, but it's a work in progress. SolveigMM is trash and the developer is useless. TMPGEnc seemed promising, until I realized it doesn't support all audio codecs, so the video I was testing it with, which is one of many I'd like to edit, couldn't be done without resulting in an output file with no audio. Pretty useless, and ridiculous for such expensive software. I'm pretty sure I've tried every software available, and few have this feature, and none do it well/properly. Unfortunately, VideoReDo, which was the gold standard and may have worked, is no longer available, so LC seems, to me, to be the most promising option. |
|
I have used this program for a few years to make rough edits of videos, but I wanted to be precise when editing my DVD copies. People in this issue have talked about some of the odd behaviour of Smart Cut, e.g. inaccuracy by a few frames, repeating content or stuttering between connected segments. I have developed the following more involved approach for precisely editing videos with minimal loss in the latest FFmpeg. Perhaps it will help you and anyone else who stumbles upon this post. Hopefully someone can improve upon this. This only works on MP4 containers, including when they contain multiple audio streams and multiple subtitles. I tested the approach with Matroska, which resulted in freezes between segments no matter what I tried. Perhaps the error is in FFmpeg, since Matroska is usually a robust format. I have tested this on MP4 files containing H.264, AV1 and VP9 video, AC-3, DTS, AAC and Opus audio and VobSub, SubRip and WebVTT subtitles, the last two encoded using
As an example: I have noted the timestamps 6:04 and 9:13 in a video. The first timestamp is moved 10 seconds back, the second 10 seconds ahead, giving us 5:54 and 9:23. Then I run this command:
The profile and level are in the STREAM section, and "1k tbn" is the 1k timebase.
On the first line, If the segment contains not one whole sequence but small pieces of content to be cut, then you will have to encode it into small pieces, e.g. INPUT_01_cut_a/b/c/d. Naturally you will have to use If you have multiple cut segments (INPUT_03_cut, INPUT_05_cut etc.), then repeat steps 6 and 7 for those as well. That is, create INPUT_03_cut_a/b, INPUT_05_cut_a/b and so on.
Note that INPUT_01_cut is not included, only the re-encoded segments INPUT_01_cut_a and INPUT_01_cut_b. If you have more segments, e.g. 03 and 04, then of course you should add their parts too, for example:
In an alternative example where only the beginning and end of the video are cut, it would look like this:
Update: #!/bin/sh
# Sanity checks
if [ $# = 0 ]; then echo "No file provided."; return 1; fi
if [ ! -f "$1" ]; then echo "File does not exist."; return 1; fi
# Extract keyframe entries with FFprobe, then count number of keyframes
printf "Computing number of keyframes... "
ffprobe -v quiet -show_frames -show_entries frame=media_type,key_frame -of csv="p=0" "$1" > .temp.txt
if [ $? != 0 ]; then return 1; fi
num_keyfr=$(grep -o -i 'video,1' .temp.txt | wc -l)
if [ $? != 0 ]; then return 1; fi
echo $num_keyfr
# Extract duration with FFprobe
printf "Extracting duration... "
vid_dur=$(ffprobe -v quiet -show_entries format=duration -of csv="p=0" "$1")
if [ $? != 0 ]; then return 1; fi
echo $vid_dur
# Compute average GOP with awk decimal arithmetic
printf "Computing average GOP... "
awk 'BEGIN {print ('$vid_dur' / '$num_keyfr')}'
if [ $? != 0 ]; then return 1; fi
rm .temp.txt |
|
Very interesting findings. So your research suggests that Anyways it's definitely something to try implementing for LosslessCut's smart cut. I wonder how well this |
|
You are probably correct that much effort has been put into perfecting the segmenter, at least for MP4 containers, since it is used to create M3U8 playlists for web video players, where uninterrupted playback is critical. FFmpeg's segment option will pick the nearest keyframe, whether it is ahead or behind the provided timestamps; that is why the timestamps must be shifted by 10 seconds. Note that Note that MP4 can contain AV1, H.265, VP9 and Opus streams as well, but I have not tested an MP4 file containing such streams yet. |
|
Here is a more advanced example. I apologise for double posting, but I do not have a blog or anything, so I will leave this here. In this case, I will take #INTRODUCTIONS (2015) by LaBeouf, Rönkkö & Turner and cut it such that only 4 of the 36 acts remain.
Since the GOP is ~1 second, I shift the timestamps -1 before a cut, +1 after.
There is some choppiness when playing OUTPUT.mp4 with hardware decoding enabled in old versions of VLC. However, the latest version plays it flawlessly, as does other software like MPV or Chrome. Until LosslessCut can cut exactly, this is my less convenient workflow. |
Perhaps relevant: I have tested my approach on an MP4 file which contained AV1, Opus and embedded text subtitles (e.g. SubRip), which worked fine as long as I used the latest FFmpeg version. It also worked fine on an MP4 file containing VP9 and Opus. Some subtitle issues were also fixed by using the latest version. I have edited my original post to reflect this. Using the latest FFmpeg did not make Matroska freezing issues go away. |
|
I found an issue where ffprobe reports a ridiculously low bitrate for the stream versus the overall bitrate for the file; documented here #1997 This results in an unusable encoded section at the beginning when using SmartCut; 1/40th the expected bitrate in this particular situation. The issue appears due to how ffprobe/ffmpeg calculates the stream bitrate, which appears to be an estimate. |
|
next version will allow setting a custom bitrate |
|
This might not be the proper place to write this up, but couldn't really find an appropriate place. What losslesscut currently tries to set for a file I'm testing: Looks about 600mbit, which should be plenty, but unfortunately, it gets silently ignored, and we end up with the default x264 options (CRF rate control, with a value of 23), which is going to be a tad low IMO. Allowing a custom bitrate in the future is nice and well, but setting a bitrate does not yield predictable quality across a range of resolution, framerates, complexity of content etc. I personally believe that a better approach would be to aim for a quality/accuracy based rate control, with a very high quality default (for x264, say CRF 15-18). Maybe even allow a an adjustable value, or at dropdown of a few presets (0 = lossless'ish, 15 = human visually lossless, 18= very high quality), for example. Keep up the good work mifi, loving your tool! Heia Norge! ;) |
|
@flaeri If there isn't a proper place for it, you can make a new issue and kindly remove the comment to prevent confusion. |
|
Was the issue of audio stuttering on smart cut ever fixed (h264 format video)? It doesn't happen constantly, but it happens for me consistently 2 seconds in to every video I do, there is a very slight stutter. It works for my purposes for now, but it's the one and only flaw I've seen in LLC. Fresh install of LLC. |
and others
We cannot cut between keyframes, it's a technical limitation of modern video codecs. So ffmpeg will choose the nearest keyframe, which will not be the exact time you requested.
Might be able to use avcut for h264 although it is experimental. See #372 (comment)
See also discussion in #13
Update: Have made an initial implementation of this. This experimental feature will re-encode the part of the video from the cutpoint until the next keyframe in order to attempt to make a 100% accurate cut while losslessly copying the rest of the segment.
VideoRedo
VideoRedo is a now-defunct software that had feature overlap with LosslessCut.
Smart Media Cutter
Recently built in order to replace VideoRedo
They open sourced their smart cutting algorithm, could take inspiration from it: https://github.com/skeskinen/smartcut
Remaining issues
Inaccurate seeking
Seeking is not always accurate, causing glitches and/or jumps in the stitch points. This is often because I haven't found a way to consistently seek to and cut on an exact frame. Sometimes it could be caused by the "audio samples" problem below. If you want to experiment, you can try to remove the audio track and cut only video to see if it makes a difference.
Audio samples problem
Cut time accuracy/glitches when audio samples are large? (try without audio and see if it makes a difference). A workaround seems to be to cut audio/video separately and then merge them, with the tradeoff of potential audio/video de-sync:
subtitles
Other improvements
Cannot find any keyframe after the desired start cut point, see #Smart cut failure: "Unable to find keyframe" #1190 (comment)Done
aomav1insteadlibsvtav1instead ofaomav1(ffmpeg default) (codec needs to compiled for mac in ffmpeg-build-script) Change smart cut AV1 encoder to libsvtav1 #1825Limitations
size/duration(may be wrong)The text was updated successfully, but these errors were encountered: