Smart Cut - option to cut audio separately in whole, then combine with video result (Request)

[bryc]

I have a lot of issues to go through, so in order to make it easier for me to help you, I ask that you please try these things first

• Check out all the menus and keyboard shortcuts. Maybe the feature already exists?
• Try with the newest version from GitHub
• Read the documentation and FAQ, Known issues, Troubleshooting to see if the feature already exists
• See if someone already suggested this under Issues or Discussions

Description

TLDR version:

Basically this could improve Smart Cut. Sometimes you get unlucky and the transition between the audio of the Smart Cut is not seamless. Usually it becomes noticeable if it occurs during speech, music, or some other form of louder transient. This audio stutter/repeat issue can be mitigated with an option to have Smart Cut be applied only to the video track, with the audio track cut separately in whole (with keyframe cut mode OFF), and then re-combined with the video track.

There could be a slight offset in audio sync, but it is likely negligible, as the results seem very usable.

---

I've been using Smart Cut often, but encounter issues sometimes, so I usually need to check and manually employ strategies to fix the issues.

The issue in my case presents itself as a sort of gap between the cut points. In VLC, this manifests as silence, so it's easily detectable. In FFplay, the issue presents itself as a slowing of framerate near the cut point, as if it stretches the video to fill this gap, and the audio appears to repeat or "echo" to fill the extra buffer space.

When the problem occurs, one way I work around this is to use an ffmpeg script to change the timebase of the original video, before proceeding with LosslessCut. In terms of video alignment, a timebase of 30K seems to work quite well in a lot of cases, particularly in 30 FPS files, though I sometimes check 16K, 24K, or 60K, if 30K doesn't work. This appears to fix both the silence in VLC, and the framerate slowing down in FFplay.

But there's still one thing. In a lot of cases, the audio is not aligned perfectly at the cut point, so there's around 10-50 milliseconds of a repeat or stutter. This is less of an issue if the cut point is quiet or constant (hum or noise) but sometimes it's more noticeable. And in these cases it'd be useful to have an option like this.

Anyway, I had an idea to potentially improve the situation.

Rather than re-encode the video with the audio and then merge - simply cut the video and audio separately, merge the video track by itself, and then combine the video and audio into the final output. This can work because audio tracks such as AAC-LC are not subject to the keyframe limitation, so they can be cut losslessly with more precision.

I've been testing this myself and it seems to work quite well. If there's any audio deync, it's not really perceptible. Aside from having to change the timebase to 30K before doing the cut, it basically allows Smart Cut to have almost no drawbacks.

This can technically be done with LosslessCut by turning off the respective audio or video track, and cutting each separately, and then in a separate window (or a new project) loading the lone video track and adding the audio track manually, but it's cumbersome and inelegant, requires you to close the current project, and requires constantly changing settings; turning the Keyframe cut option off and on (I typically leave it on), as well as the Smart Cut option.

But I think it's viable enough that it can be automated as a dedicated option.

[mifi]

Thanks for the suggestion and research. The Smart Cut operation is already quite complex and adding this would make it even more complex. I'm a bit afraid of breaking some use cases by adding this, but it's something to look into. I will add a reference to this issue inside #126 (OP), and will close this to keep issue count down.

[bryc]

I know you closed this issue, but definitely try to play around with it when you get a chance. No pressure! I know you got a ton of open issues.

I'm starting to think that muting the audio before "Smart Cut" is FAR more stable in a lot of cases, and might actually be in perfect sync with the video track, when the audio track is cut separately and re-combined later.

The sync issue itself is caused by the audio track being enabled during a vanilla "Smart Cut". Because it indeed causes a audio stutter resulting in a roughly 70ms delay. The audio often just doesn't line up perfectly at the desired cut point it seems.

So in actual fact, VLC might not be accounting for this delay in the video track at the cut point, which DOES occur in FFplay as momentary visual duplicate frames. So VLC potentially plays the video 1-3 frames earlier than the audio, after a vanilla Smart Cut operation.

When Smart Cut is applied to an MP4 video track without audio, the result appears to be far more "stable" and fluid in basically all players.

It appears that most issues or glitches that occur at the cut point are a result of the audio track being present.

Also, cutting the audio separately at the same also passes the null test for the first segment, so I don't think there's a cut granularity issue - at least in AAC audio.

And so when separately cut audio is recombined with the separately "Smart Cut" video, the result is really stable it seems.

Sorry if any of this is repetitive, just wanted to commit my thoughts to writing. Feel free to ignore it.