Extra frames show up when using `split-video` with ffmpeg

As the title: i don't get good cuts. But often the scenes start or end with the previous or next scene.

I'm using a standard: detect-content split-video.

It's happening with different sources.

python source: https://www.python.org/ftp/python/3.8.2/python-3.8.2.exe
pyscenedetect source: pip install scenedetect[opencv,progress_bar]
ffmpeg source: https://ffmpeg.zeranoe.com/builds/win64/static/ffmpeg-20200415-51db0a4-win64-static.zip

Here the debug command output:
debug.txt

Here a sample: https://gofile.io/?c=pf6fAV

The output: https://gofile.io/?c=DuSWBq

Fast issue overview:

The first clip have the umbrella in the preview. But that umbrella is just at the end of that clip. It start with a different scene which it doesn't even appear here.

Using mkvmerge is fine. I only added "-c".

[Breakthrough]

I was hoping this would go away after closing out #93, but it appears that it's still cropping up. I can try shifting the start time ahead by 1 frame, but then you might miss a frame or two on other scenes. Still not sure what the root cause is of this.

I may close this as a duplicate of #93 and reopen it. To investigate the fix, I was going to try and use frame numbers with ffmpeg instead of timecodes - still unsure as of why the issue is occuring, however, as outputting actual images of the frames it's using as timecodes seems to line up with what is expected.

Thoughts?

[Breakthrough]

@amvscenes Does the issue show up in the actual video frames, or just the thumbnail? I took a quick glance at the output you sent and didn't immediately see anything wrong. How should I be validating what you're seeing? (Extracting all frames to images?)

I can see that when playing it and if i analyze it into a movie editor.

If you export all frames you see that the split contain two scenes instead of one.

scene 03

scene 04

scene 05

I removed frames in between to show the issue.

[Breakthrough]

Thanks for the response. If you use the save-images command, are the first/last frames that PySceneDetect extracts correct?

Interestingly enough, all of the video clips seem fine when I play them using VLC. I'm still not entirely sure what the underlying issue is here, because the timecodes passed to both mkvmerge and ffmpeg are the same in both cases...

I'll try to see if I can get to the bottom of this though, hoping there's a way I can get ffmpeg to produce frame-perfect cuts without too much trouble. Thanks for the data.

Yes. The save-images output is perfect.

[Breakthrough]

Hey @amvscenes;

So I did some further digging here, and noticed something interesting - when re-encoding the file with ffmpeg, the framerate seems to double for some reason? Also the framerate of the original video is a rather odd number, can you confirm that it is correct? Although I doubt it's related here, note that PySceneDetect doesn't work with variable-framerate videos.

I'm able to replicate this issue on my end, but not sure exactly how to proceed. I was able to get frame-accurate cuts by manually specifying -frames to ffmpeg, but ffmpeg thinks the framerate is actually 47.9fps (twice the input), whereas both OpenCV and vlc indicate that the original framerate is 23.9-something FPS. Thus, the frame length I would calculate would be completely wrong (since the output seems to have twice the number of frames). I'll investigate if forcing the output framerate would be an acceptable workaround, since it seems like there is indeed duplicate frames in the output in this particular case, although why that is I'll need your help to solve.

You mentioned this happens with different sources, do they all share the same framerate/codec? Have you noticed the doubling in framerate in the output? Generally issues like that can be caused by corrupted video files, although since you mentioned this happens with several sources, I'd like to dig a little further into that and see if there is any commonalities between those sources (specifically how they were encoded).

[Breakthrough]

Hey again @amvscenes;

Would you be able to provide a longer clip? It seems like the actual timestamps get skewed over time in this particular case, and specifying a framerate override doesn't seem to help. I did fix an issue where the end timestamp could select an extra frame (pushed to master), but here it seems like over time, the actual timestamps need to be shifted backwards.

I'm not entirely sure why this is happening here, but given ffmpeg's misinterpretation of the framerate, something suspicious is definitely going on here. Additional samples and any other information you can provide would be most helpful.