Using setpts for timeshift for one specific input and using the overlay filter causes a timeshift of all inputs

[thisaintsparta]

Description
I am using FFmpegKit and executing a command with a list of arguments.

I have two video inputs, [0:v] and [1:v].
Using -filter_complex and with it the following argument:
'[0:v]setpts=PTS+2/TB[timeshiftedVid];[timeshiftedVid][1:v]overlay=[outv]'
and nothing else apart from the obvious -map [outv] and output path in order for ffmpeg to render the video.

After consideration of the ffmpeg wiki, i would expect the following in the output video:

• The second input would be playing straight away
• the first frame of the first input would be shown straight away
• the second input would start playing after 2 seconds.

What it actually does:

• The first frame of each the first and second input are shown for 2 seconds
• Both videos start playing after 2 seconds.

Now I'm happy to be told that my expectation is just wrong, but so far it dont think it is, hence this post.

I suspect the timeshifted input as the first input for the overlay to cause this. When that is timeshifted, somethign seems to go south there.

I created an example repo for this, I used android might just work out of the box for ios or macos aswell.
https://github.com/thisaintsparta/ffmpeg_timeshift

Current workaround
For positive timeshift values (meaning a delay of the playback of the input):
Timeshift the other input by the negative value and add a black screen (or the first frame) at the front for the amount of time.

Expected behavior
Described above

Current behavior
Described above

To Reproduce
Build my example repo:
https://github.com/thisaintsparta/ffmpeg_timeshift
Environment

• Platform: Android/Flutter
• Architecture: arm64,
• Version: Android version 13
• flutter doctor: Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel stable, 3.22.3, on Microsoft Windows [Version 10.0.22631.4037], locale de-DE)
[√] Windows Version (Installed version of Windows is version 10 or higher)
[√] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
[√] Chrome - develop for the web
[√] Visual Studio - develop Windows apps (Visual Studio Community 2022 17.10.3)
[√] Android Studio (version 2022.2)
[√] VS Code (version 1.92.2)
[√] Connected device (4 available)
[√] Network resources

Other
This is part of the code executing the command:

final command = [ '-y', '-i', video2.path, '-i', video.path, '-filter_complex', '[0:v]setpts=PTS+2/TB[timeshiftedVid];' '[timeshiftedVid][1:v]overlay=[outv]', '-map', '[outv]', output.path, ];
final session = await FFmpegKit.executeWithArgumentsAsync(command);

[mathieu-aubin]

Did you try this command with a standard ffmpeg binary on the command line? I don't have a ton of time to test but it might be best for you to try and see if you get the expected result... Simple linux box with a static ffmpeg would be easy and fast to set up in order to investigate...

setpts seems to generally be used to change the speed of a given input rather than used as a timeshifting method

[mathieu-aubin]

Maybe even try this -> https://superuser.com/questions/505668/ffmpeg-overlay-video-with-time-shift-or-offset

From the ffmpeg documentation...

-itsoffset offset (input)

    Set the input time offset.

    offset must be a time duration specification, see [(ffmpeg-utils)the Time duration section in the ffmpeg-utils(1) manual](https://ffmpeg.org//ffmpeg-utils.html#time-duration-syntax).

    The offset is added to the timestamps of the input files. Specifying a positive offset means that the corresponding streams are delayed by the time duration specified in offset.

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.