edit memory video files

[jl-DaDar]

Is it possible to read an in memory video file, process it, and then write back to memory.
I need to combine a video file and a audio file from aws s3 and write it back to s3. I am trying to avoid saving the video in disk first.

thanks.

[kkroening]

Yes - see the following resources:

• Official ffmpeg pipe protocol documentation (to tell ffmpeg to read from stdin and write to stdout)
• Communicating with subprocess.Popen

Then when you run ffmpeg using ffmpeg-python, you can use e.g. subprocess.Popen to pass the data to stdin and read the output from stdout:

import subprocess
import ffmpeg

input_data = download_file_from_s3(...)
args = (ffmpeg
    .input('pipe:')
    # ... extra processing here
    .output('pipe:')
    .get_args()
)
p = subprocess.Popen(['ffmpeg'] + args, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
output_data = p.communicate(input=input_data)[0]
upload_file_to_s3(output_data, ...)

This approach reads the entire file into memory from S3, passes it into ffmpeg, gets the entire output from ffmpeg, then uploads the output to S3.

It's also possible to do a streaming approach where you read a little bit of the file from S3, pump it into ffmpeg, and then stream the upload piece by piece, but it's a bit harder because you either have to use non-blocking IO or multiple threads/greenlets (e.g. using gevent). It can be done, but I'd start with the 'read-entire-file-to-memory' approach first.

[kkroening]

@jl-DaDar Let me know if you need more clarification.

[roccolocko]

Hello, How about when using 2 files.

I'm merging video+audio in to 1 file using your library but I want to do it all in memory, how can I send 2 inputs using the mentioned approach?

Thank you!

[fake-warrior8]

Yes - see the following resources:

• Official ffmpeg pipe protocol documentation (to tell ffmpeg to read from stdin and write to stdout)
• Communicating with subprocess.Popen

Then when you run ffmpeg using ffmpeg-python, you can use e.g. subprocess.Popen to pass the data to stdin and read the output from stdout:

import subprocess
import ffmpeg

input_data = download_file_from_s3(...)
args = (ffmpeg
    .input('pipe:')
    # ... extra processing here
    .output('pipe:')
    .get_args()
)
p = subprocess.Popen(['ffmpeg'] + args, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
output_data = p.communicate(input=input_data)[0]
upload_file_to_s3(output_data, ...)

This approach reads the entire file into memory from S3, passes it into ffmpeg, gets the entire output from ffmpeg, then uploads the output to S3.

It's also possible to do a streaming approach where you read a little bit of the file from S3, pump it into ffmpeg, and then stream the upload piece by piece, but it's a bit harder because you either have to use non-blocking IO or multiple threads/greenlets (e.g. using gevent). It can be done, but I'd start with the 'read-entire-file-to-memory' approach first.

I used these commands. However, I get some errors.

1. Could not find codec parameters for stream 0 (Video: mpeg4 (mp4v / 0x7634706D), none, 210 kb/s): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options

2. [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55ee01839e80] stream 0, offset 0x1bc655: partial file
pipe:: Invalid data found when processing input
Cannot determine format of input stream 0:0 after EOF
Error marking filters as finished
Conversion failed!

How to solve these problems?