Ffmpeg With OpenCVForUnity Example

Overview

• Integrate "FFmpeg for Unity" with "OpenCV for Unity".
• Replace WebCamTextureToMatHelper with FfmpegToMatHelper.
• FFmpeg receives video from various supported input sources (such as RTSP, HTTP, HLS, or local files), converts it to OpenCV’s Mat class, and performs image processing.

Environment

• (Optional) ATOM Cam Swing + atomcam_tools
  (can be replaced with any RTSP-compatible camera)
• Windows / macOS / Linux / Android / iOS
• Unity 2021.3.35f1+
• Scripting Backend: Mono / IL2CPP
• OpenCV for Unity 3.0.0+
• FFmpeg for Unity 2.11.1+
• Runtime Inspector & Hierarchy 1.7.0+
• In-game Debug Console 1.5.9+

Setup

1. Download the latest release unitypackage. FfmpegWithOpenCVForUnityExample.unitypackage
2. Create a new project. (FfmpegWithOpenCVForUnityExample)
3. Import and Setup OpenCV for Unity.
   • Download Dnn model files by ExampleAssetsDownloader.
   • Move the files from the "OpenCVForUnity/StreamingAssets/" folder to the "Assets/StreamingAssets" folder.
4. Import and Setup FFmpeg for Unity.
5. Import Runtime Inspector & Hierarchy.
6. Import In-game Debug Console.
7. Import FfmpegWithOpenCVForUnityExample.unitypackage.
8. Set the URL of RTSPServer you wish to receive.
9. Add the "Assets/FfmpegWithOpenCVForUnityExample/*.unity" files to the "Scenes In Build" list in the "Build Settings" window.
10. Build and Deploy.

ScreenShot