Custom video sink

[kalgecin]

val peerConnectionFactory = PeerConnectionFactory(audioDeviceModule)

val customVideoSource = CustomVideoSource()
val customTrack = peerConnectionFactory.createVideoTrack("custom-src", customVideoSource)
connection = peerConnectionFactory.createPeerConnection(config, peerObserver)
connection.addTrack(customTrack, listOf(customTrack.id))

val data = //generate yuv frame byte array 
val nativeI420Buffer = NativeI420Buffer.allocate(width, height)
nativeI420Buffer.dataY.put(0, data, 0, ySize)
nativeI420Buffer.dataU.put(0, data, ySize, uvSize)
nativeI420Buffer.dataV.put(0, data, ySize + uvSize, uvSize)
val videoFrame = VideoFrame(nativeI420Buffer, 0, System.currentTimeMillis() * 1000000)

customVideoSource.OnFrameCaptured(videoFrame)
videoFrame.release()

[voqaldev]

Thanks for this, @kalgecin. I can confirm this works for custom video sources.

[luokeith]

val peerConnectionFactory = PeerConnectionFactory(audioDeviceModule)

val customVideoSource = CustomVideoSource()
val customTrack = peerConnectionFactory.createVideoTrack("custom-src", customVideoSource)
connection = peerConnectionFactory.createPeerConnection(config, peerObserver)
connection.addTrack(customTrack, listOf(customTrack.id))

val data = //generate yuv frame byte array 
val nativeI420Buffer = NativeI420Buffer.allocate(width, height)
nativeI420Buffer.dataY.put(0, data, 0, ySize)
nativeI420Buffer.dataU.put(0, data, ySize, uvSize)
nativeI420Buffer.dataV.put(0, data, ySize + uvSize, uvSize)
val videoFrame = VideoFrame(nativeI420Buffer, 0, System.currentTimeMillis() * 1000000)

customVideoSource.OnFrameCaptured(videoFrame)
videoFrame.release()

Thank you very much for the code, can you provide a demo of the java syntax? I didn't understand the meaning of this piece of code. Thank you so much!
“
nativeI420Buffer.dataY.put(0, data, 0, ySize)
nativeI420Buffer.dataU.put(0, data, ySize, uvSize)
nativeI420Buffer.dataV.put(0, data, ySize + uvSize, uvSize)
“

[kalgecin]

I don't do much java as of late, so can't make a fully java example, but it should not differ from the example I provided (except for val and absence of new keyword)

nativeI420Buffer.dataY.put(0, data, 0, ySize)
nativeI420Buffer.dataU.put(0, data, ySize, uvSize)
nativeI420Buffer.dataV.put(0, data, ySize + uvSize, uvSize)

Here, I have a byte array buffer in which [0 to ysize] is the Y frame, then the next uvSize bytes are the U frame and the rest is the V frame. And i'm copying the values with the ranges specified into the nativeI420Buffer (which has the frames as separate buffers). So essentially i'm splitting my one array into the 3 byte buffers

https://download.java.net/java/early_access/panama/docs/api/java.base/java/nio/ByteBuffer.html#put(int,byte%5B%5D,int,int)

[luokeith]

I don't do much java as of late, so can't make a fully java example, but it should not differ from the example I provided (except for val and absence of new keyword)

nativeI420Buffer.dataY.put(0, data, 0, ySize)
nativeI420Buffer.dataU.put(0, data, ySize, uvSize)
nativeI420Buffer.dataV.put(0, data, ySize + uvSize, uvSize)

Here, I have a byte array buffer in which [0 to ysize] is the Y frame, then the next uvSize bytes are the U frame and the rest is the V frame. And i'm copying the values with the ranges specified into the nativeI420Buffer (which has the frames as separate buffers). So essentially i'm splitting my one array into the 3 byte buffers

https://download.java.net/java/early_access/panama/docs/api/java.base/java/nio/ByteBuffer.html#put(int,byte%5B%5D,int,int)

I got it, thank you so much! @kalgecin

[luokeith]

Hello @kalgecin , I'd like to ask you a question. I'm encountering a difficult-to-understand issue. The same piece of code can successfully execute 'mvn install' on macOS, but on the Windows 10 x64 platform, running 'mvn install' results in a compilation failure. The error log is as follows:

Creating library C:/Dev_Env/Project/webrtc-java/webrtc-jni/target/windows-x86_64/Release/webrtc-java.lib and object C:/Dev_Env/Project/webrtc-java/webrtc-jni/target/windows-x86_64/Release/webrtc-java.exp
VideoTrackDeviceSource.obj : error LNK2001: unresolved external symbol "public: class std::basic_string<char,struct std::char_traits,class std::allocator > __cdecl webrtc::webrtc_sequence_checker_internal::SequenceCheckerImpl::ExpectationToString(void)const " (?ExpectationToString@SequenceCheckerImpl@webrtc_sequence_checker_internal@webrtc@@qeba?AV?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@xz) [C:\Dev_Env/Project/webrtc-java/webrtc-jni/target/windows-x86_64/webrtc-java.vcxproj]
JNI_CustomVideoSource.obj : error LNK2001: unresolved external symbol "public: class std::basic_string<char,struct std::char_traits,class std::allocator > __cdecl webrtc::webrtc_sequence_checker_internal::SequenceCheckerImpl::ExpectationToString(void)const " (?ExpectationToString@SequenceCheckerImpl@webrtc_sequence_checker_internal@webrtc@@qeba?AV?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@xz) [C:\Dev_Env/Project/webrtc-java/webrtc-jni/target/windows-x86_64/webrtc-java.vcxproj]
JNI_VideoDesktopSource.obj : error LNK2001: unresolved external symbol "public: class std::basic_string<char,struct std::char_traits,class std::allocator > __cdecl webrtc::webrtc_sequence_checker_internal::SequenceCheckerImpl::ExpectationToString(void)const " (?ExpectationToString@SequenceCheckerImpl@webrtc_sequence_checker_internal@webrtc@@qeba?AV?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@xz) [C:\Dev_Env/Project/webrtc-java/webrtc-jni/target/windows-x86_64/webrtc-java.vcxproj]
JNI_VideoDeviceSource.obj : error LNK2001: unresolved external symbol "public: class std::basic_string<char,struct std::char_traits,class std::allocator > __cdecl webrtc::webrtc_sequence_checker_internal::SequenceCheckerImpl::ExpectationToString(void)const " (?ExpectationToString@SequenceCheckerImpl@webrtc_sequence_checker_internal@webrtc@@qeba?AV?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@xz) [C:\Dev_Env/Project/webrtc-java/webrtc-jni/target/windows-x86_64/webrtc-java.vcxproj]
VideoTrackDesktopSource.obj : error LNK2001: unresolved external symbol "public: class std::basic_string<char,struct std::char_traits,class std::allocator > __cdecl webrtc::webrtc_sequence_checker_internal::SequenceCheckerImpl::ExpectationToString(void)const " (?ExpectationToString@SequenceCheckerImpl@webrtc_sequence_checker_internal@webrtc@@qeba?AV?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@xz) [C:\Dev_Env/Project/webrtc-java/webrtc-jni/target/windows-x86_64/webrtc-java.vcxproj]
webrtc.lib(wgc_capture_source.obj) : error LNK2019: unresolved external symbol __imp_DwmGetWindowAttribute referenced in function "public: virtual struct ABI::Windows::Graphics::SizeInt32 __cdecl webrtc::WgcWindowSource::GetSize(void)" (?GetSize@WgcWindowSource@webrtc@@UEAA?AUSizeInt32@Graphics@Windows@ABI@@xz) [C:\Dev_Env/Project/webrtc-java/webrtc-jni/target/windows-x86_64/webrtc-java.vcxproj]
C:\Dev_Env/Project/webrtc-java/webrtc-jni/target/windows-x86_64/Release/webrtc-java.dll : fatal error LNK1120: 2 unresolved externals [C:\Dev_Env/Project/webrtc-java/webrtc-jni/target/windows-x86_64/webrtc-java.vcxproj]

Additional Information：

-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.20348.0 to target Windows 10.0.22631.
-- The C compiler identification is MSVC 19.29.30154.0
-- The CXX compiler identification is MSVC 19.29.30154.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- WebRTC checkout path: C:/Users/18500/webrtc
-- WebRTC checkout branch: branch-heads/5572
-- WebRTC target CPU: x64
-- WebRTC build type: Release
-- WebRTC install path: C:/Users/18500/webrtc/build
-- WebRTC: Compiled version found 'C:/Users/18500/webrtc/src/out/x64/obj/webrtc.lib'
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Dev_Env/Project/webrtc-java/webrtc-jni/target/windows-x86_64

[jetamie]

so cool! When can it be merged into the main branch? @devopvoid

[tlhe]

so cool! When can it be merged into the main branch? @devopvoid
i need it!
thank you

[kinsleykajiva]

Is there any update on this PR ? also is it possible to add custom audio source only e.g mp3 or wav file or other format also with media pause / stop function

[GeorgeBarbosa]

Updates on this one?