RTSP to Spout

[dim111]

Hi!
Need to send RTSP feed to Spout
RTSP url rtsp://admin:admin@2.0.0.201/media/video1, where admin:admin is login and password
Trying to do something like this with no success:
uv -t rtsp://admin:admin@2.0.0.201/media/video1 -d gl:spout

Help!

[MartinPulec]

Hi, as I've mentioned elsewhere, I'll try to look at it but I cannot promise anything now. Just one thing would help me - could you paste the stream SDP? AFAIK it may be grabbed from your stream with FFmpeg:

$ ffplay -v verbose rtsp://admin:admin@2.0.0.201/media/video1
[...]
[rtsp @ 0x7f1558000bc0] SDP:
v=0
o=- 16487424076730988550 16487424076730988550 IN IP4 hostname
s=Unnamed
i=N/A
c=IN IP4 127.0.0.1
[ all remaining lines until new-line ]

This could help me determine, if it is possible to support for that stream with reasonable effort.

Thanks!
Martin

[dim111]

E:\SOFT\IPCAM\ffmpeg-2021-08-04-git-3b298640e1-full_build\bin>ffplay -v verbose rtsp://admin:admin@2.0.0.201/media/video1
ffplay version 2021-08-04-git-3b298640e1-full_build-www.gyan.dev Copyright (c) 2003-2021 the FFmpeg developers
built with gcc 10.3.0 (Rev5, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil 57. 3.100 / 57. 3.100
libavcodec 59. 4.100 / 59. 4.100
libavformat 59. 4.101 / 59. 4.101
libavdevice 59. 0.100 / 59. 0.100
libavfilter 8. 1.103 / 8. 1.103
libswscale 6. 0.100 / 6. 0.100
libswresample 4. 0.100 / 4. 0.100
libpostproc 56. 0.100 / 56. 0.100
Initialized direct3d renderer.
[tcp @ 000001bbb54e1580] Starting connection attempt to 2.0.0.201 port 554
[tcp @ 000001bbb54e1580] Successfully connected to 2.0.0.201 port 554
[rtsp @ 000001bbb54de200] SDP:= 0KB vq= 0KB sq= 0B f=0/0
v=0
o=- 1001 1 IN IP4 2.0.0.201
s=VCP IPC Realtime stream
m=video 0 RTP/AVP 105
c=IN IP4 2.0.0.201
a=control:rtsp://2.0.0.201/media/video1/video
a=rtpmap:105 H264/90000
a=fmtp:105 profile-level-id=4d401f; packetization-mode=1; sprop-parameter-sets=Z01AH5WgFAFumyAAAAMAIAAAB5CA,aO48gA==
a=recvonly
m=application 0 RTP/AVP 107
c=IN IP4 2.0.0.201
a=control:rtsp://2.0.0.201/media/video1/metadata
a=rtpmap:107 vnd.onvif.metadata/90000
a=fmtp:107 DecoderTag=h3c-v3 RTCP=0
a=recvonly

[rtsp @ 000001bbb54de200] setting jitter buffer size to 500 f=0/0
[rtsp @ 000001bbb54de200] setting jitter buffer size to 500 f=0/0
[h264 @ 000001bbb54e3340] Reinit context to 1280x720, pix_fmt: yuvj420p
Input #0, rtsp, from 'rtsp://admin:admin@2.0.0.201/media/video1':
Metadata:
title : VCP IPC Realtime stream
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: h264 (Main), 1 reference frame, yuvj420p(pc, progressive, left), 1280x720, 30 fps, 30 tbr, 90k tbn
Stream #0:1: Data: none
[h264 @ 000001bbbc5cb6c0] Reinit context to 1280x720, pix_fmt: yuvj420p
[ffplay_buffer @ 000001bbb35e7d40] w:1280 h:720 pixfmt:yuvj420p tb:1/90000 fr:30/1 sar:0/1
[auto_scaler_0 @ 000001bbbc82ee40] w:iw h:ih flags:'bicubic' interl:0
[ffplay_buffersink @ 000001bbb558e940] auto-inserting filter 'auto_scaler_0' between the filter 'ffplay_buffer' and the filter 'ffplay_buffersink'
[swscaler @ 000001bbbc4433c0] deprecated pixel format used, make sure you did set range correctly
[auto_scaler_0 @ 000001bbbc82ee40] w:1280 h:720 fmt:yuvj420p sar:0/1 -> w:1280 h:720 fmt:yuv420p sar:0/1 flags:0x4
Created 1280x720 texture with SDL_PIXELFORMAT_IYUV.
20.61 M-V: 0.017 fd= 5 aq= 0KB vq= 221KB sq= 0B f=0/0

[MartinPulec]

Hi, thanks. I've pushed first couple of fixes - authentication works but it is possible that something else doesn't, it is yet WIP. However your SDP helped, I've at least relaxed the hard-coded payload type.

If you could try it out, it would be perfect. Anyways, I'll probably push another set of changes in a next few days. The command above could work, but to be precise, the argument isn't syntactically a URI (you can see -t rtsp:help for more info).

[dim111]

Hi, test results:
C:\Users\Dim\Desktop\RTSPPlayer\UltraGrid-continuous-win64>uv -t rtsp://admin:admin@2.0.0.201/media/video1 -d gl
UltraGrid 1.6+ (tags/continuous rev d584dc2 built Aug 24 2021 17:45:20)

Display device : gl
Capture device : rtsp
Audio capture : none
Audio playback : none
MTU : 9000 B
Video compression: none
Audio codec : PCM
Network protocol : UltraGrid RTP
Audio FEC : none
Video FEC : none

[NAT] Private outbound IPv4 address detected and bound to a non-dynamic port. Consider adding '-N' option for NAT traversal.
GL setup: fullscreen: OFF, deinterlace: OFF
Display initialized-gl
RTSP capture module is most likely broken, please contact ultragrid-dev@cesnet.cz if you wish to use it.
rtsp://admin:admin@2.0.0.201/media/video1

Multithreaded RTP is not recommended for MSW due to internal locking within Winsock implementation.

There is a change. Previously, the program exit with an error message. Something is happening now, but it is impossible to see it.

[dim111]

IP camera connected directly to my PC over LAN cable
Also I have active internet connection over WIFI to my router

[dim111]

I temporary disabled all ethernet adaptesr in my PC except where the camera is connected. Here's a log:

ERROR: connect(fd, (struct sockaddr *) &dst, sizeof dst), (0 - 0)
GL setup: fullscreen: OFF, deinterlace: OFF
Display initialized-gl
RTSP capture module is most likely broken, please contact ultragrid-dev@cesnet.cz if you wish to use it.
rtsp://admin:admin@2.0.0.201/media/video1

Multithreaded RTP is not recommended for MSW due to internal locking within Winsock implementation.

[MartinPulec]

Just to check, does the camera ping? Hoping that it has ping enabled...

[dim111]

C:\Users\Dim\Desktop\RTSPPlayer\UltraGrid-continuous-win64>ping 2.0.0.201

Pinging 2.0.0.201 with 32 bytes of data:
Reply from 2.0.0.201: bytes=32 time<1ms TTL=64
Reply from 2.0.0.201: bytes=32 time<1ms TTL=64
Reply from 2.0.0.201: bytes=32 time<1ms TTL=64
Reply from 2.0.0.201: bytes=32 time=2ms TTL=64

Ping statistics for 2.0.0.201:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 2ms, Average = 0ms

Also I can view stream in ffplay/vlc/gstreamer

[MartinPulec]

Ok, thaks for the info, I'll look at it what could be wrong yet.

[MartinPulec]

Hi, I have an iteration nr. 2 ready - now it should work with Windows, to some extent. I guess that there are still some performance issue to be addressed so it is not necessarily needed to test now, but if you could, you may also find something.

[dim111]

Once again-thank you very much! I am pleased with the minimal delay between the camera stream and its playback. Just in case, I will ask a question - is there a possibility to reduce this delay even more? (some options on the command line that reduce the buffer size) Maybe at the expense of quality? Realtime playback is important for me. I can sacrifice the quality.

[MartinPulec]

I'll take a look. Hard to say now - there are currently some synchronization issues in the RTSP code that may add latency. For that is possible that it will improve after I remove that issues. On the other hand, UltraGrid itself is quite low-latency, there is not much to set. It also depends on properties of the stream - if it uses B-frames it would imply higher latency, but I wouldn't necessarily expect that for a IP camera.

[dim111]

Yes, the delay between the IP camera and its display is 150 ms. This is a great result! I think less is already impossible. However, when converting to spout, another 50 ms is added. I understand that there are no miracles and processing takes time, but I always want to improve the result. Thank you again!

[MartinPulec]

Understood, I've pushed another update. I would not expect to improve the latency much but it should at least improve the CPU load (there was a busy-waiting thread).

[dim111]

Yes. 2% of CPU utilization. Excellent! Thanks

[Allethrium]

Apologies to resurrect this old discussion, but I seem to be having some issues here - FFPLAY works fine;

ffplay -v verbose rtsp://admin:admin@192.168.1.15/1
ffplay version 6.1.1 Copyright (c) 2003-2023 the FFmpeg developers
  built with gcc 14 (GCC)
  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wno-complain-wrong-lang -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld-errors -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --enable-chromaprint --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-lcms2 --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libbs2b --enable-libcodec2 --enable-libcdio --enable-libdrm --enable-libjack --enable-libjxl --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libharfbuzz --enable-libilbc --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-libplacebo --enable-librsvg --enable-librav1e --enable-librubberband --enable-libsmbclient --enable-version3 --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libshaderc --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-avfilter --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libvpl --enable-runtime-cpudetect
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Initialized opengl renderer.
[tcp @ 0x7f5160003780] Starting connection attempt to 192.168.1.15 port 554
[tcp @ 0x7f5160003780] Successfully connected to 192.168.1.15 port 554
[rtsp @ 0x7f5160000c80] SDP:aq=    0KB vq=    0KB sq=    0B f=0/0   
v=0
o=CameraStreaming 3132335948 1236907222000 IN IP4 192.168.1.15
s=10
c=IN IP4 0.0.0.0
t=0 0
a=control:*
m=video 0 RTP/AVP 26
a=control:trackID=0
a=rtpmap:26 JPEG/90000
a=framesize:26 1920-1080
m=audio 0 RTP/AVP 104
a=control:trackID=1
a=rtpmap:104 MPEG4-GENERIC/48000/2
a=fmtp:104 streamtype=5;profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1190

[rtsp @ 0x7f5160000c80] setting jitter buffer size to 5000B f=0/0   
    Last message repeated 1 times
Input #0, rtsp, from 'rtsp://admin:admin@192.168.1.15/1':0B f=0/0   
  Metadata:
    title           : 10
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: mjpeg (Baseline), 1 reference frame, yuvj420p(pc, bt470bg/unknown/unknown, center), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn
  Stream #0:1: Audio: aac (LC), 48000 Hz, stereo, fltp
[ffplay_abuffer @ 0x7f51600a7740] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:stereo
[ffplay_abuffersink @ 0x7f51600c4ec0] auto-inserting filter 'auto_aresample_0' between the filter 'ffplay_abuffer' and the filter 'ffplay_abuffersink'
[auto_aresample_0 @ 0x7f5160053500] ch:2 chl:stereo fmt:fltp r:48000Hz -> ch:2 chl:stereo fmt:s16 r:48000Hz
[ffplay_abuffer @ 0x7f51600c4ec0] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:stereo
[ffplay_abuffersink @ 0x7f5160053500] auto-inserting filter 'auto_aresample_0' between the filter 'ffplay_abuffer' and the filter 'ffplay_abuffersink'
[auto_aresample_0 @ 0x7f51600a7740] ch:2 chl:stereo fmt:fltp r:48000Hz -> ch:2 chl:stereo fmt:s16 r:48000Hz
[ffplay_buffer @ 0x7f5148118900] w:1920 h:1080 pixfmt:yuvj420p tb:1/90000 fr:30/1 sar:1/1
[auto_scale_0 @ 0x7f51481195c0] w:iw h:ih flags:'' interl:0
[ffplay_buffersink @ 0x7f5148118cc0] auto-inserting filter 'auto_scale_0' between the filter 'ffplay_buffer' and the filter 'ffplay_buffersink'
[swscaler @ 0x7f5148127a00] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x7f51481195c0] w:1920 h:1080 fmt:yuvj420p sar:1/1 -> w:1920 h:1080 fmt:yuv420p sar:1/1 flags:0x00000004
[swscaler @ 0x7f5148127a00] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x7f51481195c0] w:1920 h:1080 fmt:yuvj420p sar:1/1 -> w:1920 h:1080 fmt:yuv420p sar:1/1 flags:0x00000004
[swscaler @ 0x7f5148127a00] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x7f51481195c0] w:1920 h:1080 fmt:yuvj420p sar:1/1 -> w:1920 h:1080 fmt:yuv420p sar:1/1 flags:0x00000004
[swscaler @ 0x7f5148127a00] deprecated pixel format used, make sure you did set range correctly
[auto_scale_0 @ 0x7f51481195c0] w:1920 h:1080 fmt:yuvj420p sar:1/1 -> w:1920 h:1080 fmt:yuv420p sar:1/1 flags:0x00000004
Created 1920x1080 texture with SDL_PIXELFORMAT_IYUV.

However UltraGrid is giving me some error message:

UltraGrid.AppImage -t rtsp://admin:admin@192.168.1.15/1:rtp_rx_port=554
UltraGrid 1.9.4 (tags/v1.9.4 rev a3b3cad built Jun 27 2024 07:54:18)

Display device   : none
Capture device   : rtsp
Audio capture    : none
Audio playback   : none
MTU              : 1500 B
Video compression: none
Audio codec      : PCM
Network protocol : UltraGrid RTP
Audio FEC        : none
Video FEC        : none

[rtsp]    cURL V7.68.0 loaded
[rtsp] RTSP/1.0 405 Method Not Allowed
[rtsp]  CSeq: 5
[rtsp] Server: Rtsp Server
[rtsp] Allow: OPTIONS, DESCRIBE, SETUP, PLAY, TEARDOWN, SET_PARAMETER
[rtsp] 
[rtsp] [rtsp_teardown] curl_easy_perform(curl) failed: RTSP CSeq mismatch or invalid CSeq (85)
[rtsp error] could not configure rtsp capture properly, 
		please check your parameters. 
Exiting...

Unable to start video capture device rtsp
Unable to open capture device: rtsp
Exit

I did a few searches and found this extremely old StackOverflow post;

https://stackoverflow.com/questions/18525980/rtsp-client-sending-delimited-data-before-session-has-been-completely-set-up

The RTSP server device is a PTZ Optics HD Box camera (PoE) - it might just be that it's putting out a bad stream.

My apologies in advance if this is a poor question, and I have missed something simple.

[MartinPulec]

#400