Fix RTSP decoding errors in 1.26.4 (addresses #221)

[ebarnard]

Fixes #221. ZoneMinder now supports H.264 and MPEG4 RTSP streams.

[kylejohnson]

When I get home I'll test this. Please don't merge until I do, to help with distributing the bounty.

P.s. Thank you!

Ed Barnard notifications@github.com wrote:

Fixes #221. ZoneMinder now supports H.264 and MPEG4 RTSP streams.
You can merge this Pull Request by running:

git pull https://github.com/ebarnard/ZoneMinder rtsp-fix

Or you can view, comment on it, or merge it online at:

#259

-- Commit Summary --

• RtpSource can now distinguish between payload types and handles some
  properly.
  • Fixed bug in mpeg4 depacketizer and RTCP implementation.
  • The last packet of a fragmented mpeg4 frame is no longer rejected.
  • H264 SPS and PPS frames are now handled correctly
  • Adds support more H264 NAL types and tidies function
  • Save a couple of cpu cycles.

-- File Changes --

M src/zm_remote_camera_rtsp.cpp (25)
M src/zm_remote_camera_rtsp.h (2)
M src/zm_rtp_ctrl.cpp (7)
M src/zm_rtp_source.cpp (105)
M src/zm_rtp_source.h (14)
M src/zm_rtsp.cpp (14)

-- Patch Links --

https://github.com/ZoneMinder/ZoneMinder/pull/259.patch
https://github.com/ZoneMinder/ZoneMinder/pull/259.diff

[kylejohnson]

I'm seeing this in my log files while accessing the RTSP stream on my Axis Q6032-E camera

You should be able to access this camera at rtsp://public:public@camera.gnulnx.net:554/axis-media/media.amp

[kylejohnson]

[ebarnard]

If I open the camera in vlc it only displays one or a few frames and then freezes, and eventually crashes vlc. Are you able to view the stream in anything? I'm thinking its either a camera bug or an ffmpeg (h264 decoder) one. Which version are you building against?

[kylejohnson]

Hey there,

I'm having some very poor internet at the moment - access the RTSP
stream on the LAN in VLC works very well and doesn't freeze. I'm using
ffmpeg version N-58415-gf63ee2a

On 12/2/2013 2:00 PM, Ed Barnard wrote:

If I open the camera in vlc it only displays one or a few frames and
then freezes, and eventually crashes vlc. Are you able to view the
stream in anything? I'm thinking its either a camera bug or an ffmpeg
(h264 decoder) one. Which version are you building against?

—
Reply to this email directly or view it on GitHub
#259 (comment).

[ebarnard]

I'm getting a decent bitrate and vlc seems to be leaking memory. I'll try with zm and ffmpeg git head and see what happens.

[kylejohnson]

I'm also still having issues with another camera - a Grandstream. I can access the following URL in VLC fine, but it doesn't work in ZM: rtsp://71.251.199.163:554

[ebarnard]

Sorry, didn't look at your config. It should be RTP/RTSP and don't specify the port in the hostname. That works fine for me. No idea what was up with vlc.

[kylejohnson]

I messed up there - I didn't put the port in the hostname, was just copying and pasting from VLC. Let me try with RTP/RTSP though.

[kylejohnson]

Alright, I'm getting a monitor stream, but I'm still seeing this in the logs:

Restarting capture daemon for Monitor-1, time since last capture 7 seconds (1386012482-1386012475)

[ebarnard]

Is your zm server on the same lan as the camera? My first thought is that its simply providing frames to trigger the zmc anti-freeze function (ie kill it).
I'm guessing the other camera has never worked with zm. For some reason it's reporting a false video and audio stream which is tripping up the very simple video stream selection logic. That should be an easy fix once I fix my development machine.

[kylejohnson]

Yup - my ZM server is on the same network as the first camera that I pointed you to (the axis).

[ebarnard]

How long is it between zmc restarts?

[kylejohnson]

The logs are showing it at 7, 15, 21 and 32 seconds

[ebarnard]

While the second camera was causing zm to raise an exception due to an unexpected stream type it also doesn't seem to be standards compliant. When sent a SETUP command it fails to return a Transport parameter. It's possible zm is doing something wrong although all of the requests seem fine. I think vlc is just a bit more flexible. I think this is something better sorted out once we have new rtsp infrastructure in place (probably using curl).

I'll have a look at the first one tomorrow.

[ebarnard]

It's definately zm. Should have looked at a pcap earlier. I think I'll just rewrite to use curl. It shouldn't take too long.

[POKKAHOH]

Really zm is restarted (see log`s in forum http://ftp.cinepost.ru/temp/from_rd/tcp_dump.zip (~100M)). But the picture shows :) It's not ideal, but agree that better than working through ffmpeg stream.

@ebarnard - see stream rtsp://ipcam.zapto.org/cam/realmonitor?channel=1&subtype=0&authbasic=YWRtaW46YWRtaW4=

Vlc report - two video stream: #0 codec H264-MPEG-4 AVC (part10) (H264) and #2 704*480 PYUV4:2:0 codec H264-MPEG-4 AVC (part10) (H264)
#1 and #3 - sound

zm show only frame differences (as i think - use stream #0).

[ebarnard]

I've been quite busy over the last coupe of days but just a heads up on what I'm doing. Having rewritten zm to use curl for rtsp I then discovered that curl doesn't yet support authentication for rtsp. In fact their source includes the helpful comments

/* TODO: auth? */
/* TODO: proxy? */

which is a little frustrating.

I'm fairly certain none of the cams above worked in 1.26.3 so my suggestion would be to merge this which would fix the regression and enable a few more cameras to work. I'll then rewrite to use live555. It turns out it shouldn't be too hard to patch for ipv6 if it's wanted (someone did it in 2011) although the rtsp 1.0 spec doesn't actually allow ipv6 (rtp does but if you can access the rtsp server to get the session descriptor then you can just use ipv4 anyway). I think that's a lot easier than trying to debug code that only exhibits flaws with certain cameras.

[kylejohnson]

I'd like @mastertheknife the chime in before I merge, but otherwise I agree with you. He's also ran into the curl issues with RTSP.

[POKKAHOH]

@ebarnard exactly - none of the RTP/RTSP cams worked till 1.26.3
Now we can use it "as is" (with errors or warnings) In future will rewrite it (use live555 or make authentication for curl).
RTSP authentication is not hard to do (send DESCRIBE without Autorisation , recive ansver 401 instead 200, again send DESCRIBE with Autorisation field with base64 encoded loogin:password (":" obligatory). All rest RTSP packets sends with Authorisation field)
It`s same as HTTP/1.1 autorisation.

[ebarnard]

@kylejohnson
Great. Although now I wish I'd done a bit of research beforehand.

@POKKAHOH
I agree basic auth would be easy to implement but you'd also have to support digest and in theory the myriad of other HTTP/1.1 authentication types. Also curl doesn't provide any easy way to support rtsp over http as it doesn't give you an easy way to provide custom transport.

[connortechnology]

Please try my crappy camera (Internet Eye MPEG4).

at www.connortechnology.com:554

Not sure if the path matters. ffmpeg works without a path, but from what I've read, you can use /cam1/mpeg4 or /cam1/h264 to get different stream types.

I have not got this working with the rtsp code in zm. However ffmpeg works.

admin:password@

[ebarnard]

@connortechnology
I used that as one of my test cams. It works fine on mpeg4. Didn't know it did h264 so can't speak to that.

[appnouvea]

i'm a newbie and was wondering if anyone can try to view my camera in ZM? I can't see video in ZM, only in VLC.
rtsp://ipcam.zapto.org/cam/realmonitor?channel=1&subtype=0&authbasic=YWRtaW46YWRtaW4=

(admin:admin)

The logs show a ton of "Error while decoding frame 0" messages - see below.
My Camera is a Dahua IPC-HDW2100 with H264 stream, 704X480, 15fps, VBR, Quality 4, bitrate 640, I frame interval 30.

tnx

11/30/13 17:17:08.391631 zmc_m5[2009].ERR-zm_remote_camera_rtsp.cpp/216 [Error while decoding frame 0]
11/30/13 17:17:08.391729 zmc_m5[2009].ERR-zm_remote_camera_rtsp.cpp/217 [256: 7c 81 9b 18 5f c9 25 62 aa 13 e9 7b b8 9d f0 97 60 3f 22 fa 29 bc 22 60 59 7b 20 43 ca 50 7b 62 53 f0 fc 7c 45 75 20 fe fe a1 b0 da 19 43 c2 f0 8c 64 0d bc 03 1a f1 d3 2f 4f 80 b7 e0 36 8d fc

[chippy99]

I've been running it live for 5 days now with no problems. Only using mpeg4, but did test briefly with h.264 and it worked ok

[POKKAHOH]

@connortechnology - time out :)
2013-12-06 10:40:15.972855 zmc_m6 10210 ERR Failed to pre-capture monitor 6 (0/1) zmc.cpp 260
2013-12-06 10:40:06.908062 zmc_m6 10378 ERR RTP timed out zm_rtp_data.cpp 89
2013-12-06 10:40:02.407320 zmc_m6 10210 INF Starting Capture zmc.cpp 195

in vlc sometimes see patterned glass .... sometimes - gray screen

@appnouvea with ebarnard`s patch - rtsr log look like this
1386313502.867313 zmc_m5 14015 0 INF Starting Capture zmc.cpp 195
1386313504.276145 zmc_m5 14031 -1 WAR The device is using a codec that may not be supported. Do not be surprised if things don't work.

If use ffmpeg stream - error

1386314902.867313 zmc_m5 12583 0 INF Starting Capture zmc.cpp 195
1386314902.913612 zmc_m5 12583 0 INF Priming capture from rtsp://ipcam.zapto.org/cam/realmonitor?channel=1&subtype=0&authbasic=YWRtaW46YWRtaW4= zm_ffmpeg_camera.cpp 112
1386314914.252233 zmc_m5 12583 -3 FAT swscale does not support the codec format: ÿÿÿÿ

vlc play this stream... format is mjpg

[connortechnology]

@POKKAHOH the patterned glass is what you should be seeing or the top of my head :) Can I ask what other settings you are using? cuz I can only get it to work with ffmpeg. SO I use RTSP for protocol, I have tried all of the Remote methods, and both no path, and various paths.

[connortechnology]

@POKKAHOH nevermind, apparentlly I had it "disabled" bad UI! It works.

[kylejohnson]

@ebarnard Any idea why I'm getting those zmc restarts?

[knight-of-ni]

Adding my test results to the mix. I used the camera @appnouvea posted.

Before applying the rtsp fix:

Image appears fine, but the logs contain lots of warnings about incomplete frames or perhaps frames that are out of order:

Once I apply the rtsp fix, the image becomes almost entirely gray:

The log files are similar:

I'm not sure what to make of this. The image is certainly worse than before, but I am not sure I am receiving a good stream from the camera in the first place. VLC is unable to play the stream.

[POKKAHOH]

@connortechnology - perfectlly...

@kylejohnson - i think zmc restarts on timeout (send TEARDOWN command)

@knnniggett - seems appnouvea`s cam send pure mjpeg (my vlc(v2.0.0) play it)

[ebarnard]

@kylejohnson zmwatch.pl restarts zmc if it captured an image in the last 5 seconds even if packets are still arriving (so not necessarily an rtp timeout). As for why it's not capturing frames I'm really not sure but I haven't managed to successfully get a stream from that camera with vlc or ffmpeg (different rtsp/rtp backend).

@knnniggett I get h264 from appnouvea's cam. It looks like keyframes are being dropped (I now see what you meant by only showing differences @POKKAHOH). I think that's because my patch is stricter on how the end of fragmented frame sequences are marked. I'll see if I can get it working tomorrow.

[ebarnard]

@appnouvea @knnniggett Latest commit fixes that camera.

[appnouvea]

@ebarnard this is great news (i'm assuming you meant my camera - the link i published above). I'm working on building your fork and will report back with the results. Thanks

[knight-of-ni]

@ebarnard Thanks for your work. I can now view the stream from @appnouvea camera with the latest changes.

However, the log is still filling up quite rapidly with "packet in sequence" & "discarding frame" warnings.

Is this an indication of:
A) That the rtsp code needs to be tweaked further?
or
B) The desired result when in fact the camera is not telling us the end of the fragment or is otherwise not sending us the right information.

If B, then I think we may need to look into creating a user-toggle-able option to suppress these warnings. Trying to troubleshoot other problems with all those warnings in the logs will make life difficult.

Thoughts?

[ebarnard]

@appnouvea Yeah I meant yours.

@knnniggett I'm pretty sure it's B as the reason it didn't work originally is it wasn't conforming to rtp specs. I agree about the warnings although for the moment you can filter by component. In fact it might be better to improve the filtering system to be able to selectively exclude components than to add the ability to disable a specific warning. Even if it does turn out to be A the longer term plan is to move to using something like live555 for rtsp/rtp once we've fixed the regressions in 1.26.4.

[appnouvea]

@ebarnard - works like a charm. It seems like all ZM features are working well - Thanks a lot for the fix!

I still get some errors though - see attached image, just wanted to find out if this is normal or there is something i can do about it.

[kylejohnson]

Let us keep this conversation going. As I see, here are the current concerns with this pull:

1. Scoping. This pull is actually a regressing fix, not a cure-all RTSP fix. We should rename this pull, and the referenced issue RTSP decoding errors in 1.26.4 [$450 awarded] #221, to reflect that.
2. Open another issue to track the additional cameras which are not covered under this regression fix.
3. Logs. Even for the cameras which are now working with this pull, the log files are filling up with various messages. We need a way of suppressing or otherwise handling this, as the log table will become too large, and will grind the server to a hult.
4. zmc. With at least my reference Axis camera, zmc is continually restarting. @mastertheknife seemed to have some information on this.

Did I miss anything else?

[knight-of-ni]

@kylejohnson That list is all that I am aware of.

The following pertains to item 3.
This is not the first time we have run into a camera that "works" but leaves a lot of warnings in the logs, nor do I expect this to be the last time we run into this. Recall that several Foscam cameras caused the jpeg library to complain a lot, and we silenced that warning at the time.

@kylejohnson it sounds like we are on the same page.
Agree that we should discuss a better way to handle these. Perhaps we should create a new issue for this as well. I don't like specifically disabling warnings, but on the other hand I'm not sure how far we should go redesigning things to make the logs more flexible... especially if these changes become irrelevant with the new UI.

And there I just reminded myself a second time today that I should start testing the new UI.

[kylejohnson]

Quick note from the mobile, I bring up suppressing these messages as they have more than once made my system completely unusable by filling up the logs table and eating up ram and disk space. Easy to happen of you don't know better.

I would suggest a 'logs' tab in the config, with one option "do not log $these warnings'

Andrew Bauer notifications@github.com wrote:

@kylejohnson That list is all that I am aware of.

The following pertains to item 3.
This is not the first time we have run into a camera that "works" but
leaves a lot of warnings in the logs, nor do I expect this to be the
last time we run into this. Recall that several Foscam cameras caused
the jpeg library to complain a lot, and we silenced that warning at the
time.

@kylejohnson it sounds like we are on the same page.
Agree that we should discuss a better way to handle these. Perhaps we
should create a new issue for this as well. I don't like specifically
disabling warnings, but on the other hand I'm not sure how far we
should go redesigning things to make the logs more flexible...
especially if these changes become irrelevant with the new UI.

And there I just reminded myself a second time today that I should
start testing the new UI.

---

Reply to this email directly or view it on GitHub:
#259 (comment)

[POKKAHOH]

My opinion #221 (comment)

[ebarnard]

@kylejohnson I'd suggest moving all those warnings to debug messages and then have an overall warning such as "Dropped 50 bad frames out of 300" similarly to the fps message. I just can't think of a situation other than debugging when you'd want all those messages and if you were debugging you'd most likely want all the other debug messages as well.

@POKKAHOH I've finally got round to getting my libvlc camera to work. It's the libvlc branch in my repo. At the moment it only works consistently with rgb32 colourspace and I've only tested building with cmake although it should build with autotools.