HomeKit Secure Video

[Supereg]

♻️ Current situation

PR #904 originally proposed to implement HomeKit Secure Video support in HAP-NodeJS. The PR of the original author is considered stale since a few weeks now.

💡 Proposed solution

This PR replaces #904 and realizes the requested changes, building upon the existing PR and moving it from a rather experimental state to a state where the API can be considered stable. The implementation itself was extended to accommodate for much more extensive error and edge case handling.

To highlight some advancements to the previous PR (from a API interface perspective):

• The CameraControllerOptions were refactored:
  • Added detailed documentation to all fields
  • EventTriggerOptions are now derived automatically based on the CameraController or DoorbellController configuration
  • Added proper typing to all fields
  • A MotionSensor (and a OccupancySensor) can be either be enabled by flag or passed as an instance. The second option can be used to support people in migrating from preexisting MotionSensor instances to managing them with the CameraController.
• Rethought the CameraRecordingDelegate interface:
  • Added documentation.
  • Introduces calls to be notified of changes to the SelectedRecordingConfiguration and the Active state. Second one is used to enable or disable the prebuffer.
  • Added dedicated calls to close the streaming request. Previously this was done through AsyncGenerator.throw which wasn't ideal as (a) one couldn't transport additional context or information, (b) it is a feature which isn't really intuitive nor documented well, (c) it is slow, as the generator is only notified on the next yield and (d) it is easy to make mistakes, resulting in a potential application crash if done wrong. The new API calls are more intuitive and allow to instantly abort the generator.
  • Properly enforce a single recording stream at a time.
  • Added support to mark a recording stream as endOfStream.
• SnapshotRequest are now checked if they are actually allowed in the current state (Previously this was expected to be done by every individual developer).
• Implemented state handling. The HSV camera will save and restore its configuration state across reboots.
• Fixed some bugs in service and state handling.

⚙️ Release Notes

This section is addressed once merged, such that we can link to docs pages.

➕ Additional Information

• Short spec writeup: https://github.com/Supereg/secure-video-specification. Though inline docs can be considered more detailed and contain a bit newer information from my latest discoveries and tests.

Testing

The PR adds some minimal unit tests for the CameraController.
Further, it implements a minimal CameraRecordingDelegate in the IPCamera_accessory.ts example, providing a rough overview on how to use the API. But most importantly this serves as a test vector to test against Apples HomeKit Controller implementation.

Reviewer Nudging

Also applying to users interesting in implementing HomeKit Secure Video.
In any case I would propose that you carefully read through the inline documentation provided with user facing interfaces. Additionally, have a look at the example HAP-NodeJS accessory implementation, giving a minimal but working recording delegate.

[coveralls]

Pull Request Test Coverage Report for Build 1659769639

• 343 of 648 (52.93%) changed or added relevant lines in 13 files are covered.
• 5 unchanged lines in 3 files lost coverage.
• Overall coverage increased (+1.6%) to 50.541%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/lib/Service.ts	0	1	0.0%
src/lib/model/ControllerStorage.ts	0	1	0.0%
src/lib/controller/AdaptiveLightingController.ts	0	2	0.0%
src/lib/datastream/DataStreamParser.ts	0	2	0.0%
src/lib/Characteristic.ts	4	7	57.14%
src/lib/datastream/DataStreamServer.ts	5	9	55.56%
src/lib/Accessory.ts	2	7	28.57%
src/lib/controller/RemoteController.ts	3	9	33.33%
src/lib/controller/DoorbellController.ts	13	30	43.33%
src/lib/camera/RTPStreamManagement.ts	20	42	47.62%

Files with Coverage Reduction	New Missed Lines	%
src/lib/Accessory.ts	1	41.08%
src/lib/camera/RTPStreamManagement.ts	2	45.42%
src/lib/controller/CameraController.ts	2	41.04%

Totals
Change from base Build 1659543468:	1.6%
Covered Lines:	5677
Relevant Lines:	9990

---

💛 - Coveralls

[seydx]

Works great. Thx @Supereg

Possibly worth mentioning:

• If already existing cameras in Apple Home now become HSV capable, it may happen that they do not record immediately. Turning "Stream & Recording" within Apple Home off and on several times seems to solve the problem. If that doesn't help, restart the hub.

• If a movement is triggered "manually" (e.g. via a switch), don't be surprised that you don't get a notification. HSV seems to analyze the recording (?) and only if there really is motion, you get a notification or you can see the recording.

• If vcodec = copy is used via FFmpeg, you should make sure that the camera settings are suitable for HSV. FPS should be > 20 and Key Frame Interval = FPS x 4 (e.g. 20 x 4 = 80).

My camera settings:

FPS: 25
Key Frame Interval: 100
Resolution: 1920x1080
Bitrate: 2048 kbps (CBR)

My implementation:

https://github.com/seydx/homebridge-camera-ui/blob/hsv/src/accessories/camera.js
https://github.com/seydx/homebridge-camera-ui/blob/hsv/src/services/recording.service.js

[Supereg]

Maybe to give some context to these points:

If already existing cameras in Apple Home now become HSV capable, it may happen that they do not record immediately.

Recording of newly added HSV camera is always turned off by default (same behavior as for certified cameras). So the step of manually enabling recording is expected and required.

HSV seems to analyze the recording (?) and only if there really is motion, you get a notification or you can see the recording.

Every HSV camera has two major options controlling these behaviors. The recording options (which only appear if recording is enabled) which control when a recording is saved (e.g. checking for people, animals etc; also consider that activity zones might influence this behavior). And secondly you have the Activity Notifications setting in your Notifications settings which is off by default. With this toggle you can selected to be notified when A Clip is Recorded, Any Motion is Detected or Specific Motion is Detected (last options ties back to the recording options).
I would assume your findings are probably related to those options?

If vcodec = copy is used via FFmpeg, you should make sure that the camera settings are suitable for HSV. FPS should be > 20 and Key Frame Interval = FPS x 4 (e.g. 20 x 4 = 80).

This is really dependent on the camera. In general, it is not advised to use vcodec=copy as your cameras video configuration is most likely different to the recording options HomeKit selects and expects (therefore HomeKit might reject the recordings).
What you are seeing here is the enforcement of "every fragment starts with a key frame" (assuming you have a fragment size of 4000ms configured). Therefore you only reach this requirement (considering your fixed key fragment length) with your fixed camera setting when using a fps count of 20. This obviously doesn't hold for everyone.