├── .github
└── FUNDING.yml
├── .gitignore
├── LICENSE
├── README.md
└── examples
├── README.md
├── chunks
├── mediaInitialization.mp4
├── sequence2_1.mp4
├── sequence2_2.mp4
├── sequence3_1.mp4
├── sequence3_2.mp4
├── sequence4_1.mp4
└── sequence4_2.mp4
├── fragments
├── mediaInitialization.mp4
├── sequence2.mp4
├── sequence3.mp4
└── sequence4.mp4
└── fully-assembled.mp4
/.github/FUNDING.yml:
--------------------------------------------------------------------------------
1 | github: Supereg
2 |
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/.gitignore
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2020 Andi
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # HomeKit Secure Video Unofficial Specification - R3
2 |
3 | ## 1. Overview
4 |
5 | ### Service Definitions
6 |
7 | #### Cameras
8 |
9 | HomeKit Secure-Video cameras need to expose the same services as normal cameras with the following changes:
10 |
11 | * Every `RTPStreamManagement` service must add the `Active` characteristic
12 | (This is used to indicate that the camera is fully turned off)
13 | * The required amount of `RTPStreamManagement` services was dropped from two to one.
14 | * The `MotionSensor` service is required (to indicate movement and thus a start and stop of a recording)
15 | * The `CameraOperatingMode` service is required
16 | * The `DataStreamManagement` service is required (to initiate HomeKit Data Stream communication)
17 | * The `CameraEventRecordingManagement` service is required. It needs to link to the `MotionSensor` and `DataStreamManagement` service
18 |
19 | If `MotionSensor` or `OccupancySensor` are added, they must expose the `Active` characteristic.
20 |
21 | #### Doorbells
22 |
23 | HomeKit Secure-Video doorbells need to expose the same services as both doorbells and secure-video cameras.
24 |
25 | ### Active states
26 |
27 | Every Secure-Video enabled camera can be set to four different states: `Off`, `Detect Activity`, `Stream`
28 | and `Stream & Allow Recording`.
29 | Depending on the state the following `Active` characteristics for the given services are set.
30 |
31 | | Camera-States | RTPStreamManagement `Active` | CameraOperatingMode `HomeKitCameraActive` | CameraEventRecordingManagement `Active` |
32 | | :----------------------: | :---: | :---: | :---: |
33 | | Off | false | false | false |
34 | | Detect Activity | false | true | false |
35 | | Stream | true | true | false |
36 | | Stream & Allow Recording | true | true | true |
37 |
38 | ### Recording requirements
39 |
40 | * The camera needs so support basic motion detection
41 | * The camera needs to support encoding, buffering and transmitting of fragmented MP4 - Pretty similar to Section 3.3
42 | [RFC 8216](https://tools.ietf.org/html/rfc8216#section-3.3) (HLS). The RFC refers to
43 | [ISO/IEC 14496-12 ISO base media file format](https://mpeg.chiariglione.org/standards/mpeg-4/iso-base-media-file-format/text-isoiec-14496-12-5th-edition).
44 | How exactly the data is transmitted can be read
45 | in the [HDS Packet Formats](#4-homekit-data-stream-packet-formats) section and the [Flow of events](#6-flow-of-events)
46 | section.
47 | * Supported video codecs are: h.264, h.265 (possibly)
48 | * Supported audio codecs are: AAC-LC, AAC-ELD
49 | * Typical resolutions one might support (a camera might support more):
50 | * 640x480
51 | * 1024x768
52 | * 1280x960
53 | * 1600x1200
54 | * 2048x1536
55 | * 640x360
56 | * 1280x720 (Mandatory)
57 | * 1920x1080 (Mandatory, default at 24 or 30 fps)
58 | * 3840x2160
59 | * Frame rates:
60 | * 15 fps (Mandatory)
61 | * 24 fps
62 | * 30 fps (One of 24 fps or 30 fps is mandatory)
63 |
64 |
65 | ## 2. Services
66 |
67 | ### 2.1 CameraOperatingMode
68 |
69 | | Property | Value |
70 | | ------------------------- | ------------------------------------ |
71 | | UUID | 0000021A-0000-1000-8000-0026BB765291 |
72 | | Type | public.hap.service.camera-operating-mode |
73 | | Required Characteristics | [3.2 EventSnapshotActive](#32-eventsnapshotsactive)
[3.3 HomeKitCameraActive](#33-homekitcameraactive)
[3.6 PeriodicSnapshotsActive](#36-periodicsnapshotsactive) |
74 | | Optional Characteristics | [3.4 ManuallyDisabled](#34-manuallydisabled)
[3.5 NightVision](#35-nightvision)
[3.12 ThirdPartyCameraActive](#312-thirdpartycameraactive)
[3.1 CameraOperatingModeIndicator](#31-cameraoperatingmodeindicator)
[3.13 Diagonal Field of View](#313-diagonalfieldofview) |
75 |
76 | ### 2.2 CameraEventRecordingManagement
77 |
78 | | Property | Value |
79 | | ------------------------- | ------------------------------------ |
80 | | UUID | 00000204-0000-1000-8000-0026BB765291 |
81 | | Type | public.hap.service.camera-recording-management |
82 | | Required Characteristics | Active
[3.8 SupportedCameraRecordingConfiguration](#38-supportedcamerarecordingconfiguration)
[3.9 SupportedVideoRecordingConfiguration](#39-supportedvideorecordingconfiguration)
[3.10 SupportedAudioRecordingConfiguration](#310-supportedaudiorecordingconfiguration)
[3.11 SelectedCameraRecordingConfiguration](#311-selectedcamerarecordingconfiguration)
[3.7 RecordingAudioActive](#37-recordingaudioactive) |
83 |
84 | ## 3. Characteristics
85 |
86 | ### 3.1 CameraOperatingModeIndicator
87 |
88 | This characteristic indicates if the camera LED, which shows the current state of the camera (see [states](#active-states)),
89 | should be turned on. Controlled by "Camera status light" setting in the Home App.
90 |
91 | | Property | Value |
92 | | ------------------------- | ------------------------------------ |
93 | | UUID | 0000021D-0000-1000-8000-0026BB765291 |
94 | | Type | public.hap.characteristics.camera-operating-mode-indicator |
95 | | Permissions | Paired Read, Paired Write, Notify, Timed Write |
96 | | Format | bool |
97 | | Valid Values | 0 - "Hardware LED is disabled"
1 - "Hardware LED is enabled" |
98 |
99 | ### 3.2 EventSnapshotsActive
100 |
101 | This characteristic indicates if the option _"Camera Settings" -> Notifications -> "Allow Snapshots in Notifications"_
102 | is turned on. If this option is turned on, a notification from this camera sent to anyone in this home, will include
103 | a snapshot of the motion or activity.
104 |
105 | | Property | Value |
106 | | ------------------------- | ------------------------------------ |
107 | | UUID | 00000223-0000-1000-8000-0026BB765291 |
108 | | Type | public.hap.characteristics.event-snapshots-active |
109 | | Permissions | Paired Read, Paired Write, Notify, Timed Write |
110 | | Format | bool |
111 | | Valid Values | 0 - "Snapshots in notifications are turned off"
1 - "Snapshots in notifications are turned on" |
112 |
113 | ### 3.3 HomeKitCameraActive
114 |
115 | This characteristic indicates if the camera should detect activity (Unsure if activity just means motion detection or
116 | also button presses for doorbell accessories).
117 |
118 | | Property | Value |
119 | | ------------------------- | ------------------------------------ |
120 | | UUID | 0000021B-0000-1000-8000-0026BB765291 |
121 | | Type | public.hap.characteristics.homekit-camera-active |
122 | | Permissions | Paired Read, Paired Write, Notify, Timed Write |
123 | | Format | bool |
124 | | Valid Values | 0 - "Activity detection should not be enabled"
1 - "Activity detection should be enabled" |
125 |
126 | ### 3.4 ManuallyDisabled
127 |
128 | This characteristic indicates if the camera was manually turned off, for example using a physical switch on the camera.
129 |
130 | | Property | Value |
131 | | ------------------------- | ------------------------------------ |
132 | | UUID | 00000227-0000-1000-8000-0026BB765291 |
133 | | Type | public.hap.characteristics.manually-disabled |
134 | | Permissions | Paired Read, Notify |
135 | | Format | bool |
136 | | Valid Values | 0 - "Camera is not manually disabled"
1 - "Camera was manually disabled" |
137 |
138 | ### 3.5 NightVision
139 |
140 | _This characteristic is already present in the current HAP spec_
141 | This characteristic indicates if automatic night vision should be turned on.
142 |
143 | | Property | Value |
144 | | ------------------------- | ------------------------------------ |
145 | | UUID | 0000011B-0000-1000-8000-0026BB765291 |
146 | | Type | public.hap.characteristics.night-vision |
147 | | Permissions | Paired Read, Paired Write, Notify |
148 | | Format | bool |
149 | | Valid Values | 0 - "Disable night-vision mode"
1 - "Enable night-vision mode" |
150 |
151 | ### 3.6 PeriodicSnapshotsActive
152 |
153 | Exact behaviour unclear. Seems to be always set to `true` regardless of anyone viewing periodic snapshots or not.
154 |
155 | | Property | Value |
156 | | ------------------------- | ------------------------------------ |
157 | | UUID | 00000225-0000-1000-8000-0026BB765291 |
158 | | Type | public.hap.characteristics.periodic-snapshots-active |
159 | | Permissions | Paired Read, Paired Write, Notify, Timed Write |
160 | | Format | bool |
161 |
162 | ### 3.7 RecordingAudioActive
163 |
164 | This characteristic indicates if recordings should include audio.
165 |
166 | | Property | Value |
167 | | ------------------------- | ------------------------------------ |
168 | | UUID | 00000226-0000-1000-8000-0026BB765291 |
169 | | Type | public.hap.characteristics.recording-audio-active |
170 | | Permissions | Paired Read, Paired Write, Notify, Timed Write |
171 | | Format | uint8 |
172 | | Valid Values | 0 - "Audio should not be included in recordings"
1 - "Audio recording is active" |
173 |
174 | ### 3.8 SupportedCameraRecordingConfiguration
175 |
176 | | Property | Value |
177 | | ------------------------- | ------------------------------------ |
178 | | UUID | 00000205-0000-1000-8000-0026BB765291 |
179 | | Type | public.hap.characteristics.supported-camera-recording-configuration |
180 | | Permissions | Paired Read, Notify |
181 | | Format | tlv8 |
182 |
183 | A read request on this characteristic returns the following structure:
184 |
185 | ##### Supported Camera Recording Configuration:
186 |
187 | | Type | Name | Format | Description |
188 | | --- | --- | --- | --- |
189 | | 1 | Prebuffer length | 4 | Size of the prebuffer in milliseconds.
It must be at least 4000ms.
(typical encountered values: 4000ms, 8000ms) |
190 | | 2 | Event Trigger Options | 8 | Bitmask of trigger types:
0x01 - Motion
0x02 - Doorbell |
191 | | 3 | Media Container Configurations | N | List of supported media container configurations.
Most cameras out there do only expose one entry. |
192 |
193 | ##### Media Container Configuration:
194 |
195 | | Type | Name | Format | Description |
196 | | --- | --- | --- | --- |
197 | | 1 | Media Container Type | 1 | Container types:
0 - Fragmented MP4 |
198 | | 2 | Media Container Parameters | N | Media container parameters |
199 |
200 | ##### Media Container Parameters:
201 |
202 | | Type | Name | Format | Description |
203 | | --- | --- | --- | --- |
204 | | 1 | Fragment Length | 4 | Length of one mp4 fragment in milliseconds
(typically 4000ms) |
205 |
206 | ### 3.9 SupportedVideoRecordingConfiguration
207 |
208 | | Property | Value |
209 | | ------------------------- | ------------------------------------ |
210 | | UUID | 00000206-0000-1000-8000-0026BB765291 |
211 | | Type | public.hap.characteristics.supported-video-recording-configuration |
212 | | Permissions | Paired Read, Notify |
213 | | Format | tlv8 |
214 |
215 | The value of this characteristic is a TLV8-encoded list of supported video codecs:
216 |
217 | ##### Supported Video Recording Configuration:
218 |
219 | | Type | Name | Format | Description |
220 | | --- | --- | --- | --- |
221 | | 1 | Codec Configuration | N | Codec information and the configurations supported for the codec
There is one TLV of this type per supported codec |
222 |
223 | ##### Video Codec Configuration:
224 |
225 | | Type | Name | Format | Description |
226 | | --- | --- | --- | --- |
227 | | 1 | Codec | 1 | Type of video codec:
0 - H.264
1 - H-265 |
228 | | 2 | Video Codec Parameters | N | Video Codec specific parameters |
229 | | 3 | Video Attributes | N | Video Attributes supported for the codec (tlv list) |
230 |
231 | Video Codec Configuration TLV contains exact one tlv of 'Video Codec Parameters' and one entry of
232 | 'Video Attributes' per supported resolution/frame rate combination.
233 |
234 | ##### Video Codec Parameters:
235 |
236 | | Type | Name | Format | Description |
237 | | --- | --- | --- | --- |
238 | | 1 | ProfileID | 1 | List of supported H.264 profiles (tlv list is separated by empty tlvs):
0 - Baseline Profile
1 - Main Profile
2 - High Profile |
239 | | 2 | Level | 1 | List of supported H.264 levels (tlv list is separated by empty tlvs):
0 - 3.1
1 - 3.2
2 - 4 |
240 | | 3 | Bitrate | 4 | Only present in the Selected Camera Recording Configuration request:
Selected video bitrate. Typically, secure video requests 2000kbps when face recognition is enabled, and 800kbps otherwise. |
241 | | 4 | iFrame_Interval | 4 | Only present in the Selected Camera Recording Configuration request:
Selected key frame interval in milliseconds. Typically 4000ms. Seems to be the same value as the fragment length. So every mp4 fragment MUST begin with a keyframe |
242 |
243 | ##### Video Codec Attributes:
244 |
245 | | Type | Name | Format | Description |
246 | | --- | --- | --- | --- |
247 | | 1 | Image width | 2 | Image width in pixels |
248 | | 2 | Image height | 2 | Image height in pixels |
249 | | 3 | Frame rate | 1 | Maximum frame rate |
250 |
251 | ### 3.10 SupportedAudioRecordingConfiguration
252 |
253 | | Property | Value |
254 | | ------------------------- | ------------------------------------ |
255 | | UUID | 00000207-0000-1000-8000-0026BB765291 |
256 | | Type | public.hap.characteristics.supported-audio-recording-configuration |
257 | | Permissions | Paired Read, Notify |
258 | | Format | tlv8 |
259 |
260 | The value of this characteristic is a TLV8-encoded list of supported audio codecs:
261 |
262 | ##### Supported Audio Recording Configuration:
263 |
264 | | Type | Name | Format | Description |
265 | | --- | --- | --- | --- |
266 | | 1 | Codec Configuration | N | Codec information and the configurations supported for the codec
There is one TLV of this type per supported codec |
267 |
268 | ##### Audio Codec Configuration:
269 |
270 | | Type | Name | Format | Description |
271 | | --- | --- | --- | --- |
272 | | 1 | Codec | 1 | Type of audio codec:
0 - AAC-LC
1 - AAC-ELD |
273 | | 2 | Audio Codec Parameters | N | Video Codec specific parameters |
274 |
275 | ##### Audio Codec Parameters:
276 |
277 | | Type | Name | Format | Description |
278 | | --- | --- | --- | --- |
279 | | 1 | Channels | 1 | Count of audio channels |
280 | | 2 | Bitrate Modes | 1 | List (probably empty tlv separated?) of supported audio bitrate modes:
0 - Variable
1 - Constant |
281 | | 3 | Sample rates | 1 | List (probably empty tlv separated?) of supported sample rates:
0 - 8 kHz
1 - 16 kHz
2 - 24 kHz
3 - 32 kHz
4 - 44.1 kHz
5 - 48 kHz|
282 | | 4 | Max Audio Bitrate | 4 | Only present in the Selected Camera Recording Configuration request:
maximum selected audio bitrate |
283 |
284 | ### 3.11 SelectedCameraRecordingConfiguration
285 |
286 | | Property | Value |
287 | | ------------------------- | ------------------------------------ |
288 | | UUID | 00000209-0000-1000-8000-0026BB765291 |
289 | | Type | public.hap.characteristics.selected-camera-recording-configuration |
290 | | Permissions | Paired Read, Paired Write, Notify |
291 | | Format | tlv8 |
292 |
293 | The structure of the write value looks like the following:
294 |
295 | ##### Selected Camera Recording Configuration:
296 |
297 | | Type | Name | Format | Description |
298 | | --- | --- | --- | --- |
299 | | 1 | Selected General Configuration | N | The selected [recording configuration](#supported-camera-recording-configuration) |
300 | | 2 | Selected Video Configuration | N | The selected [video recording configuration](#supported-video-recording-configuration) |
301 | | 3 | Selected Audio Configuration | N | The selected [audio recording configuration](#supported-audio-recording-configuration) |
302 |
303 | ### 3.12 ThirdPartyCameraActive
304 |
305 | _Usage and behaviour of this characteristic is currently pretty unclear._
306 |
307 | | Property | Value |
308 | | ------------------------- | ------------------------------------ |
309 | | UUID | 0000021C-0000-1000-8000-0026BB765291 |
310 | | Type | public.hap.characteristics.third-party-camera-active |
311 | | Permissions | Paired Read, Notify |
312 | | Format | bool |
313 |
314 | ### 3.13 DiagonalFieldOfView
315 |
316 | | Property | Value |
317 | | ------------------------- | ------------------------------------ |
318 | | UUID | 00000224-0000-1000-8000-0026BB765291 |
319 | | Type | public.hap.characteristics.diagonal-fov |
320 | | Permissions | Paired Read, Notify |
321 | | Format | float |
322 | | Minimum Value | 0 |
323 | | Maximum Value | 360 |
324 | | Unit | arcdegrees |
325 |
326 | ## 4. HomeKit Data Stream Packet Formats
327 | ### 4.1 Start
328 |
329 | When the camera detects motion it will send a hap event for the characteristic as usual.
330 | After that, one of the connected Home Hubs will send an open request.
331 |
332 | The header should use `dataSend` as the protocol and `open` as the topic.
333 | The **request** has the following message fields:
334 |
335 | | Key | Type | Description |
336 | | --- | ---- | ----------- |
337 | | target | string | `home hub` - to signify the direction of the send |
338 | | type | string | `ipcamera.recording` - the type of the stream |
339 | | streamId | int | used to identify this stream; chosen by the home hub |
340 |
341 | The **response** has the following message fields:
342 |
343 | | Key | Type | Description |
344 | | --- | ---- | ----------- |
345 | | status | int | Indicates if stream could be opened. Available codes are unknown:
0 - Success |
346 |
347 | ### 4.2 Binary Data
348 |
349 | The header should use `dataSend` as the protocol and `data` as the topic.
350 | The **event** has the following message fields:
351 |
352 | | Key | Type | Description |
353 | | --- | ---- | ----------- |
354 | | streamId | int | Same identifier used in the dataSend.open |
355 | | packets | array | Array of dictionaries. Usually length = 1 |
356 |
357 | A packet dictionary looks like the following:
358 |
359 | | Key | Type | Description |
360 | | --- | ---- | ----------- |
361 | | data | bytes | Packet data |
362 | | metadata | dictionary | Metadata for the packet |
363 |
364 | Metadata for recording chunks is defined as:
365 |
366 | | Key | Type | Description |
367 | | --- | ---- | ----------- |
368 | | dataType | string | `mediaInitialization` - for the first event message which contains mp4 initializing `ftyp` and `moof` boxes
`mediaFragment` - for all other packets which contains fragmented mp4 segments |
369 | | dataSequenceNumber | int | Starting by `1` (with the mediaInitialization packet) and incrementing for every mp4 segment |
370 | | dataChunkSequenceNumber | int | Starting by `1`; enumerates every data chunk of a mp4 segment (if a mp4 segment is to big it can be split in multiple packets using this chunk number) |
371 | | isLastDataChunk | boolean | `true` when the data chunk is the last for the current sequence/mp4 segment |
372 |
373 | ### 4.3 Close
374 |
375 | This event closes the stream and is sent by the home hub once the motion sensor is set back to "No motion detected"
376 | (It seems that the home hub still waits for the last mp4 segment to be sent).
377 | The header should use `dataSend` as the protocol and `close` as the topic.
378 |
379 | The **event** has the following message fields:
380 |
381 | | Key | Type | Description |
382 | | --- | ---- | ----------- |
383 | | streamId | int | Same identifier used in the dataSend.open |
384 | | reason | int | Example reasons:
0 - Normal - Normal Close
1 - Not Allowed - Home hub will not allow the Accessory to send this transfer
2 - Busy - Home hub cannot accept this transfer right now
3 - Cancelled - Accessory will not finish the transfer
4 - Unsupported - Home hub does not support this stream type
5 - Unexpected Failure - Some other protocol error occurred and the stream has failed
6 - Timeout - Accessory could not start the session |
385 |
386 | The accessory can also send this event message to indicate that the session errored unexpectedly and should be aborted.
387 |
388 | ## 5. Image Snapshots
389 |
390 | The POST body of the `POST /resource` request received a new optional property `reason` with number type.
391 |
392 | With the `reason` property a controller indicates the reason for a snapshot request:
393 | * `0`: Request is the result of a periodic snapshot request.
394 | * `1`: Request is the result of an event snapshot (e.g. to display image for a motion event).
395 |
396 | If the accessory has [PeriodicSnapshotsActive](#36-periodicsnapshotsactive) turned off, any snapshot request without
397 | a `reason` property or the `reason` property set to `0` must be rejected.
398 |
399 | If the accessory has [EventSnapshotsActive](#32-eventsnapshotsactive) turned off, any snapshot request without
400 | a `reason` property or the `reason` property set to `1` must be rejected.
401 |
402 |
403 | When rejecting a snapshot request the accessory must return `HTTP 207 Multi-Status` and
404 | a HAP Status code of `-70401` (`INSUFFICIENT_PRIVILEGES`; if it was rejected due to the missing `reason` property)
405 | or `-70412` (`NOT_ALLOWED_IN_CURRENT_STATE`, if the `reason` doesn't match the current set rules).
406 |
407 | ## 6. Flow of events
408 |
409 | In this section I will give a brief overlook on how an activity will be recorded using secure-video.
410 |
411 | * The secure-video camera gets paired.
412 | * The user sets the current [camera state](#active-states) to `Stream & Allow Recording`
413 | * The configuration of the camera will be set up:
414 | * The accessory will receive a write request on the
415 | [SelectedCameraRecordingConfiguration](#311-selectedcamerarecordingconfiguration) characteristic
416 | * Important settings like the `prebuffer length` are configured
417 | * The `Active` characteristic of the [CameraEventRecordingManagement](#22-cameraeventrecordingmanagement) service will be
418 | set to true (and all other active characteristics getting updated according to the [camera state](#active-states))
419 | * If the camera is set to detect motion it will continuously check the video stream for any movement as usual.
420 | If recording is enabled, the camera will fill the pre buffer with mp4 fragments according to the First-In-Last-Out principle.
421 | * When the motion service reports activity, all available Home Hubs will open an HomeKit Data Stream Connection to the accessory
422 | * If the camera detects motion (analogous for doorbell button presses) if will set the `Motion Detected` characteristic
423 | of the `Motion Sensor` to true
424 | * After that, a home hub will initiate a bulk send session over HDS and sends a [`dataSend` `start` request](#41-start)
425 | with a new streamId.
426 | * The camera will now send a [mediaInitialization](#42-binary-data) `dataSend` `data` event with the below listed metadata.
427 | The mp4 data contains a `ftyp` and `moov` box.
428 | * `dataSequenceNumber`: 1
429 | * `dataChunkSequenceNumber`: 1
430 | * `isLastDataChunk`: true
431 | * After that the actual mp4 fragments are getting sent using [mediaFragment](#42-binary-data) `dataSend` `data` events.
432 | The mp4 data contains a `moof` and a `mdata` box and must start with a keyframe.
433 | The accessory will begin immediately by sending the fragments currently contained in the prebuffer (typically 2x4 seconds in length).
434 | After that the accessory will send any newly recorded mp4 fragments (typically 4s in length) when they become available
435 | (any fragment will be sent, where the recording was started while motion was still detected).
436 | Every mp4 fragment gets a new incrementally assigned `dataSequenceNumber` (starting by 2 for the first segment in the preBuffer).
437 | If the size of one mp4 fragment is too big, it can be split into multiple chunks. Then every chunk is enumerated
438 | by the `dataChunkSequenceNumber`, while the last chunk must always be marked with `isLastDataChunk` equal to true.
439 | Current cameras seems to use a **maximum chunk size of 262,144 bytes** (or 0x40000 bytes).
440 | * The HomeKit Home Hub receiving the mp4 fragments will analyze every mp4 fragment for moving objects, recognize faces, and decide,
441 | according to the configured motion settings, if a given fragment will be flagged for motion or not.
442 | The Home Hub will then assemble a recording from the flagged mp4 fragments and save it in iCloud.
443 | iCloud will then tell iPhones, iPads, HomePods and AppleTV to notify the user based on their notification settings.
444 | * When motion stops the accessory will set the `Motion Detected` characteristic of the `Motion Sensor` to false.
445 | The camera will still send out the last mp4 fragment which is currently recorded (remember: typically fixed 4s fragment length).
446 | After a short time the Home Hub will send a [`dataSend` `close` event](#43-close) to indicate that the given
447 | transmission for the `streamId` is closed.
448 |
449 | Example fmp4 files of a transmitted recording can be found in the [examples](./examples) directory.
450 | Additionally, a full writeup of the transmitted HomeKit Data Stream payloads for the given example can be found
451 | [here](./examples/README.md).
452 |
--------------------------------------------------------------------------------
/examples/README.md:
--------------------------------------------------------------------------------
1 | # DataSend Recording Example
2 |
3 | This example will demonstrate the file encoding and the datastream communication for
4 | one recording.
5 |
6 | In the `chunks` folder are the individual data chunks per file in their given order as
7 | they would be transferred over HDS.
8 |
9 | In the `fragments` folder the chunks got assembled to a mp4 fragment.
10 |
11 | And the `fully-assembled.mp4` file represent the whole recording assembled to one file.
12 |
13 | A tool like http://mp4parser.com can be used to look at the boxes contained in the mp4 files.
14 |
15 | ## DataStream `dataSend` communication
16 |
17 | All HDS payloads below are encoded in json objects which HAP-NodeJS would also use to
18 | encode/decode those payloads
19 |
20 | ### open
21 |
22 | Request (Controller -> Accessory):
23 |
24 | ```json
25 | {
26 | "header": {
27 | "protocol": "dataSend",
28 | "request": "open",
29 | "id": 0
30 | },
31 | "message": {
32 | "type": "ipcamera.recording",
33 | "target": "controller",
34 | "streamId": 1
35 | }
36 | }
37 | ```
38 |
39 | Response (Accessory -> Controller):
40 |
41 | ```json
42 | {
43 | "header": {
44 | "protocol": "dataSend",
45 | "response": "open",
46 | "id": 0,
47 | "status": 0
48 | },
49 | "message": {
50 | "status": 0
51 | }
52 | }
53 | ```
54 |
55 | ### data transmission
56 |
57 | #### Media Initialization
58 |
59 | ```json
60 | {
61 | "header": {
62 | "protocol": "dataSend",
63 | "event": "data"
64 | },
65 | "message": {
66 | "streamId": 1,
67 | "packets": {
68 | "metadata": [{
69 | "dataType": "mediaInitialization",
70 | "dataSequenceNumber": 1,
71 | "isLastDataChunk": true,
72 | "dataChunkSequenceNumber": 1
73 | }],
74 | "data": "contents of ./chunks/mediaInitialization.mp4"
75 | }
76 | }
77 | }
78 | ```
79 |
80 | ### Transmission of the Pre Buffer
81 |
82 | #### Sequence 2
83 |
84 | ```json
85 | {
86 | "header": {
87 | "protocol": "dataSend",
88 | "event": "data"
89 | },
90 | "message": {
91 | "streamId": 1,
92 | "packets": [{
93 | "metadata": {
94 | "dataType": "mediaFragment",
95 | "dataSequenceNumber": 2,
96 | "isLastDataChunk": false,
97 | "dataChunkSequenceNumber": 1
98 | },
99 | "data": "contents of ./chunks/sequence2_1.mp4"
100 | }]
101 | }
102 | }
103 | ```
104 |
105 | ```json
106 | {
107 | "header": {
108 | "protocol": "dataSend",
109 | "event": "data"
110 | },
111 | "message": {
112 | "streamId": 1,
113 | "packets": {
114 | "metadata": [{
115 | "dataType": "mediaFragment",
116 | "dataSequenceNumber": 2,
117 | "isLastDataChunk": true,
118 | "dataChunkSequenceNumber": 2
119 | }],
120 | "data": "contents of ./chunks/sequence2_2.mp4"
121 | }
122 | }
123 | }
124 | ```
125 |
126 | #### Sequence 3
127 |
128 | ```json
129 | {
130 | "header": {
131 | "protocol": "dataSend",
132 | "event": "data"
133 | },
134 | "message": {
135 | "streamId": 1,
136 | "packets": {
137 | "metadata": [{
138 | "dataType": "mediaFragment",
139 | "dataSequenceNumber": 3,
140 | "isLastDataChunk": false,
141 | "dataChunkSequenceNumber": 1
142 | }],
143 | "data": "contents of ./chunks/sequence3_1.mp4"
144 | }
145 | }
146 | }
147 | ```
148 |
149 | ```json
150 | {
151 | "header": {
152 | "protocol": "dataSend",
153 | "event": "data"
154 | },
155 | "message": {
156 | "streamId": 1,
157 | "packets": {
158 | "metadata": [{
159 | "dataType": "mediaFragment",
160 | "dataSequenceNumber": 3,
161 | "isLastDataChunk": true,
162 | "dataChunkSequenceNumber": 2
163 | }],
164 | "data": "contents of ./chunks/sequence3_2.mp4"
165 | }
166 | }
167 | }
168 | ```
169 |
170 | ### Transmission of recordings
171 |
172 | #### Sequence 4
173 |
174 | ```json
175 | {
176 | "header": {
177 | "protocol": "dataSend",
178 | "event": "data"
179 | },
180 | "message": {
181 | "streamId": 1,
182 | "packets": {
183 | "metadata": [{
184 | "dataType": "mediaFragment",
185 | "dataSequenceNumber": 4,
186 | "isLastDataChunk": false,
187 | "dataChunkSequenceNumber": 1
188 | }],
189 | "data": "contents of ./chunks/sequence4_1.mp4"
190 | }
191 | }
192 | }
193 | ```
194 |
195 | ```json
196 | {
197 | "header": {
198 | "protocol": "dataSend",
199 | "event": "data"
200 | },
201 | "message": {
202 | "streamId": 1,
203 | "packets": {
204 | "metadata": [{
205 | "dataType": "mediaFragment",
206 | "dataSequenceNumber": 4,
207 | "isLastDataChunk": true,
208 | "dataChunkSequenceNumber": 2
209 | }],
210 | "data": "contents of ./chunks/sequence4_2.mp4"
211 | }
212 | }
213 | }
214 | ```
215 |
216 | ### close
217 |
218 | ```json
219 | {
220 | "header": {
221 | "protocol": "dataSend",
222 | "event": "close"
223 | },
224 | "message": {
225 | "streamId": 1,
226 | "status": 0
227 | }
228 | }
229 | ```
230 |
--------------------------------------------------------------------------------
/examples/chunks/mediaInitialization.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/mediaInitialization.mp4
--------------------------------------------------------------------------------
/examples/chunks/sequence2_1.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence2_1.mp4
--------------------------------------------------------------------------------
/examples/chunks/sequence2_2.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence2_2.mp4
--------------------------------------------------------------------------------
/examples/chunks/sequence3_1.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence3_1.mp4
--------------------------------------------------------------------------------
/examples/chunks/sequence3_2.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence3_2.mp4
--------------------------------------------------------------------------------
/examples/chunks/sequence4_1.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence4_1.mp4
--------------------------------------------------------------------------------
/examples/chunks/sequence4_2.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence4_2.mp4
--------------------------------------------------------------------------------
/examples/fragments/mediaInitialization.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/fragments/mediaInitialization.mp4
--------------------------------------------------------------------------------
/examples/fragments/sequence2.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/fragments/sequence2.mp4
--------------------------------------------------------------------------------
/examples/fragments/sequence3.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/fragments/sequence3.mp4
--------------------------------------------------------------------------------
/examples/fragments/sequence4.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/fragments/sequence4.mp4
--------------------------------------------------------------------------------
/examples/fully-assembled.mp4:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/fully-assembled.mp4
--------------------------------------------------------------------------------