├── .github └── FUNDING.yml ├── .gitignore ├── LICENSE ├── README.md └── examples ├── README.md ├── chunks ├── mediaInitialization.mp4 ├── sequence2_1.mp4 ├── sequence2_2.mp4 ├── sequence3_1.mp4 ├── sequence3_2.mp4 ├── sequence4_1.mp4 └── sequence4_2.mp4 ├── fragments ├── mediaInitialization.mp4 ├── sequence2.mp4 ├── sequence3.mp4 └── sequence4.mp4 └── fully-assembled.mp4 /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | github: Supereg 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Andi 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # HomeKit Secure Video Unofficial Specification - R3 2 | 3 | ## 1. Overview 4 | 5 | ### Service Definitions 6 | 7 | #### Cameras 8 | 9 | HomeKit Secure-Video cameras need to expose the same services as normal cameras with the following changes: 10 | 11 | * Every `RTPStreamManagement` service must add the `Active` characteristic 12 | (This is used to indicate that the camera is fully turned off) 13 | * The required amount of `RTPStreamManagement` services was dropped from two to one. 14 | * The `MotionSensor` service is required (to indicate movement and thus a start and stop of a recording) 15 | * The `CameraOperatingMode` service is required 16 | * The `DataStreamManagement` service is required (to initiate HomeKit Data Stream communication) 17 | * The `CameraEventRecordingManagement` service is required. It needs to link to the `MotionSensor` and `DataStreamManagement` service 18 | 19 | If `MotionSensor` or `OccupancySensor` are added, they must expose the `Active` characteristic. 20 | 21 | #### Doorbells 22 | 23 | HomeKit Secure-Video doorbells need to expose the same services as both doorbells and secure-video cameras. 24 | 25 | ### Active states 26 | 27 | Every Secure-Video enabled camera can be set to four different states: `Off`, `Detect Activity`, `Stream` 28 | and `Stream & Allow Recording`. 29 | Depending on the state the following `Active` characteristics for the given services are set. 30 | 31 | | Camera-States | RTPStreamManagement `Active` | CameraOperatingMode `HomeKitCameraActive` | CameraEventRecordingManagement `Active` | 32 | | :----------------------: | :---: | :---: | :---: | 33 | | Off | false | false | false | 34 | | Detect Activity | false | true | false | 35 | | Stream | true | true | false | 36 | | Stream & Allow Recording | true | true | true | 37 | 38 | ### Recording requirements 39 | 40 | * The camera needs so support basic motion detection 41 | * The camera needs to support encoding, buffering and transmitting of fragmented MP4 - Pretty similar to Section 3.3 42 | [RFC 8216](https://tools.ietf.org/html/rfc8216#section-3.3) (HLS). The RFC refers to 43 | [ISO/IEC 14496-12 ISO base media file format](https://mpeg.chiariglione.org/standards/mpeg-4/iso-base-media-file-format/text-isoiec-14496-12-5th-edition). 44 | How exactly the data is transmitted can be read 45 | in the [HDS Packet Formats](#4-homekit-data-stream-packet-formats) section and the [Flow of events](#6-flow-of-events) 46 | section. 47 | * Supported video codecs are: h.264, h.265 (possibly) 48 | * Supported audio codecs are: AAC-LC, AAC-ELD 49 | * Typical resolutions one might support (a camera might support more): 50 | * 640x480 51 | * 1024x768 52 | * 1280x960 53 | * 1600x1200 54 | * 2048x1536 55 | * 640x360 56 | * 1280x720 (Mandatory) 57 | * 1920x1080 (Mandatory, default at 24 or 30 fps) 58 | * 3840x2160 59 | * Frame rates: 60 | * 15 fps (Mandatory) 61 | * 24 fps 62 | * 30 fps (One of 24 fps or 30 fps is mandatory) 63 | 64 | 65 | ## 2. Services 66 | 67 | ### 2.1 CameraOperatingMode 68 | 69 | | Property | Value | 70 | | ------------------------- | ------------------------------------ | 71 | | UUID | 0000021A-0000-1000-8000-0026BB765291 | 72 | | Type | public.hap.service.camera-operating-mode | 73 | | Required Characteristics | [3.2 EventSnapshotActive](#32-eventsnapshotsactive)
[3.3 HomeKitCameraActive](#33-homekitcameraactive)
[3.6 PeriodicSnapshotsActive](#36-periodicsnapshotsactive) | 74 | | Optional Characteristics | [3.4 ManuallyDisabled](#34-manuallydisabled)
[3.5 NightVision](#35-nightvision)
[3.12 ThirdPartyCameraActive](#312-thirdpartycameraactive)
[3.1 CameraOperatingModeIndicator](#31-cameraoperatingmodeindicator)
[3.13 Diagonal Field of View](#313-diagonalfieldofview) | 75 | 76 | ### 2.2 CameraEventRecordingManagement 77 | 78 | | Property | Value | 79 | | ------------------------- | ------------------------------------ | 80 | | UUID | 00000204-0000-1000-8000-0026BB765291 | 81 | | Type | public.hap.service.camera-recording-management | 82 | | Required Characteristics | Active
[3.8 SupportedCameraRecordingConfiguration](#38-supportedcamerarecordingconfiguration)
[3.9 SupportedVideoRecordingConfiguration](#39-supportedvideorecordingconfiguration)
[3.10 SupportedAudioRecordingConfiguration](#310-supportedaudiorecordingconfiguration)
[3.11 SelectedCameraRecordingConfiguration](#311-selectedcamerarecordingconfiguration)
[3.7 RecordingAudioActive](#37-recordingaudioactive) | 83 | 84 | ## 3. Characteristics 85 | 86 | ### 3.1 CameraOperatingModeIndicator 87 | 88 | This characteristic indicates if the camera LED, which shows the current state of the camera (see [states](#active-states)), 89 | should be turned on. Controlled by "Camera status light" setting in the Home App. 90 | 91 | | Property | Value | 92 | | ------------------------- | ------------------------------------ | 93 | | UUID | 0000021D-0000-1000-8000-0026BB765291 | 94 | | Type | public.hap.characteristics.camera-operating-mode-indicator | 95 | | Permissions | Paired Read, Paired Write, Notify, Timed Write | 96 | | Format | bool | 97 | | Valid Values | 0 - "Hardware LED is disabled"
1 - "Hardware LED is enabled" | 98 | 99 | ### 3.2 EventSnapshotsActive 100 | 101 | This characteristic indicates if the option _"Camera Settings" -> Notifications -> "Allow Snapshots in Notifications"_ 102 | is turned on. If this option is turned on, a notification from this camera sent to anyone in this home, will include 103 | a snapshot of the motion or activity. 104 | 105 | | Property | Value | 106 | | ------------------------- | ------------------------------------ | 107 | | UUID | 00000223-0000-1000-8000-0026BB765291 | 108 | | Type | public.hap.characteristics.event-snapshots-active | 109 | | Permissions | Paired Read, Paired Write, Notify, Timed Write | 110 | | Format | bool | 111 | | Valid Values | 0 - "Snapshots in notifications are turned off"
1 - "Snapshots in notifications are turned on" | 112 | 113 | ### 3.3 HomeKitCameraActive 114 | 115 | This characteristic indicates if the camera should detect activity (Unsure if activity just means motion detection or 116 | also button presses for doorbell accessories). 117 | 118 | | Property | Value | 119 | | ------------------------- | ------------------------------------ | 120 | | UUID | 0000021B-0000-1000-8000-0026BB765291 | 121 | | Type | public.hap.characteristics.homekit-camera-active | 122 | | Permissions | Paired Read, Paired Write, Notify, Timed Write | 123 | | Format | bool | 124 | | Valid Values | 0 - "Activity detection should not be enabled"
1 - "Activity detection should be enabled" | 125 | 126 | ### 3.4 ManuallyDisabled 127 | 128 | This characteristic indicates if the camera was manually turned off, for example using a physical switch on the camera. 129 | 130 | | Property | Value | 131 | | ------------------------- | ------------------------------------ | 132 | | UUID | 00000227-0000-1000-8000-0026BB765291 | 133 | | Type | public.hap.characteristics.manually-disabled | 134 | | Permissions | Paired Read, Notify | 135 | | Format | bool | 136 | | Valid Values | 0 - "Camera is not manually disabled"
1 - "Camera was manually disabled" | 137 | 138 | ### 3.5 NightVision 139 | 140 | _This characteristic is already present in the current HAP spec_ 141 | This characteristic indicates if automatic night vision should be turned on. 142 | 143 | | Property | Value | 144 | | ------------------------- | ------------------------------------ | 145 | | UUID | 0000011B-0000-1000-8000-0026BB765291 | 146 | | Type | public.hap.characteristics.night-vision | 147 | | Permissions | Paired Read, Paired Write, Notify | 148 | | Format | bool | 149 | | Valid Values | 0 - "Disable night-vision mode"
1 - "Enable night-vision mode" | 150 | 151 | ### 3.6 PeriodicSnapshotsActive 152 | 153 | Exact behaviour unclear. Seems to be always set to `true` regardless of anyone viewing periodic snapshots or not. 154 | 155 | | Property | Value | 156 | | ------------------------- | ------------------------------------ | 157 | | UUID | 00000225-0000-1000-8000-0026BB765291 | 158 | | Type | public.hap.characteristics.periodic-snapshots-active | 159 | | Permissions | Paired Read, Paired Write, Notify, Timed Write | 160 | | Format | bool | 161 | 162 | ### 3.7 RecordingAudioActive 163 | 164 | This characteristic indicates if recordings should include audio. 165 | 166 | | Property | Value | 167 | | ------------------------- | ------------------------------------ | 168 | | UUID | 00000226-0000-1000-8000-0026BB765291 | 169 | | Type | public.hap.characteristics.recording-audio-active | 170 | | Permissions | Paired Read, Paired Write, Notify, Timed Write | 171 | | Format | uint8 | 172 | | Valid Values | 0 - "Audio should not be included in recordings"
1 - "Audio recording is active" | 173 | 174 | ### 3.8 SupportedCameraRecordingConfiguration 175 | 176 | | Property | Value | 177 | | ------------------------- | ------------------------------------ | 178 | | UUID | 00000205-0000-1000-8000-0026BB765291 | 179 | | Type | public.hap.characteristics.supported-camera-recording-configuration | 180 | | Permissions | Paired Read, Notify | 181 | | Format | tlv8 | 182 | 183 | A read request on this characteristic returns the following structure: 184 | 185 | ##### Supported Camera Recording Configuration: 186 | 187 | | Type | Name | Format | Description | 188 | | --- | --- | --- | --- | 189 | | 1 | Prebuffer length | 4 | Size of the prebuffer in milliseconds.
It must be at least 4000ms.
(typical encountered values: 4000ms, 8000ms) | 190 | | 2 | Event Trigger Options | 8 | Bitmask of trigger types:
0x01 - Motion
0x02 - Doorbell | 191 | | 3 | Media Container Configurations | N | List of supported media container configurations.
Most cameras out there do only expose one entry. | 192 | 193 | ##### Media Container Configuration: 194 | 195 | | Type | Name | Format | Description | 196 | | --- | --- | --- | --- | 197 | | 1 | Media Container Type | 1 | Container types:
0 - Fragmented MP4 | 198 | | 2 | Media Container Parameters | N | Media container parameters | 199 | 200 | ##### Media Container Parameters: 201 | 202 | | Type | Name | Format | Description | 203 | | --- | --- | --- | --- | 204 | | 1 | Fragment Length | 4 | Length of one mp4 fragment in milliseconds
(typically 4000ms) | 205 | 206 | ### 3.9 SupportedVideoRecordingConfiguration 207 | 208 | | Property | Value | 209 | | ------------------------- | ------------------------------------ | 210 | | UUID | 00000206-0000-1000-8000-0026BB765291 | 211 | | Type | public.hap.characteristics.supported-video-recording-configuration | 212 | | Permissions | Paired Read, Notify | 213 | | Format | tlv8 | 214 | 215 | The value of this characteristic is a TLV8-encoded list of supported video codecs: 216 | 217 | ##### Supported Video Recording Configuration: 218 | 219 | | Type | Name | Format | Description | 220 | | --- | --- | --- | --- | 221 | | 1 | Codec Configuration | N | Codec information and the configurations supported for the codec
There is one TLV of this type per supported codec | 222 | 223 | ##### Video Codec Configuration: 224 | 225 | | Type | Name | Format | Description | 226 | | --- | --- | --- | --- | 227 | | 1 | Codec | 1 | Type of video codec:
0 - H.264
1 - H-265 | 228 | | 2 | Video Codec Parameters | N | Video Codec specific parameters | 229 | | 3 | Video Attributes | N | Video Attributes supported for the codec (tlv list) | 230 | 231 | Video Codec Configuration TLV contains exact one tlv of 'Video Codec Parameters' and one entry of 232 | 'Video Attributes' per supported resolution/frame rate combination. 233 | 234 | ##### Video Codec Parameters: 235 | 236 | | Type | Name | Format | Description | 237 | | --- | --- | --- | --- | 238 | | 1 | ProfileID | 1 | List of supported H.264 profiles (tlv list is separated by empty tlvs):
0 - Baseline Profile
1 - Main Profile
2 - High Profile | 239 | | 2 | Level | 1 | List of supported H.264 levels (tlv list is separated by empty tlvs):
0 - 3.1
1 - 3.2
2 - 4 | 240 | | 3 | Bitrate | 4 | Only present in the Selected Camera Recording Configuration request:
Selected video bitrate. Typically, secure video requests 2000kbps when face recognition is enabled, and 800kbps otherwise. | 241 | | 4 | iFrame_Interval | 4 | Only present in the Selected Camera Recording Configuration request:
Selected key frame interval in milliseconds. Typically 4000ms. Seems to be the same value as the fragment length. So every mp4 fragment MUST begin with a keyframe | 242 | 243 | ##### Video Codec Attributes: 244 | 245 | | Type | Name | Format | Description | 246 | | --- | --- | --- | --- | 247 | | 1 | Image width | 2 | Image width in pixels | 248 | | 2 | Image height | 2 | Image height in pixels | 249 | | 3 | Frame rate | 1 | Maximum frame rate | 250 | 251 | ### 3.10 SupportedAudioRecordingConfiguration 252 | 253 | | Property | Value | 254 | | ------------------------- | ------------------------------------ | 255 | | UUID | 00000207-0000-1000-8000-0026BB765291 | 256 | | Type | public.hap.characteristics.supported-audio-recording-configuration | 257 | | Permissions | Paired Read, Notify | 258 | | Format | tlv8 | 259 | 260 | The value of this characteristic is a TLV8-encoded list of supported audio codecs: 261 | 262 | ##### Supported Audio Recording Configuration: 263 | 264 | | Type | Name | Format | Description | 265 | | --- | --- | --- | --- | 266 | | 1 | Codec Configuration | N | Codec information and the configurations supported for the codec
There is one TLV of this type per supported codec | 267 | 268 | ##### Audio Codec Configuration: 269 | 270 | | Type | Name | Format | Description | 271 | | --- | --- | --- | --- | 272 | | 1 | Codec | 1 | Type of audio codec:
0 - AAC-LC
1 - AAC-ELD | 273 | | 2 | Audio Codec Parameters | N | Video Codec specific parameters | 274 | 275 | ##### Audio Codec Parameters: 276 | 277 | | Type | Name | Format | Description | 278 | | --- | --- | --- | --- | 279 | | 1 | Channels | 1 | Count of audio channels | 280 | | 2 | Bitrate Modes | 1 | List (probably empty tlv separated?) of supported audio bitrate modes:
0 - Variable
1 - Constant | 281 | | 3 | Sample rates | 1 | List (probably empty tlv separated?) of supported sample rates:
0 - 8 kHz
1 - 16 kHz
2 - 24 kHz
3 - 32 kHz
4 - 44.1 kHz
5 - 48 kHz| 282 | | 4 | Max Audio Bitrate | 4 | Only present in the Selected Camera Recording Configuration request:
maximum selected audio bitrate | 283 | 284 | ### 3.11 SelectedCameraRecordingConfiguration 285 | 286 | | Property | Value | 287 | | ------------------------- | ------------------------------------ | 288 | | UUID | 00000209-0000-1000-8000-0026BB765291 | 289 | | Type | public.hap.characteristics.selected-camera-recording-configuration | 290 | | Permissions | Paired Read, Paired Write, Notify | 291 | | Format | tlv8 | 292 | 293 | The structure of the write value looks like the following: 294 | 295 | ##### Selected Camera Recording Configuration: 296 | 297 | | Type | Name | Format | Description | 298 | | --- | --- | --- | --- | 299 | | 1 | Selected General Configuration | N | The selected [recording configuration](#supported-camera-recording-configuration) | 300 | | 2 | Selected Video Configuration | N | The selected [video recording configuration](#supported-video-recording-configuration) | 301 | | 3 | Selected Audio Configuration | N | The selected [audio recording configuration](#supported-audio-recording-configuration) | 302 | 303 | ### 3.12 ThirdPartyCameraActive 304 | 305 | _Usage and behaviour of this characteristic is currently pretty unclear._ 306 | 307 | | Property | Value | 308 | | ------------------------- | ------------------------------------ | 309 | | UUID | 0000021C-0000-1000-8000-0026BB765291 | 310 | | Type | public.hap.characteristics.third-party-camera-active | 311 | | Permissions | Paired Read, Notify | 312 | | Format | bool | 313 | 314 | ### 3.13 DiagonalFieldOfView 315 | 316 | | Property | Value | 317 | | ------------------------- | ------------------------------------ | 318 | | UUID | 00000224-0000-1000-8000-0026BB765291 | 319 | | Type | public.hap.characteristics.diagonal-fov | 320 | | Permissions | Paired Read, Notify | 321 | | Format | float | 322 | | Minimum Value | 0 | 323 | | Maximum Value | 360 | 324 | | Unit | arcdegrees | 325 | 326 | ## 4. HomeKit Data Stream Packet Formats 327 | ### 4.1 Start 328 | 329 | When the camera detects motion it will send a hap event for the characteristic as usual. 330 | After that, one of the connected Home Hubs will send an open request. 331 | 332 | The header should use `dataSend` as the protocol and `open` as the topic. 333 | The **request** has the following message fields: 334 | 335 | | Key | Type | Description | 336 | | --- | ---- | ----------- | 337 | | target | string | `home hub` - to signify the direction of the send | 338 | | type | string | `ipcamera.recording` - the type of the stream | 339 | | streamId | int | used to identify this stream; chosen by the home hub | 340 | 341 | The **response** has the following message fields: 342 | 343 | | Key | Type | Description | 344 | | --- | ---- | ----------- | 345 | | status | int | Indicates if stream could be opened. Available codes are unknown:
0 - Success | 346 | 347 | ### 4.2 Binary Data 348 | 349 | The header should use `dataSend` as the protocol and `data` as the topic. 350 | The **event** has the following message fields: 351 | 352 | | Key | Type | Description | 353 | | --- | ---- | ----------- | 354 | | streamId | int | Same identifier used in the dataSend.open | 355 | | packets | array | Array of dictionaries. Usually length = 1 | 356 | 357 | A packet dictionary looks like the following: 358 | 359 | | Key | Type | Description | 360 | | --- | ---- | ----------- | 361 | | data | bytes | Packet data | 362 | | metadata | dictionary | Metadata for the packet | 363 | 364 | Metadata for recording chunks is defined as: 365 | 366 | | Key | Type | Description | 367 | | --- | ---- | ----------- | 368 | | dataType | string | `mediaInitialization` - for the first event message which contains mp4 initializing `ftyp` and `moof` boxes
`mediaFragment` - for all other packets which contains fragmented mp4 segments | 369 | | dataSequenceNumber | int | Starting by `1` (with the mediaInitialization packet) and incrementing for every mp4 segment | 370 | | dataChunkSequenceNumber | int | Starting by `1`; enumerates every data chunk of a mp4 segment (if a mp4 segment is to big it can be split in multiple packets using this chunk number) | 371 | | isLastDataChunk | boolean | `true` when the data chunk is the last for the current sequence/mp4 segment | 372 | 373 | ### 4.3 Close 374 | 375 | This event closes the stream and is sent by the home hub once the motion sensor is set back to "No motion detected" 376 | (It seems that the home hub still waits for the last mp4 segment to be sent). 377 | The header should use `dataSend` as the protocol and `close` as the topic. 378 | 379 | The **event** has the following message fields: 380 | 381 | | Key | Type | Description | 382 | | --- | ---- | ----------- | 383 | | streamId | int | Same identifier used in the dataSend.open | 384 | | reason | int | Example reasons:
0 - Normal - Normal Close
1 - Not Allowed - Home hub will not allow the Accessory to send this transfer
2 - Busy - Home hub cannot accept this transfer right now
3 - Cancelled - Accessory will not finish the transfer
4 - Unsupported - Home hub does not support this stream type
5 - Unexpected Failure - Some other protocol error occurred and the stream has failed
6 - Timeout - Accessory could not start the session | 385 | 386 | The accessory can also send this event message to indicate that the session errored unexpectedly and should be aborted. 387 | 388 | ## 5. Image Snapshots 389 | 390 | The POST body of the `POST /resource` request received a new optional property `reason` with number type. 391 | 392 | With the `reason` property a controller indicates the reason for a snapshot request: 393 | * `0`: Request is the result of a periodic snapshot request. 394 | * `1`: Request is the result of an event snapshot (e.g. to display image for a motion event). 395 | 396 | If the accessory has [PeriodicSnapshotsActive](#36-periodicsnapshotsactive) turned off, any snapshot request without 397 | a `reason` property or the `reason` property set to `0` must be rejected. 398 | 399 | If the accessory has [EventSnapshotsActive](#32-eventsnapshotsactive) turned off, any snapshot request without 400 | a `reason` property or the `reason` property set to `1` must be rejected. 401 | 402 | 403 | When rejecting a snapshot request the accessory must return `HTTP 207 Multi-Status` and 404 | a HAP Status code of `-70401` (`INSUFFICIENT_PRIVILEGES`; if it was rejected due to the missing `reason` property) 405 | or `-70412` (`NOT_ALLOWED_IN_CURRENT_STATE`, if the `reason` doesn't match the current set rules). 406 | 407 | ## 6. Flow of events 408 | 409 | In this section I will give a brief overlook on how an activity will be recorded using secure-video. 410 | 411 | * The secure-video camera gets paired. 412 | * The user sets the current [camera state](#active-states) to `Stream & Allow Recording` 413 | * The configuration of the camera will be set up: 414 | * The accessory will receive a write request on the 415 | [SelectedCameraRecordingConfiguration](#311-selectedcamerarecordingconfiguration) characteristic 416 | * Important settings like the `prebuffer length` are configured 417 | * The `Active` characteristic of the [CameraEventRecordingManagement](#22-cameraeventrecordingmanagement) service will be 418 | set to true (and all other active characteristics getting updated according to the [camera state](#active-states)) 419 | * If the camera is set to detect motion it will continuously check the video stream for any movement as usual. 420 | If recording is enabled, the camera will fill the pre buffer with mp4 fragments according to the First-In-Last-Out principle. 421 | * When the motion service reports activity, all available Home Hubs will open an HomeKit Data Stream Connection to the accessory 422 | * If the camera detects motion (analogous for doorbell button presses) if will set the `Motion Detected` characteristic 423 | of the `Motion Sensor` to true 424 | * After that, a home hub will initiate a bulk send session over HDS and sends a [`dataSend` `start` request](#41-start) 425 | with a new streamId. 426 | * The camera will now send a [mediaInitialization](#42-binary-data) `dataSend` `data` event with the below listed metadata. 427 | The mp4 data contains a `ftyp` and `moov` box. 428 | * `dataSequenceNumber`: 1 429 | * `dataChunkSequenceNumber`: 1 430 | * `isLastDataChunk`: true 431 | * After that the actual mp4 fragments are getting sent using [mediaFragment](#42-binary-data) `dataSend` `data` events. 432 | The mp4 data contains a `moof` and a `mdata` box and must start with a keyframe. 433 | The accessory will begin immediately by sending the fragments currently contained in the prebuffer (typically 2x4 seconds in length). 434 | After that the accessory will send any newly recorded mp4 fragments (typically 4s in length) when they become available 435 | (any fragment will be sent, where the recording was started while motion was still detected). 436 | Every mp4 fragment gets a new incrementally assigned `dataSequenceNumber` (starting by 2 for the first segment in the preBuffer). 437 | If the size of one mp4 fragment is too big, it can be split into multiple chunks. Then every chunk is enumerated 438 | by the `dataChunkSequenceNumber`, while the last chunk must always be marked with `isLastDataChunk` equal to true. 439 | Current cameras seems to use a **maximum chunk size of 262,144 bytes** (or 0x40000 bytes). 440 | * The HomeKit Home Hub receiving the mp4 fragments will analyze every mp4 fragment for moving objects, recognize faces, and decide, 441 | according to the configured motion settings, if a given fragment will be flagged for motion or not. 442 | The Home Hub will then assemble a recording from the flagged mp4 fragments and save it in iCloud. 443 | iCloud will then tell iPhones, iPads, HomePods and AppleTV to notify the user based on their notification settings. 444 | * When motion stops the accessory will set the `Motion Detected` characteristic of the `Motion Sensor` to false. 445 | The camera will still send out the last mp4 fragment which is currently recorded (remember: typically fixed 4s fragment length). 446 | After a short time the Home Hub will send a [`dataSend` `close` event](#43-close) to indicate that the given 447 | transmission for the `streamId` is closed. 448 | 449 | Example fmp4 files of a transmitted recording can be found in the [examples](./examples) directory. 450 | Additionally, a full writeup of the transmitted HomeKit Data Stream payloads for the given example can be found 451 | [here](./examples/README.md). 452 | -------------------------------------------------------------------------------- /examples/README.md: -------------------------------------------------------------------------------- 1 | # DataSend Recording Example 2 | 3 | This example will demonstrate the file encoding and the datastream communication for 4 | one recording. 5 | 6 | In the `chunks` folder are the individual data chunks per file in their given order as 7 | they would be transferred over HDS. 8 | 9 | In the `fragments` folder the chunks got assembled to a mp4 fragment. 10 | 11 | And the `fully-assembled.mp4` file represent the whole recording assembled to one file. 12 | 13 | A tool like http://mp4parser.com can be used to look at the boxes contained in the mp4 files. 14 | 15 | ## DataStream `dataSend` communication 16 | 17 | All HDS payloads below are encoded in json objects which HAP-NodeJS would also use to 18 | encode/decode those payloads 19 | 20 | ### open 21 | 22 | Request (Controller -> Accessory): 23 | 24 | ```json 25 | { 26 | "header": { 27 | "protocol": "dataSend", 28 | "request": "open", 29 | "id": 0 30 | }, 31 | "message": { 32 | "type": "ipcamera.recording", 33 | "target": "controller", 34 | "streamId": 1 35 | } 36 | } 37 | ``` 38 | 39 | Response (Accessory -> Controller): 40 | 41 | ```json 42 | { 43 | "header": { 44 | "protocol": "dataSend", 45 | "response": "open", 46 | "id": 0, 47 | "status": 0 48 | }, 49 | "message": { 50 | "status": 0 51 | } 52 | } 53 | ``` 54 | 55 | ### data transmission 56 | 57 | #### Media Initialization 58 | 59 | ```json 60 | { 61 | "header": { 62 | "protocol": "dataSend", 63 | "event": "data" 64 | }, 65 | "message": { 66 | "streamId": 1, 67 | "packets": { 68 | "metadata": [{ 69 | "dataType": "mediaInitialization", 70 | "dataSequenceNumber": 1, 71 | "isLastDataChunk": true, 72 | "dataChunkSequenceNumber": 1 73 | }], 74 | "data": "contents of ./chunks/mediaInitialization.mp4" 75 | } 76 | } 77 | } 78 | ``` 79 | 80 | ### Transmission of the Pre Buffer 81 | 82 | #### Sequence 2 83 | 84 | ```json 85 | { 86 | "header": { 87 | "protocol": "dataSend", 88 | "event": "data" 89 | }, 90 | "message": { 91 | "streamId": 1, 92 | "packets": [{ 93 | "metadata": { 94 | "dataType": "mediaFragment", 95 | "dataSequenceNumber": 2, 96 | "isLastDataChunk": false, 97 | "dataChunkSequenceNumber": 1 98 | }, 99 | "data": "contents of ./chunks/sequence2_1.mp4" 100 | }] 101 | } 102 | } 103 | ``` 104 | 105 | ```json 106 | { 107 | "header": { 108 | "protocol": "dataSend", 109 | "event": "data" 110 | }, 111 | "message": { 112 | "streamId": 1, 113 | "packets": { 114 | "metadata": [{ 115 | "dataType": "mediaFragment", 116 | "dataSequenceNumber": 2, 117 | "isLastDataChunk": true, 118 | "dataChunkSequenceNumber": 2 119 | }], 120 | "data": "contents of ./chunks/sequence2_2.mp4" 121 | } 122 | } 123 | } 124 | ``` 125 | 126 | #### Sequence 3 127 | 128 | ```json 129 | { 130 | "header": { 131 | "protocol": "dataSend", 132 | "event": "data" 133 | }, 134 | "message": { 135 | "streamId": 1, 136 | "packets": { 137 | "metadata": [{ 138 | "dataType": "mediaFragment", 139 | "dataSequenceNumber": 3, 140 | "isLastDataChunk": false, 141 | "dataChunkSequenceNumber": 1 142 | }], 143 | "data": "contents of ./chunks/sequence3_1.mp4" 144 | } 145 | } 146 | } 147 | ``` 148 | 149 | ```json 150 | { 151 | "header": { 152 | "protocol": "dataSend", 153 | "event": "data" 154 | }, 155 | "message": { 156 | "streamId": 1, 157 | "packets": { 158 | "metadata": [{ 159 | "dataType": "mediaFragment", 160 | "dataSequenceNumber": 3, 161 | "isLastDataChunk": true, 162 | "dataChunkSequenceNumber": 2 163 | }], 164 | "data": "contents of ./chunks/sequence3_2.mp4" 165 | } 166 | } 167 | } 168 | ``` 169 | 170 | ### Transmission of recordings 171 | 172 | #### Sequence 4 173 | 174 | ```json 175 | { 176 | "header": { 177 | "protocol": "dataSend", 178 | "event": "data" 179 | }, 180 | "message": { 181 | "streamId": 1, 182 | "packets": { 183 | "metadata": [{ 184 | "dataType": "mediaFragment", 185 | "dataSequenceNumber": 4, 186 | "isLastDataChunk": false, 187 | "dataChunkSequenceNumber": 1 188 | }], 189 | "data": "contents of ./chunks/sequence4_1.mp4" 190 | } 191 | } 192 | } 193 | ``` 194 | 195 | ```json 196 | { 197 | "header": { 198 | "protocol": "dataSend", 199 | "event": "data" 200 | }, 201 | "message": { 202 | "streamId": 1, 203 | "packets": { 204 | "metadata": [{ 205 | "dataType": "mediaFragment", 206 | "dataSequenceNumber": 4, 207 | "isLastDataChunk": true, 208 | "dataChunkSequenceNumber": 2 209 | }], 210 | "data": "contents of ./chunks/sequence4_2.mp4" 211 | } 212 | } 213 | } 214 | ``` 215 | 216 | ### close 217 | 218 | ```json 219 | { 220 | "header": { 221 | "protocol": "dataSend", 222 | "event": "close" 223 | }, 224 | "message": { 225 | "streamId": 1, 226 | "status": 0 227 | } 228 | } 229 | ``` 230 | -------------------------------------------------------------------------------- /examples/chunks/mediaInitialization.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/mediaInitialization.mp4 -------------------------------------------------------------------------------- /examples/chunks/sequence2_1.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence2_1.mp4 -------------------------------------------------------------------------------- /examples/chunks/sequence2_2.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence2_2.mp4 -------------------------------------------------------------------------------- /examples/chunks/sequence3_1.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence3_1.mp4 -------------------------------------------------------------------------------- /examples/chunks/sequence3_2.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence3_2.mp4 -------------------------------------------------------------------------------- /examples/chunks/sequence4_1.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence4_1.mp4 -------------------------------------------------------------------------------- /examples/chunks/sequence4_2.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/chunks/sequence4_2.mp4 -------------------------------------------------------------------------------- /examples/fragments/mediaInitialization.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/fragments/mediaInitialization.mp4 -------------------------------------------------------------------------------- /examples/fragments/sequence2.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/fragments/sequence2.mp4 -------------------------------------------------------------------------------- /examples/fragments/sequence3.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/fragments/sequence3.mp4 -------------------------------------------------------------------------------- /examples/fragments/sequence4.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/fragments/sequence4.mp4 -------------------------------------------------------------------------------- /examples/fully-assembled.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bauer-andreas/secure-video-specification/1d5aa5e564138755a78aafa8819e3d7d1c4689b8/examples/fully-assembled.mp4 --------------------------------------------------------------------------------