├── .github └── workflows │ └── build.yml ├── .gitignore ├── Makefile ├── README.md ├── docs ├── _config.yml ├── basics.md ├── channels.md ├── cryptography.md ├── index.md ├── message.md ├── nano.md ├── rtpheader.png └── simple_message.md ├── mkdocs.yml └── requirements.txt /.github/workflows/build.yml: -------------------------------------------------------------------------------- 1 | name: build 2 | on: 3 | push: 4 | branches: 5 | - master 6 | 7 | jobs: 8 | build: 9 | name: Deploy docs 10 | runs-on: ubuntu-latest 11 | steps: 12 | - name: Checkout master 13 | uses: actions/checkout@v2 14 | 15 | - name: Deploy docs 16 | uses: mhausenblas/mkdocs-deploy-gh-pages@master 17 | env: 18 | GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} 19 | REQUIREMENTS: requirements.txt 20 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # macOS 2 | .DS_Store 3 | ._.DS_Store 4 | 5 | # Windows 6 | Thumbs.db 7 | 8 | # mkdocs build dir 9 | site/ 10 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | build: ## Build docs locally 2 | mkdocs build --clean 3 | 4 | serve: ## Serve live version of your docs 5 | mkdocs serve 6 | 7 | all: build 8 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Smartglass Documentation 2 | 3 | [![GitHub Workflow - Build](https://img.shields.io/github/workflow/status/OpenXbox/smartglass-documentation/build?label=build)](https://github.com/OpenXbox/smartglass-documentation/actions?query=workflow%3Abuild) 4 | [![Discord](https://img.shields.io/discord/338946086775554048)](https://openxbox.org/discord) 5 | 6 | Documentation of the Xbox One SmartGlass protocol 7 | 8 | Python framework `mkdocs` is used to render the Markdown documentation. 9 | 10 | ## Contribute 11 | 12 | Contributions are very welcome. Here's how you can help: 13 | 14 | - Add / correct / expand technical information 15 | - Improve documentation style 16 | - Correct spelling / grammar 17 | - Check ISSUES tab... 18 | 19 | ### Workflow 20 | 21 | 1. __Fork__ this repo 22 | 1. Make changes 23 | 1. __Verify__ your changes are formatted properly 24 | 1. Send a __Pull Request__ 25 | 26 | ## Local testing 27 | 28 | Serve the documentation 29 | 30 | ```sh 31 | make serve 32 | ``` 33 | 34 | Build the documentation 35 | 36 | ```sh 37 | make build 38 | ``` 39 | -------------------------------------------------------------------------------- /docs/_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-cayman -------------------------------------------------------------------------------- /docs/basics.md: -------------------------------------------------------------------------------- 1 | # SmartGlass Protocol 2 | 3 | - [SmartGlass Protocol](#smartglass-protocol) 4 | - [Basic communication](#basic-communication) 5 | - [Client connection](#client-connection) 6 | - [Packet layout](#packet-layout) 7 | - [Strings in SmartGlass packets](#strings-in-smartglass-packets) 8 | 9 | ## Basic communication 10 | 11 | The SmartGlass protocol communicates over _UDP_ port 5050. 12 | Only the discovery and power on messages are transmitted in plain text, the rest is encrypted (see [Cryptography](cryptography.md)). 13 | 14 | Nano protocol uses dynamic ports (UDP/TCP), negotioated via SmartGlass protocol's 15 | broadcast channel. 16 | 17 | ### Client connection 18 | 19 | 1. Console might be powered on by [Power On packet](simple_message.md#power-on-request) 20 | 2. Client sends [Discovery Request](simple_message.md#discovery-request) to console 21 | 3. Console responds with [Discovery Response](simple_message.md#discovery-response) 22 | 4. Client parses [Certificate](simple_message.md#certificate) in that response 23 | 5. Client sets up a [Crypto Context](cryptography.md) using the console's public key 24 | 6. Client sends a [Connect Request](simple_message.md#connect-request) to the console 25 | 7. Console responds with [Connect Response](simple_message.md#connect-response) 26 | 8. Client sends a [Local Join Message](message.md#local-join), announcing itself 27 | 9. Upon [Acknowledgement](message.md#acknowledgement) client opens several [Channels](channels.md) 28 | 10. Received / sent [Heartbeat](channels.md#acknowledging-messages) packets ensure that 29 | client/host is alive 30 | 31 | ### Packet layout 32 | 33 | General packet layout looks like the following: 34 | 35 | | Name | Note | 36 | | ------------------- | ------------------------------------------------------------------------------------------------------------ | 37 | | Packet Header | Either [SimpleMessage](simple_message.md#header) or [Message](message.md#header) | 38 | | Unprotected Payload | For all [SimpleMessage packets](simple_message.md) | 39 | | Protected Payload | For [Connect](simple_message.md#connect-packet) or [Message](message.md) packet | 40 | | \*Hash | Only if packet has `Protected Payload`, see [Message Authentication](cryptography.md#message-authentication) | 41 | 42 | > NOTE: All numeric values in the SmartGlass Protocol are in network / big-endian byteorder. 43 | 44 | ### Strings in SmartGlass packets 45 | 46 | Usually strings are represented like the following: 47 | 48 | | Type | Description | 49 | | --------------- | ------------------------------------- | 50 | | uint16 | String length (excl. null-terminator) | 51 | | uchar \* length | String | 52 | | uchar '\\0' | Null-terminator | 53 | 54 | In this documentation, these strings are referenced as `SGString` 55 | -------------------------------------------------------------------------------- /docs/channels.md: -------------------------------------------------------------------------------- 1 | # Channels 2 | 3 | There are messages and status reports which service a specific area, they 4 | communicate on their own Channel (indicated by the [Service Channel](#service-channels) field 5 | in the message header). 6 | 7 | All channel communication is of type [Message](message.md). 8 | 9 | - [Channels](#channels) 10 | - [Core Channel](#core-channel) 11 | - [Acknowledging Messages](#acknowledging-messages) 12 | - [Service Channels](#service-channels) 13 | - [Acquiring a channel](#acquiring-a-channel) 14 | - [Input Channel](#input-channel) 15 | - [Input TV Remote Channel](#input-tv-remote-channel) 16 | - [Stump Message Type](#stump-message-type) 17 | - [Message Id Generation](#message-id-generation) 18 | - [Assemble HTTP Streaming URL](#assemble-http-streaming-url) 19 | - [HDMI GUID](#hdmi-guid) 20 | - [Stump Source](#stump-source) 21 | - [Stump Quality](#stump-quality) 22 | - [Stump Notification](#stump-notification) 23 | - [Stump Filter Type](#stump-filter-type) 24 | - [Stump Base Message](#stump-base-message) 25 | - [Request](#request) 26 | - [Response](#response) 27 | - [Get Configuration](#get-configuration) 28 | - [Get Headend Info](#get-headend-info) 29 | - [Get Live TV Info](#get-live-tv-info) 30 | - [Get Program Info](#get-program-info) 31 | - [Get Tuner Lineups](#get-tuner-lineups) 32 | - [Get AppChannel Lineups](#get-appchannel-lineups) 33 | - [Get AppChannel Program Data](#get-appchannel-program-data) 34 | - [Get AppChannel Data](#get-appchannel-data) 35 | - [Ensure Streaming started](#ensure-streaming-started) 36 | - [Set Channel Params](#set-channel-params) 37 | - [Get recent Channels](#get-recent-channels) 38 | - [Send Key](#send-key) 39 | - [Media Channel](#media-channel) 40 | - [Text Channel](#text-channel) 41 | - [Broadcast Channel](#broadcast-channel) 42 | - [How to start gamestreaming?](#how-to-start-gamestreaming) 43 | - [Broadcast Message Type](#broadcast-message-type) 44 | - [Messages](#messages) 45 | - [Gamestream Start Message](#gamestream-start-message) 46 | - [Gamestream Stop Message](#gamestream-stop-message) 47 | - [Gamestream State Message](#gamestream-state-message) 48 | - [Gamestream State](#gamestream-state) 49 | - [Initializing](#initializing) 50 | - [Started](#started) 51 | - [Stopped](#stopped) 52 | - [Paused](#paused) 53 | - [Gamestream Enabled Message](#gamestream-enabled-message) 54 | - [Gamestream Error Message](#gamestream-error-message) 55 | - [Gamestream Error](#gamestream-error) 56 | - [Gamestream Telemetry Message](#gamestream-telemetry-message) 57 | - [Gamestream Preview Status Message](#gamestream-preview-status-message) 58 | - [Title Channel](#title-channel) 59 | - [Title handshaking](#title-handshaking) 60 | - [Packet structure](#packet-structure) 61 | 62 | ## Core Channel 63 | 64 | Core Channel is `0`. 65 | 66 | ## Acknowledging Messages 67 | 68 | Console sends packets of type [Acknowledgement](message.md#acknowledgement) 69 | on channel `0x1000000000000000`, Client sends `Acks` on [Core Channel](#core-channel). 70 | 71 | ## Service Channels 72 | 73 | Click on the channel in the following table to get more information. 74 | 75 | | Channel | GUID | 76 | | ----------------------------------------------- | ------------------------------------- | 77 | | [SystemInput](#input-channel) | `fa20b8ca-66fb-46e0-adb60b978a59d35f` | 78 | | [SystemInputTVRemote](#input-tv-remote-channel) | `d451e3b3-60bb-4c71-b3dbf994b1aca3a7` | 79 | | [SystemMedia](#media-channel) | `48a9ca24-eb6d-4e12-8c43d57469edd3cd` | 80 | | [SystemText](#text-channel) | `7af3e6a2-488b-40cb-a93179c04b7da3a0` | 81 | | [SystemBroadcast](#broadcast-channel) | `b6a117d8-f5e2-45d7-862e8fd8e3156476` | 82 | | [Title](#title-channel) | `00000000-0000-0000-0000000000000000` | 83 | 84 | ### Acquiring a channel 85 | 86 | After [Local Join](message.md#local-join) is done successfully, client 87 | can start additional [Service Channel](#service-channels). 88 | 89 | **Example** 90 | The following messaging happens on [Core Channel](#core-channel). 91 | 92 | 1. Start with `Request Id` 0 93 | 2. Client sends [Channel Start Request](message.md#channel-start-request) with 94 | current `Request Id` and [Target Service Channel GUID](#service-channels) 95 | to console 96 | 3. Console replies with [Channel Start Response](message.md#channel-start-response) 97 | 4. Make sure `Request Id` in response matches the request 98 | 5. Check result code of response, has to be `0` for success 99 | 6. Use `Target Channel Id`, delivered in the response, to communicate 100 | with that channel 101 | 7. ... Send next [Channel Start Request](message.md#channel-start-request) with 102 | incremented `Request Id` ... 103 | 104 | ### Input Channel 105 | 106 | Control console via gamepad controls, touch, accelerometer, gyroscope etc. 107 | 108 | Used messages: 109 | 110 | - [Gamepad](message.md#gamepad) 111 | - [Title Touch](message.md#touch) 112 | - [System Touch](message.md#touch) 113 | - [Accelerometer](message.md#accelerometer) 114 | - [Gyrometer](message.md#gyrometer) 115 | - [Inclinometer](message.md#inclinometer) 116 | - [Compass](message.md#compass) 117 | - [Orientation](message.md#orientation) 118 | 119 | ### Input TV Remote Channel 120 | 121 | Also known as `Stump`, controls configured infrared devices via Xbox's IR Blaster 122 | and makes it possible to stream LiveTV and maybe HDMI IN. 123 | Communication is completely [JSON](message.md#json)-based. 124 | 125 | #### Stump Message Type 126 | 127 | | Name | String Value | 128 | | --------------------------- | ------------------------ | 129 | | Error | Error | 130 | | Ensure Streaming Started | EnsureStreamingStarted | 131 | | Get Configuration | GetConfiguration | 132 | | Get Headend Info | GetHeadendInfo | 133 | | Get LiveTV Info | GetLiveTVInfo | 134 | | Get Program Info | GetProgrammInfo | 135 | | Get Recent Channels | GetRecentChannels | 136 | | Get Tuner Lineups | GetTunerLineups | 137 | | Get AppChannel Data | GetAppChannelData | 138 | | Get AppChannel Lineups | GetAppChannelLineups | 139 | | Get AppChannel Program Data | GetAppChannelProgramData | 140 | | Send Key | SendKey | 141 | | Set Channel | SetChannel | 142 | 143 | #### Message Id Generation 144 | 145 | An example of a messageid (Android client): `7edcf9d0.14`. 146 | 147 | - Generate a random int, called `Message Id Prefix`, from `0` to `0x7FFFFFFF`. 148 | - Initialize a `Message Id Index` with `1` 149 | - Concat both values, splitted by `.`: Prefix: `7edcf9d0`, Index: `1` => `7edcf9d0.1` 150 | - For each new request that's sent to the console, `Message Id Index` gets 151 | incremented: `7edcf9d0.2`, `7edcf9d0.3` 152 | 153 | > NOTE: AppChannel stuff uses a **GUID** as `Message Id` 154 | 155 | #### Assemble HTTP Streaming URL 156 | 157 | TODO 158 | 159 | #### HDMI GUID 160 | 161 | `BA5EBA11-DEA1-4BAD-BA11-FEDDEADFAB1E` 162 | 163 | #### Stump Source 164 | 165 | | Name | String Value | 166 | | ------------ | ------------ | 167 | | HDMI | hdmi | 168 | | USB TV Tuner | tuner | 169 | 170 | #### Stump Quality 171 | 172 | | Name | String Value | 173 | | -------------- | ------------ | 174 | | Low Quality | low | 175 | | Medium Quality | medium | 176 | | High Quality | high | 177 | | Best Quality | best | 178 | 179 | #### Stump Notification 180 | 181 | | Name | String Value | 182 | | --------------------- | -------------------- | 183 | | Streaming Error | StreamingError | 184 | | Channel Changed | ChannelChanged | 185 | | Channel Type Changed | ChannelTypeChanged | 186 | | Configuration Changed | ConfigurationChanged | 187 | | Device UI Changed | DeviceUIChanged | 188 | | Headend Changed | HeadendChanged | 189 | | Video Format Changed | VideoFormatChanged | 190 | | Program Changed | ProgrammChanged | 191 | | Tunerstate Changed | TunerStateChanged | 192 | 193 | #### Stump Filter Type 194 | 195 | | Name | String Value | 196 | | --------- | ------------ | 197 | | All | ALL | 198 | | HD and SD | HDSD | 199 | | Only HD | HD | 200 | 201 | #### Stump Base Message 202 | 203 | ##### Request 204 | 205 | Direction: _Client -> Console_ 206 | 207 | ```json 208 | { 209 | "request": "Stump Message Type goes here", 210 | "msgid": "Generated Message Id goes here", 211 | "params": "Either null/None or some other parameters" 212 | } 213 | ``` 214 | 215 | ##### Response 216 | 217 | Direction: _Console -> Client_ 218 | 219 | **Regular response** 220 | 221 | ```json 222 | { 223 | "response": "Same as request value", 224 | "msgid": "Same as request value", 225 | "params": "Requested data" 226 | } 227 | ``` 228 | 229 | **Notification** 230 | 231 | ```json 232 | { 233 | "notification": "notification value", 234 | // Maybe some other value? 235 | } 236 | ``` 237 | 238 | **Error** 239 | 240 | ```json 241 | { 242 | "response": "Error", 243 | "msgid": "Same as request value", 244 | "error": "Some error text" 245 | } 246 | ``` 247 | 248 | #### Get Configuration 249 | 250 | Value: `GetConfiguration` 251 | 252 | **Request Parameters** 253 | `None` 254 | 255 | **Response Parameters** 256 | 257 | ```json 258 | [ 259 | { 260 | "device_id": "0", 261 | "device_type": "tv", 262 | "buttons": { 263 | "btn.back": "Back", 264 | "btn.up": "Up", 265 | "btn.red": "Red", 266 | // and more... 267 | } 268 | }, 269 | { 270 | "device_id": "1", 271 | "device_type": "stb", 272 | "buttons": {} 273 | }, 274 | { 275 | // ... 276 | "device_type": "tuner" 277 | } 278 | ] 279 | ``` 280 | 281 | #### Get Headend Info 282 | 283 | Value: `GetHeadendInfo` 284 | 285 | **Request Parameters** 286 | `None` 287 | 288 | **Response Parameters** 289 | 290 | ```json 291 | { 292 | "headendId": "dbd2530a-fcd5-8ff0-b89d-20cd7e021502", 293 | "providerName": "Sky Deutschland", 294 | "headendLocale": "de-DE", 295 | "providers": [ 296 | { 297 | "headendId": "DBD2530A-FCD5-8FF0-B89D-20CD7E021502", 298 | "providerName": "Sky Deutschland", 299 | "source": "hdmi", 300 | "titleId": "162615AD", 301 | "filterPreference": "HDSD", 302 | "canStream": "false" 303 | }, 304 | { 305 | "headendId": "D130DB0D-6CBC-328C-A30F-A514C0D0F377", 306 | "providerName": "Unity Media", 307 | "source": "tuner", 308 | "titleId": "162615AD", 309 | "filterPreference": "ALL", 310 | "canStream": "true" 311 | } 312 | ], 313 | "blockExplicitContentPerShow": false, 314 | "dvrEnabled": false 315 | } 316 | ``` 317 | 318 | #### Get Live TV Info 319 | 320 | Value: `GetLiveTVInfo` 321 | 322 | **Request Parameters** 323 | `None` 324 | 325 | **Response Parameters** 326 | 327 | ```json 328 | { "inHdmiMode": "1" } 329 | ``` 330 | 331 | #### Get Program Info 332 | 333 | Value: `GetProgrammInfo` 334 | 335 | **Request Parameters** 336 | `None` 337 | 338 | **Response Parameters** 339 | 340 | ```json 341 | // Unknown 342 | ``` 343 | 344 | #### Get Tuner Lineups 345 | 346 | Value: `GetTunerLineups` 347 | 348 | **Request Parameters** 349 | `None` 350 | 351 | **Response Parameters** 352 | 353 | ```json 354 | { 355 | "providers":[ 356 | { 357 | "headendId":"D130DB0D-6CBC-328C-A30F-A514C0D0F377", 358 | "cqsChannels":[ 359 | 360 | ], 361 | "foundChannels":[ 362 | 363 | ] 364 | } 365 | ] 366 | } 367 | ``` 368 | 369 | #### Get AppChannel Lineups 370 | 371 | Value: `GetAppChannelLineups` 372 | 373 | **Request Parameters** 374 | `None` 375 | 376 | **Response Parameters** 377 | 378 | ```json 379 | [ 380 | { 381 | "id":"LiveTvHdmiProvider", 382 | "providerName":"OneGuide", 383 | "primaryColor":"ff107c10", 384 | "secondaryColor":"ffebebeb", 385 | "titleId":"0BB0AAE5", 386 | "channels":[ 387 | 388 | ] 389 | }, 390 | { 391 | "id":"LiveTvPlaylistProvider", 392 | "providerName":"OneGuide", 393 | "primaryColor":"ff107c10", 394 | "secondaryColor":"ffebebeb", 395 | "titleId":"0BB0AAE5", 396 | "channels":[ 397 | { 398 | "id":"PromoAppChannel", 399 | "name":"More app channels" 400 | } 401 | ] 402 | }, 403 | { 404 | "id":"LiveTvUsbProvider", 405 | "providerName":"OneGuide", 406 | "primaryColor":"ff107c10", 407 | "secondaryColor":"ffebebeb", 408 | "titleId":"0BB0AAE5", 409 | "channels":[ 410 | ] 411 | } 412 | ] 413 | ``` 414 | 415 | #### Get AppChannel Program Data 416 | 417 | Value: `GetAppChannelProgramData` 418 | 419 | **Request Parameters** 420 | 421 | ```json 422 | { 423 | "providerId": "provider id goes here", 424 | "programId": "program id goes here" 425 | } 426 | ``` 427 | 428 | **Response Parameters** 429 | 430 | ```json 431 | // Unknown 432 | ``` 433 | 434 | #### Get AppChannel Data 435 | 436 | Value: `GetAppChannelData` 437 | 438 | **Request Parameters** 439 | 440 | ```json 441 | { 442 | "providerId": "provider id goes here", 443 | "channelId": "channel id goes here", 444 | "id": "channel id again" 445 | } 446 | ``` 447 | 448 | **Response Parameters** 449 | 450 | ```json 451 | // Unknown 452 | ``` 453 | 454 | #### Ensure Streaming started 455 | 456 | Value: `EnsureStreamingStarted` 457 | 458 | **Request Parameters** 459 | 460 | ```json 461 | { "source": "stump source goes here"} 462 | ``` 463 | 464 | **Response Parameters** 465 | 466 | ```json 467 | // Unknown 468 | ``` 469 | 470 | #### Set Channel Params 471 | 472 | Value: `SetChannel` 473 | 474 | **Request Parameters** 475 | By Id: 476 | 477 | ```json 478 | { "channelId": "target Channel Id", "lineupInstanceId": "target Lineup Id" } 479 | ``` 480 | 481 | By Name: 482 | 483 | ```json 484 | { "channel_name": "target channel name" } 485 | ``` 486 | 487 | **Response Parameters** 488 | 489 | ```json 490 | // Unknown 491 | ``` 492 | 493 | #### Get recent Channels 494 | 495 | Value: `GetRecentChannels` 496 | 497 | **Request Paramters** 498 | 499 | ```json 500 | { "startindex": 0, "count": 50 } 501 | ``` 502 | 503 | **Response Parameters** 504 | 505 | ```json 506 | [ "list", "of", "channels"] 507 | ``` 508 | 509 | #### Send Key 510 | 511 | Value: `SendKey` 512 | 513 | **Request Parameters** 514 | 515 | ```json 516 | { 517 | "button_id": "button id string", 518 | // optional: 519 | // "device_id": "target device id" 520 | } 521 | ``` 522 | 523 | **Response Parameters** 524 | `True` on success, `False` on error 525 | 526 | ### Media Channel 527 | 528 | Get info about currently playing media and control the playback. 529 | 530 | Used messages: 531 | 532 | - [Media Controller Removed](message.md#media-controller-removed) 533 | - [Media Command](message.md#media-command) 534 | - [Media Command Result](message.md#media-command-result) 535 | - [Media State](message.md#media-state) 536 | 537 | ### Text Channel 538 | 539 | Input Text whenever a text-dialog is shown on Xbox. 540 | 541 | Used messages: 542 | 543 | - [Title Text Configuration](message.md#text-configuration) 544 | - [Title Text Input](message.md#title-text-input) 545 | - [Title Text Selection](message.md#title-text-selection) 546 | - [System Text Configuration](message.md#text-configuration) 547 | - [System Text Input](message.md#system-text-input) 548 | - [System Text Acknowledge](message.md#system-text-acknowledge) 549 | - [System Text Done](message.md#system-text-done) 550 | 551 | ### Broadcast Channel 552 | 553 | Initializing of Gamestreaming is done over this channel. 554 | Communication is completely [JSON](message.md#json)-based. 555 | Actual streaming is done over [Nano](nano.md). 556 | 557 | Messages actively sent from client to console: 558 | 559 | - [Gamestream Start Message](#gamestream-start-message) 560 | - [Gamestream Stop Message](#gamestream-stop-message) 561 | 562 | The other messages are basically status messages sent from console to client. 563 | 564 | #### How to start gamestreaming? 565 | 566 | 1. Open the [SystemBroadcast](#acquiring-a-channel) channel. 567 | 2. Console sends a [Gamestream Enabled Message](#gamestream-enabled-message). 568 | 3. Check the field `enabled` if it's possible to use Gamestreaming (e.g. if it's activated). 569 | 4. Send a [Gamestream Start Message](#gamestream-start-message) to the console. 570 | 5. Console sends some [Gamestream State Messages](#gamestream-state-message). 571 | 6. Get the connection data from [State: Initializing](#initializing). 572 | 7. Use [Nano](nano.md) to connect to the console. 573 | 574 | #### Broadcast Message Type 575 | 576 | | Type | Value | Sent by | 577 | | ---------------------------------------------------- | ----- | ------- | 578 | | [Start Gamestream](#gamestream-start-message) | 0x01 | Client | 579 | | [Stop Gamestream](#gamestream-stop-message) | 0x02 | Client | 580 | | [Gamestream State](#gamestream-state-message) | 0x03 | Console | 581 | | [Gamestream Enabled](#gamestream-enabled-message) | 0x04 | Console | 582 | | [Gamestream Error](#gamestream-error-message) | 0x05 | Console | 583 | | [Telemetry](#gamestream-telemetry-message) | 0x06 | Console | 584 | | [Preview Status](#gamestream-preview-status-message) | 0x07 | Console | 585 | 586 | #### Messages 587 | 588 | ##### Gamestream Start Message 589 | 590 | Type: `0x1` 591 | 592 | Direction: _Client -> Console_ 593 | 594 | > NOTE: Some values _could_ be int or bool rather than string 595 | > but the following example is the original format that gets sent 596 | > from the original Windows 10 client 597 | 598 | ```json 599 | { 600 | "type": 1, 601 | "configuration": { 602 | "urcpType": "0", 603 | "urcpFixedRate": "-1", 604 | "urcpMaximumWindow": "1310720", 605 | "urcpMinimumRate": "256000", 606 | "urcpMaximumRate": "10000000", 607 | "urcpKeepAliveTimeoutMs": "0", 608 | "audioFecType": "0", 609 | "videoFecType": "0", 610 | "videoFecLevel": "3", 611 | "videoPacketUtilization": "0", 612 | "enableDynamicBitrate": "false", 613 | "dynamicBitrateScaleFactor": "1", 614 | "dynamicBitrateUpdateMs": "5000", 615 | "sendKeyframesOverTCP": "false", 616 | "videoMaximumWidth": "1280", 617 | "videoMaximumHeight": "720", 618 | "videoMaximumFrameRate": "60", 619 | "videoPacketDefragTimeoutMs": "16", 620 | "enableVideoFrameAcks": "false", 621 | "enableAudioChat": "true", 622 | "audioBufferLengthHns": "10000000", 623 | "audioSyncPolicy": "1", 624 | "audioSyncMinLatency": "10", 625 | "audioSyncDesiredLatency": "40", 626 | "audioSyncMaxLatency": "170", 627 | "audioSyncCompressLatency": "100", 628 | "audioSyncCompressFactor": "0.99", 629 | "audioSyncLengthenFactor": "1.01", 630 | "enableOpusAudio": "false", 631 | "enableOpusChatAudio": "true", 632 | "inputReadsPerSecond": "120", 633 | "udpMaxSendPacketsInWinsock": "250", 634 | "udpSubBurstGroups": "0", 635 | "udpBurstDurationMs": "12" 636 | }, 637 | "reQueryPreviewStatus": false 638 | } 639 | ``` 640 | 641 | ##### Gamestream Stop Message 642 | 643 | Type: `0x2` 644 | 645 | Direction: _Client -> Console / never seen in the wild_ 646 | 647 | ```json 648 | { 649 | "type": 2 650 | } 651 | ``` 652 | 653 | ##### Gamestream State Message 654 | 655 | Type: `0x3` 656 | 657 | Direction: _Console -> Client_ 658 | 659 | All gamestream state messages use a `state` field, 660 | declared in the following table. 661 | 662 | ###### Gamestream State 663 | 664 | | State | Value | 665 | | ------------ | ----- | 666 | | Initializing | 0x01 | 667 | | Started | 0x02 | 668 | | Stopped | 0x03 | 669 | | Paused | 0x04 | 670 | 671 | ###### Initializing 672 | 673 | ```json 674 | { 675 | "type": 3, 676 | "state": 1, 677 | "sessionId": "{123E4567-E89b-12D3-A456-426655440000}", 678 | "udpPort": 55068, 679 | "tcpPort": 52556 680 | } 681 | ``` 682 | 683 | ###### Started 684 | 685 | ```json 686 | { 687 | "type": 3, 688 | "state": 2, 689 | "sessionId": "{123E4567-E89b-12D3-A456-426655440000}", 690 | "isWirelessConnection": false, 691 | "wirelessChannel": 0, 692 | "transmitLinkSpeed": 1000000000 693 | } 694 | ``` 695 | 696 | ###### Stopped 697 | 698 | ```json 699 | { 700 | "type": 3, 701 | "state": 3, 702 | "sessionId": "{123E4567-E89b-12D3-A456-426655440000}" 703 | } 704 | ``` 705 | 706 | ###### Paused 707 | 708 | ```json 709 | // Never seen in the wild 710 | ``` 711 | 712 | ##### Gamestream Enabled Message 713 | 714 | Type: `0x4` 715 | 716 | Direction: _Console -> Client_ 717 | 718 | ```json 719 | { 720 | "type": 4, 721 | "enabled": true, 722 | "canBeEnabled": true, 723 | "majorProtocolVersion": 6, 724 | "minorProtocolVersion": 0 725 | } 726 | ``` 727 | 728 | ##### Gamestream Error Message 729 | 730 | Type: `0x5` 731 | 732 | Direction: _Console -> Client_ 733 | 734 | ```json 735 | { 736 | "type": 5, 737 | "errorType": 1, 738 | "errorValue": 1234 739 | } 740 | ``` 741 | 742 | ###### Gamestream Error 743 | 744 | | Error | Value | 745 | | ----------------------- | ----- | 746 | | General | 0x01 | 747 | | Failed to instantiate | 0x02 | 748 | | Failed to initialize | 0x03 | 749 | | Failed to start | 0x04 | 750 | | Failed to stop | 0x05 | 751 | | No Controller | 0x06 | 752 | | Different MSA active | 0x07 | 753 | | DRM Video | 0x08 | 754 | | HDCP Video | 0x09 | 755 | | Kinect Title | 0x0A | 756 | | Prohibited Game | 0x0B | 757 | | Poor network connection | 0x0C | 758 | | Streaming disabled | 0x0D | 759 | | Cannot reach console | 0x0E | 760 | | Generic Error | 0x0F | 761 | | Version mismatch | 0x10 | 762 | | No profile | 0x11 | 763 | | Broadcast in progress | 0x12 | 764 | 765 | ##### Gamestream Telemetry Message 766 | 767 | Type: `0x6` 768 | 769 | Direction: _Unknown / never seen in the wild_ 770 | 771 | ```json 772 | { 773 | "type": 6 774 | } 775 | ``` 776 | 777 | ##### Gamestream Preview Status Message 778 | 779 | Type: `0x7` 780 | 781 | Direction: _Console -> Client_ 782 | 783 | ```json 784 | { 785 | "type": 7, 786 | "isPublicPreview": false, 787 | "isInternalPreview": false 788 | } 789 | ``` 790 | 791 | ### Title Channel 792 | 793 | The title channel is used if the running title (aka. game) supports 794 | the `SmartGlass Experience` (for example: Fallout 4). 795 | 796 | For opening the `Service Channel` you have to send a `Title Id` 797 | while [acquiring the channel](#acquiring-a-channel). 798 | 799 | Title channel does basic communication over the `Core channel`: 800 | 801 | - [Auxiliary Stream](message.md#auxiliary-stream) 802 | - [Active Surface Change](message.md#active-surface-change) 803 | 804 | > NOTE: For title specific data additional sockets are used! 805 | 806 | #### Title handshaking 807 | 808 | Flow to get connection data for title communication: 809 | 810 | 1. Client sends [Auxiliary Stream Handshake](message.md#auxiliary-stream) 811 | 2. Host responds with [Auxiliary Stream Handshake](message.md#auxiliary-stream) 812 | 3. Client sends [Auxiliary Stream Handshake](message.md#auxiliary-stream) again 813 | 4. Host responds with [Auxiliary Stream Connection Info](message.md#auxiliary-stream) 814 | 5. Client sets up [Auxiliary crypto context](cryptography.md#auxiliary-stream-encryption) 815 | 6. Client connects to advertised `Title endpoints` 816 | 817 | #### Packet structure 818 | 819 | TODO 820 | -------------------------------------------------------------------------------- /docs/cryptography.md: -------------------------------------------------------------------------------- 1 | # Cryptography 2 | 3 | - [Cryptography](#cryptography) 4 | - [Key Exchange](#key-exchange) 5 | - [KDF Salts](#kdf-salts) 6 | - [IV Generation](#iv-generation) 7 | - [Connect Request](#connect-request) 8 | - [Messages](#messages) 9 | - [Protected Payload Length](#protected-payload-length) 10 | - [Padding](#padding) 11 | - [Example](#example) 12 | - [Message Authentication](#message-authentication) 13 | - [Auxiliary Stream encryption](#auxiliary-stream-encryption) 14 | 15 | At a high level, SmartGlass uses: 16 | 17 | - ECDH over `prime256/384/521v1` with a salted `SHA-512` KDF for key exchange 18 | - `AES-128-CBC` for message encryption 19 | - `HMAC-SHA-256` for message authentication 20 | 21 | ## Key Exchange 22 | 23 | In detail, key exchange works like this: 24 | 25 | 1. On Discovery, the console responds with a 26 | Discovery Response including a certificate, this 27 | certificate holds the console's public key. 28 | 2. The client generates **elliptic curve** 29 | and derives the **shared secret** with console's public key using ECDH and 30 | a randomly generated public/private keypair 31 | 3. The **shared secret** is salted using static salts, see `KDF Salts` 32 | 4. The **salted shared secret** is hashed using `SHA-512` 33 | 5. The **salted & hashed shared secret** is split into the following 34 | individual keys: 35 | - **bytes 0-16**: Encryption key (`AES-128-CBC`) 36 | - **bytes 16-32**: Initialization Vector key (`AES-128-CBC`) 37 | - **bytes 32-64**: Hashing key (`HMAC-SHA-256`) 38 | 6. The client's public key is sent inside the [Connect Request](#connect-request) 39 | message to the console 40 | 41 | ## KDF Salts 42 | 43 | - Prepend: `D6 37 F1 AA E2 F0 41 8C` 44 | - Append: `A8 F8 1A 57 4E 22 8A B7` 45 | 46 | ## IV Generation 47 | 48 | ### Connect Request 49 | 50 | The **IV** is randomly generated and transmitted inside 51 | the **unprotected payload** section of the [Connect Request packet](simple_message.md#connect-request). 52 | 53 | ### Messages 54 | 55 | The **IV** is generated by encrypting the first 16 bytes 56 | of the packet header with `AES-CBC-128` and a null IV. 57 | 58 | ## Protected Payload Length 59 | 60 | The protected payload length inside the header 61 | indicates the data length **without** padding. 62 | 63 | ## Padding 64 | 65 | Protected payloads are padded using 66 | **out of specification** 67 | [PKCS#7](https://en.wikipedia.org/wiki/Padding_(cryptography)#PKCS7) 68 | algorithm to match needed aligment for encryption. 69 | 70 | **How it differs from specification**: 71 | Plaintext needs to be aligned to 16 bytes. 72 | If the plaintext is 16 bytes aligned already, 73 | **NO PADDING** is used. 74 | 75 | > NOTE: By specification, a whole padding block (16 bytes) 76 | > would be appended to the plaintext. 77 | 78 | ### Example 79 | 80 | **Plaintext (12 bytes)** 81 | `DE AD BE EF DE AD BE EF DE AD BE EF` 82 | 83 | **Padded (12+4 bytes)** 84 | `DE AD BE EF DE AD BE EF DE AD BE EF 04 04 04 04` 85 | 86 | As you can see, the padding consists of the _count of 87 | bytes_: 88 | 89 | **2 byte padding**: `02 02` 90 | 91 | **3 byte padding**: `03 03 03` 92 | 93 | **6 byte padding**: `06 06 06 06 06 06` 94 | 95 | ## Message Authentication 96 | 97 | All encrypted packets are authenticated using `HMAC-SHA-256`. 98 | The produced hash has a length of 32 bytes (0x20). 99 | 100 | ## Auxiliary Stream encryption 101 | 102 | Cryptography parameters for [`Title channel` aka. `Auxiliary Stream`](channels.md#title-channel) are 103 | transmitted via [Auxiliary Stream message](message.md#auxiliary-stream). 104 | 105 | The host sets all the crypto parameters and informs the client about it. 106 | Client then uses received `AES key`, `HMAC key` and `Client/Server IV` to setup 107 | the crypto context. 108 | -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- 1 | # Welcome 2 | 3 | This documentation details the SmartGlass protocol, used by the **Xbox One** gaming console family. 4 | 5 | ## Introduction 6 | 7 | SmartGlass is a remote control protocol developed by Microsoft for their Xbox gaming system. 8 | It was originally developed for the Xbox 360, where it relied on an active Xbox Live connection for 9 | communication with the console. With the Xbox One it directly communicates over the local network. 10 | 11 | In this documentation only the Xbox One variant of SmartGlass is documented. 12 | 13 | ## Browse the wiki 14 | 15 | - [Basics](basics.md) 16 | - [Cryptography](cryptography.md) 17 | - [Simple Message](simple_message.md) 18 | - [Message](message.md) 19 | - [Channels](channels.md) 20 | - [Nano Protocol](nano.md) 21 | 22 | ## Capabilities 23 | 24 | Here is a rough overview about the capabilities of the SmartGlass protocol. 25 | 26 | - ~~Start games / apps via TitleID~~ (this feature got removed from the protocol by Microsoft, with ~Sysupdate 10.0.11763.4088 // 04/2019) 27 | - Controller input ([Input Channel](channels.md#input-channel)) 28 | - Media player control ([Media Channel](channels.md#media-channel)) 29 | - Text input ([Text Channel](channels.md#text-channel)) 30 | - Live TV streaming ([Stump Channel](channels.md#input-tv-remote-channel)) 31 | - Gamestreaming ([Broadcast Channel](channels.md#broadcast-channel) / [NANO](nano.md)) 32 | - and more... 33 | 34 | ## Related specifications 35 | 36 | The following specifications share similarities with the Xbox specific protocols. 37 | 38 | ### SmartGlass protocol 39 | 40 | - [MS-CDP: Connected Devices Platform](https://msdn.microsoft.com/en-us/library/mt766144.aspx) 41 | 42 | ### Nano protocol (v1/v2) 43 | 44 | - [MS-RDPEUDP: RDP:UDP TransportExtension](https://msdn.microsoft.com/en-us/library/hh536846.aspx) 45 | 46 | ### XCloud (Nano v3) 47 | 48 | - [MS-SRTP: Secure Real-time Transport Protocol (SRTP) Profile](https://docs.microsoft.com/en-us/openspecs/office_protocols/ms-srtp/d9641c95-b152-4cc7-8311-d178f3241f1f) 49 | 50 | ## Disclaimer 51 | 52 | Xbox, Xbox One, Smartglass and Xbox Live are trademarks of Microsoft Corporation. 53 | Team OpenXbox is in no way endorsed by or affiliated with Microsoft Corporation, or any associated subsidiaries, logos or trademarks. 54 | -------------------------------------------------------------------------------- /docs/message.md: -------------------------------------------------------------------------------- 1 | # Message 2 | 3 | After the initial connection is made, the actual communication happens 4 | via packets of type `0xD00D`. We refer to them as messages of type `Message`. 5 | 6 | - [Message](#message) 7 | - [Header](#header) 8 | - [Header Flags](#header-flags) 9 | - [Message Types](#message-types) 10 | - [Message Payloads](#message-payloads) 11 | - [Local Join](#local-join) 12 | - [Device Capabilities](#device-capabilities) 13 | - [Examples](#examples) 14 | - [Channel Start Request](#channel-start-request) 15 | - [Channel Start Response](#channel-start-response) 16 | - [Channel Stop](#channel-stop) 17 | - [Console Status](#console-status) 18 | - [Active Title](#active-title) 19 | - [Active Title Location](#active-title-location) 20 | - [Active Surface Change](#active-surface-change) 21 | - [Active Surface Type](#active-surface-type) 22 | - [Fragment](#fragment) 23 | - [Acknowledgement](#acknowledgement) 24 | - [Json](#json) 25 | - [Json Fragmentation](#json-fragmentation) 26 | - [Auxiliary Stream](#auxiliary-stream) 27 | - [Endpoint](#endpoint) 28 | - [Disconnect](#disconnect) 29 | - [Disconnect Reason](#disconnect-reason) 30 | - [Power Off](#power-off) 31 | - [Game DVR Record](#game-dvr-record) 32 | - [Unsnap](#unsnap) 33 | - [Gamepad](#gamepad) 34 | - [Gamepad Button](#gamepad-button) 35 | - [Paired Identity State Changed](#paired-identity-state-changed) 36 | - [Paired Identity State](#paired-identity-state) 37 | - [Media State](#media-state) 38 | - [Sound Level](#sound-level) 39 | - [Media Playback Status](#media-playback-status) 40 | - [Media Transport State](#media-transport-state) 41 | - [Media Type](#media-type) 42 | - [Media Metadata](#media-metadata) 43 | - [Media Controller Removed](#media-controller-removed) 44 | - [Media Command Result](#media-command-result) 45 | - [Media Command](#media-command) 46 | - [Media Control Command](#media-control-command) 47 | - [Orientation](#orientation) 48 | - [Compass](#compass) 49 | - [Inclinometer](#inclinometer) 50 | - [Gyrometer](#gyrometer) 51 | - [Accelerometer](#accelerometer) 52 | - [Touch](#touch) 53 | - [Touch Action](#touch-action) 54 | - [Touchpoint](#touchpoint) 55 | - [Title Launch](#title-launch) 56 | - [System Text Done](#system-text-done) 57 | - [Text Result](#text-result) 58 | - [System Text Acknowledge](#system-text-acknowledge) 59 | - [System Text Input](#system-text-input) 60 | - [Text Delta](#text-delta) 61 | - [Title Text Selection](#title-text-selection) 62 | - [Title Text Input](#title-text-input) 63 | - [Text Configuration](#text-configuration) 64 | - [Text Input Scope](#text-input-scope) 65 | - [Text Option Flags](#text-option-flags) 66 | 67 | ## Header 68 | 69 | | Offset (hex) | Offset (dec) | Type | Description | 70 | | ------------ | ------------ | ------ | --------------------------------------------- | 71 | | 0x00 | 0 | uint16 | Packet Type | 72 | | 0x02 | 2 | uint16 | Protected Payload Length | 73 | | 0x04 | 4 | uint32 | Sequence Number | 74 | | 0x08 | 8 | uint32 | Target Participant Id | 75 | | 0x0C | 12 | uint32 | Source Participant Id | 76 | | 0x10 | 16 | uint16 | Flags (Version, NeedAck, IsFragment, MsgType) | 77 | | 0x12 | 18 | uint64 | Channel Id | 78 | 79 | - **Packet Type**: Always `0xD00D` for message packet 80 | - **Protected Payload Length**: Payload length before encryption **excluding** 81 | padding. That is, the length of the plaintext 82 | - **Sequence Number**: Incrementing sequence number - if packet 83 | was not acknowledged even if requested, message gets sent 84 | again with same sequence number. Start index is `1`. 85 | - **Target Participant Id**: Target Id as seen from sender, client sets this to 0 86 | - **Source Participant Id**: Id of sender, client gets that from 87 | [Connect Response](simple_message.md#connect-response) 88 | - **Flags**: See [Header Flags](#header-flags) 89 | - **Channel Id**: Negotioated Channel Id (see 90 | [Channel Start Response](#channel-start-response)) 91 | 92 | ### Header Flags 93 | 94 | | Flag | Bits | Mask | 95 | | -------------------- | --------------------- | ------ | 96 | | Version | `1100 0000 0000 0000` | 0xC000 | 97 | | Need Acknowledgement | `0010 0000 0000 0000` | 0x2000 | 98 | | Is Fragment | `0001 0000 0000 0000` | 0x1000 | 99 | | Message Type | `0000 1111 1111 1111` | 0x0FFF | 100 | 101 | - **Version**: Always `2` 102 | - **Need Acknowledgement**: Indicates if the message needs to 103 | be acknowledged by a message of type [Acknowledgement](#acknowledgement) 104 | - **Is Fragment**: Indicates fragmented payload, see [Fragment](#fragment) 105 | - **Message Type**: See [Message Types](#message-types) 106 | 107 | ## Message Types 108 | 109 | Direction as seen from the client perspective: 110 | 111 | - → Sent from client to console 112 | - ← Sent from console to client 113 | - ↔ Both parties send this messagetype 114 | - x Unknown 115 | 116 | | Name | Value | Direction | Channel | 117 | | --------------------------------------------------------------- | ----- | :-------: | ----------- | 118 | | [Acknowledgement](#acknowledgement) | 0x01 | ↔ | _Various_ | 119 | | Group | 0x02 | x | x | 120 | | [Local Join](#local-join) | 0x03 | → | Core | 121 | | Stop Activity | 0x05 | x | x | 122 | | [Auxiliary Stream](#auxiliary-stream) | 0x19 | ↔ | Title | 123 | | [Active Surface Change](#active-surface-change) | 0x1A | ← | Title | 124 | | Navigate | 0x1B | x | x | 125 | | [Json](#json) | 0x1C | ↔ | _Various_ | 126 | | Tunnel | 0x1D | x | x | 127 | | [Console Status](#console-status) | 0x1E | ← | Core | 128 | | [Title Text Configuration](#text-configuration) | 0x1F | ← | SystemText | 129 | | [Title Text Input](#title-text-input) | 0x20 | ↔ | SystemText | 130 | | [Title Text Selection](#title-text-selection) | 0x21 | → | SystemText | 131 | | MirroringRequest | 0x22 | x | x | 132 | | [Title Launch](#title-launch) | 0x23 | → | Core | 133 | | [Channel Start Request](#channel-start-request) | 0x26 | → | Core | 134 | | [Channel Start Response](#channel-start-response) | 0x27 | ← | Core | 135 | | [Channel Stop](#channel-stop) | 0x28 | x | Core | 136 | | System | 0x29 | x | x | 137 | | [Disconnect](#disconnect) | 0x2A | → | Core | 138 | | [Title Touch](#touch) | 0x2E | → | SystemInput | 139 | | [Accelerometer](#accelerometer) | 0x2F | → | SystemInput | 140 | | [Gyrometer](#gyrometer) | 0x30 | → | SystemInput | 141 | | [Inclinometer](#inclinometer) | 0x31 | → | SystemInput | 142 | | [Compass](#compass) | 0x32 | → | SystemInput | 143 | | [Orientation](#orientation) | 0x33 | → | SystemInput | 144 | | [Paired Identity State Changed](#paired-identity-state-changed) | 0x36 | ← | Core | 145 | | [Unsnap](#unsnap) | 0x37 | → | Core | 146 | | [Game DVR Record](#game-dvr-record) | 0x38 | → | Core | 147 | | [Power Off](#power-off) | 0x39 | → | Core | 148 | | [Media Controller Removed](#media-controller-removed) | 0xF00 | ← | SystemMedia | 149 | | [Media Command](#media-command) | 0xF01 | → | SystemMedia | 150 | | [Media Command Result](#media-command-result) | 0xF02 | ← | SystemMedia | 151 | | [Media State](#media-state) | 0xF03 | ← | SystemMedia | 152 | | [Gamepad](#gamepad) | 0xF0A | → | SystemInput | 153 | | [System Text Configuration](#text-configuration) | 0xF2B | ← | SystemText | 154 | | [System Text Input](#system-text-input) | 0xF2C | ↔ | SystemText | 155 | | [System Touch](#touch) | 0xF2E | → | SystemInput | 156 | | [System Text Acknowledge](#system-text-acknowledge) | 0xF34 | ↔ | SystemText | 157 | | [System Text Done](#system-text-done) | 0xF35 | ↔ | SystemText | 158 | 159 | ## Message Payloads 160 | 161 | ### Local Join 162 | 163 | **Message Type**: `0x03` 164 | **Response**: None 165 | **Requests Ack**: `YES` 166 | 167 | Pair client to console. 168 | 169 | | Offset (hex) | Offset (dec) | Type | Description | 170 | | -----------: | -----------: | -------- | ------------------- | 171 | | 0x00 | 0 | uint16 | Device Type | 172 | | 0x02 | 2 | uint16 | Native Width | 173 | | 0x04 | 4 | uint16 | Native Height | 174 | | 0x06 | 6 | uint16 | DPI X | 175 | | 0x08 | 8 | uint16 | DPI Y | 176 | | 0x0A | 10 | uint64 | Device Capabilities | 177 | | 0x12 | 18 | uint32 | Client Version | 178 | | 0x16 | 22 | uint32 | OS Major Version | 179 | | 0x1A | 26 | uint32 | OS Minor Version | 180 | | 0x1E | 30 | SGString | Display Name | 181 | 182 | - **Device Type**: See [Client Type](simple_message.md#client-type) 183 | - **Native Width**: Display resolution width from connecting client 184 | - **Native Height**: Display resolution height from connecting client 185 | - **DPI X**: Pixel Density on X-axis from client display 186 | - **DPI Y**: Pixel Density on Y-axis from client display 187 | - **Device Capabilities**: See [Device Capabilities](#device-capabilities) 188 | - **Client Version**: SmartGlass client version 189 | - **OS Major Version**: Operating System major version 190 | - **OS Minor Version**: Operating System minor version 191 | - **Display Name**: Client's display name 192 | 193 | #### Device Capabilities 194 | 195 | | Capability | Bits | Mask | 196 | | ------------- | --------------------- | ------------------ | 197 | | None | `0000 0000 0000 0000` | 0x00 | 198 | | Streaming | `0000 0000 0000 0001` | 0x01 | 199 | | Audio | `0000 0000 0000 0010` | 0x02 | 200 | | Accelerometer | `0000 0000 0000 0100` | 0x04 | 201 | | Compass | `0000 0000 0000 1000` | 0x08 | 202 | | Gyrometer | `0000 0000 0001 0000` | 0x10 | 203 | | Inclinometer | `0000 0000 0010 0000` | 0x20 | 204 | | Orientation | `0000 0000 0100 0000` | 0x40 | 205 | | All | `1111 1111 1111 1111` | 0xFFFFFFFFFFFFFFFF | 206 | 207 | #### Examples 208 | 209 | **Windows** 210 | 211 | ```ini 212 | DeviceType = ClientType.WindowsStore 213 | NativeWidth = 1080 214 | NativeHeight = 1920 215 | DpiX = 96 216 | DpiY = 96 217 | DeviceCapabilities = DeviceCapabilities.All 218 | ClientVersion = 15 219 | OSMajor = 6 220 | OSMinor = 2 221 | DisplayName = "SmartGlass-PC" 222 | ``` 223 | 224 | **Android** 225 | 226 | ```ini 227 | DeviceType = ClientType.Android 228 | # Resolution is portrait mode 229 | NativeWidth = 720 230 | NativeHeight = 1280 231 | DpiX = 160 232 | DpiY = 160 233 | DeviceCapabilities = DeviceCapabilities.All 234 | ClientVersion = 151117100 # v2.4.1511.17100-Beta 235 | OSMajor = 22 # Android 5.1.1 - API Version 22 236 | OSMinor = 0 237 | DisplayName = "com.microsoft.xboxone.smartglass.beta" 238 | ``` 239 | 240 | ### Channel Start Request 241 | 242 | **Message Type**: `0x26` 243 | **Response**: [Channel Start Response](#channel-start-response) 244 | **Requests Ack**: `NO` 245 | 246 | Start opening a `Service Channel`. 247 | 248 | | Offset (hex) | Offset (dec) | Type | Description | 249 | | -----------: | -----------: | --------- | -------------------- | 250 | | 0x00 | 0 | uint32 | Channel Request Id | 251 | | 0x04 | 4 | uint32 | Title Id | 252 | | 0x08 | 8 | byte\[16] | Service Channel GUID | 253 | | 0x18 | 24 | uint32 | Activity Id | 254 | 255 | - **Channel Request Id**: Incrementing number, it's used to 256 | match with [Channel Start Response](#channel-start-response) 257 | - **Title Id**: Set for [Title channel](channels.md#title-channel), otherwise `0` 258 | - **Service Channel GUID**: See [Service Channels](channels.md#service-channels) 259 | - **Activity Id**: Always `0` 260 | 261 | ### Channel Start Response 262 | 263 | **Message Type**: `0x27` 264 | **Response**: None 265 | **Requests Ack**: `NO` 266 | 267 | Response to [Channel Start Request](#channel-start-request). 268 | 269 | | Offset (hex) | Offset (dec) | Type | Description | 270 | | -----------: | -----------: | ------ | ------------------ | 271 | | 0x00 | 0 | uint32 | Channel Request Id | 272 | | 0x04 | 4 | uint64 | Target Channel Id | 273 | | 0x0C | 12 | uint32 | Result | 274 | 275 | - **Channel Request Id**: Matches with 276 | [Channel Start Request](#channel-start-request) 277 | - **Target Channel Id**: Assigned `Channel Id` to be used in 278 | [Message Header](#header) 279 | - **Result**: Result code, `0` on success 280 | 281 | ### Channel Stop 282 | 283 | **Message Type**: `0x28` 284 | **Response**: None 285 | **Requests Ack**: `NO` 286 | 287 | Stop an opened `Service Channel` 288 | 289 | | Offset (hex) | Offset (dec) | Type | Description | 290 | | -----------: | -----------: | ------ | ----------------- | 291 | | 0x00 | 0 | uint64 | Target Channel Id | 292 | 293 | - **Target Channel Id**: `Channel Id` received by 294 | [Channel Start Response](#channel-start-response) 295 | 296 | ### Console Status 297 | 298 | **Message Type**: `0x1E` 299 | **Response**: None 300 | **Requests Ack**: `YES` 301 | 302 | Informs client about running titles and Xbox OS version. 303 | 304 | | Offset (hex) | Offset (dec) | Type | Description | 305 | | -----------: | -----------: | ------------------- | ------------------ | 306 | | 0x00 | 0 | uint32 | Live TV Provider | 307 | | 0x04 | 4 | uint32 | Major Version | 308 | | 0x08 | 8 | uint32 | Minor Version | 309 | | 0x0C | 12 | uint32 | Build Number | 310 | | 0x10 | 16 | SGString | Locale | 311 | | 0x?? | ? | uint16 | Active Title Count | 312 | | 0x?? | ? | ActiveTitle\[count] | Active Titles | 313 | 314 | - **Live TV Provider**: Live TV provider Id 315 | - **Major Version**: Major OS version 316 | - **Minor Version**: Minor OS version 317 | - **Build Number**: OS builder number 318 | - **Locale**: Locale string 319 | - **Active Title Count**: Number of `Active Titles` 320 | - **Active Titles**: Array of [Active Title](#active-title) 321 | 322 | #### Active Title 323 | 324 | | Offset (hex) | Offset (dec) | Type | Description | 325 | | -----------: | -----------: | --------- | ----------------- | 326 | | 0x00 | 0 | uint32 | Title Id | 327 | | 0x04 | 4 | uint16 | Title Disposition | 328 | | 0x06 | 6 | byte\[16] | Product Id | 329 | | 0x16 | 22 | byte\[16] | Sandbox Id | 330 | | 0x26 | 38 | SGString | AUM Id | 331 | 332 | - **Title Id**: Title Id 333 | - **Title Disposition**: 1 bit: HasFocus-Flag, 15 bits: [Active Title Location](#active-title-location) 334 | - **Product Id**: Product Id 335 | - **Sandbox Id**: Sandbox Id 336 | - **AUM Id**: Application User Model Id 337 | 338 | #### Active Title Location 339 | 340 | | Location | Value | 341 | | ---------- | ----- | 342 | | Full | 0x00 | 343 | | Fill | 0x01 | 344 | | Snapped | 0x02 | 345 | | Start View | 0x03 | 346 | | System UI | 0x04 | 347 | | Default | 0x05 | 348 | 349 | ### Active Surface Change 350 | 351 | **Message Type**: `0x1A` 352 | **Response**: None 353 | **Requests Ack**: `NO` 354 | 355 | Informs client about surface change, used in auxiliary-stream context. 356 | 357 | | Offset (hex) | Offset (dec) | Type | Description | 358 | | -----------: | -----------: | --------- | ------------------ | 359 | | 0x00 | 0 | uint16 | Surface Type | 360 | | 0x02 | 2 | uint16 | Server TCP Port | 361 | | 0x04 | 4 | uint16 | Server UDP Port | 362 | | 0x06 | 6 | byte\[16] | Session Id | 363 | | 0x16 | 22 | uint16 | Render Width | 364 | | 0x18 | 24 | uint16 | Render Height | 365 | | 0x1A | 26 | byte\[16] | Master Session Key | 366 | 367 | - **Surface Type**: See [Active Surface Type](#active-surface-type) 368 | - **Server TCP Port**: Used with `Auxiliary Stream` 369 | - **Server UDP Port**: Used with `Auxiliary Stream` 370 | - **Session Id**: Used with `Auxiliary Stream` 371 | - **Render Width**: Used with `Auxiliary Stream` 372 | - **Render Height**: Used with `Auxiliary Stream` 373 | - **Master Session Key**: Used with `Auxiliary Stream` 374 | 375 | #### Active Surface Type 376 | 377 | | Type | Value | 378 | | ----------------- | ----- | 379 | | Blank | 0x00 | 380 | | Direct | 0x01 | 381 | | HTML | 0x02 | 382 | | Title Text Entity | 0x03 | 383 | 384 | ### Fragment 385 | 386 | **Message Type**: _variable_ 387 | **Response**: _variable_ 388 | **Requests Ack**: _variable_ 389 | **Is Fragment**: `YES` 390 | 391 | Used for messages that need to be fragmented. 392 | When all fragments are received, concatenate the data blobs and parse 393 | the assembled data as indicated `Message Type`. 394 | 395 | | Offset (hex) | Offset (dec) | Type | Description | 396 | | -----------: | -----------: | ---------- | -------------- | 397 | | 0x00 | 0 | uint32 | Sequence Begin | 398 | | 0x04 | 4 | uint32 | Sequence End | 399 | | 0x08 | 8 | uint16 | Data length | 400 | | 0x0A | 10 | byte\[len] | Data | 401 | 402 | - **Sequence Begin**: First sequence number of the fragment-set 403 | - **Sequence End**: Last sequence number (+1) of the fragement-set 404 | - **Data**: Data fragment 405 | 406 | ### Acknowledgement 407 | 408 | **Message Type**: `0x01` 409 | **Response**: None 410 | **Requests Ack**: `NO` 411 | 412 | Acknowledge a message from sender, alternatively used to request hearbeat from peer. 413 | 414 | | Offset (hex) | Offset (dec) | Type | Description | 415 | | -----------: | -----------: | ------------ | --------------------- | 416 | | 0x00 | 0 | uint32 | Low Watermark | 417 | | 0x04 | 4 | uint32 | Processed List Length | 418 | | 0x08 | 8 | uint32\[len] | Processed List | 419 | | 0x?? | ? | uint32 | Rejected List Length | 420 | | 0x?? | ? | uint32\[len] | Rejected List | 421 | 422 | - **Low Watermark**: Last received/processed sequence number 423 | - **Processed List**: Processed sequence numbers (array of `uint32`) 424 | - **Rejected List**: Rejected sequence numbers (array of `uint32`) 425 | 426 | ### Json 427 | 428 | **Message Type**: `0x1C` 429 | **Response**: _variable_ 430 | **Requests Ack**: _variable_ 431 | 432 | Used to transfer commands or info in text-form. 433 | 434 | | Offset (hex) | Offset (dec) | Type | Description | 435 | | -----------: | -----------: | -------- | ----------- | 436 | | 0x00 | 0 | SGString | Text | 437 | 438 | - **Text**: JSON Body 439 | 440 | #### Json Fragmentation 441 | 442 | In case the `Protected Payload` of a Json Message exceeds `1024 bytes`, 443 | the message gets fragmented. 444 | 445 | Fragmentation is done by base64-encoding and splitting-up the Json string, the maximum 446 | fragment length being `905 bytes` (When serializing without spaces). 447 | 448 | Example fragment-set: 449 | 450 | ```json 451 | # Fragment #0 452 | {"datagram_size":"24","datagram_id":"1","fragment_offset":"0","fragment_length":"12","fragment_data":"eyJ0ZXN0Ijoi"} 453 | # Fragment #1 454 | {"datagram_size":"24","datagram_id":"1","fragment_offset":"12","fragment_length":"12","fragment_data":"dmFsdWUifQ=="} 455 | 456 | # After concatenation 457 | "eyJ0ZXN0IjoidmFsdWUifQ==" 458 | 459 | # After decoding 460 | '{"test":"value"}' 461 | ``` 462 | 463 | - **datagram_size**: Total base64-string length 464 | - **datagram_id**: Identifier of the fragment-set 465 | - **fragment_offset**: Position of the current fragment 466 | - **fragment_length**: Length of the current fragment 467 | - **fragment_data**: Base64 string 468 | 469 | Receiving participant checks if a set of fragments is received completely by summarizing `fragment_length` fields 470 | for the specific `datagram_id` and checking it against `datagram_size`. 471 | 472 | When all fragments are received, they are ordered by `fragment_offset` and the `fragment_data` is concatenated and 473 | base64-decoded. 474 | 475 | ### Auxiliary Stream 476 | 477 | **Message Type**: `0x19` 478 | **Response**: None 479 | **Requests Ack**: `YES` 480 | 481 | Used for _SmartGlass Experience_ aka. game companion stuff. 482 | The only known utilization is in Fallout 4. 483 | 484 | | Offset (hex) | Offset (dec) | Type | Description | 485 | | -----------: | -----------: | --------------- | ----------------------------- | 486 | | 0x00 | 0 | byte | Connection Info Flag | 487 | | | | | If Connection Info Flag == 1: | 488 | | 0x01 | 1 | uint16 | AES Key length | 489 | | 0x03 | 3 | byte\[len] | AES Key | 490 | | ?? | ?? | uint16 | Server IV length | 491 | | ?? | ?? | byte\[len] | Server IV | 492 | | ?? | ?? | uint16 | Client IV length | 493 | | ?? | ?? | byte\[len] | Client IV | 494 | | ?? | ?? | uint16 | HMAC Key length | 495 | | ?? | ?? | byte\[len] | HMAC Key | 496 | | ?? | ?? | uint16 | Endpoints Size | 497 | | ?? | ?? | Endpoint\[size] | Endpoints | 498 | 499 | - **Connection Info Flag**: Handshake: `0`, Connection Data: `1` 500 | - **AES Key**: AES-CBC Key 501 | - **Server IV**: Server's Initialization Vector 502 | - **Client IV**: Client's Initialization Vector 503 | - **HMAC Key**: HMAC key 504 | - **Endpoints Size**: Endpoint count 505 | - **Endpoints**: Advertised title endpoints, See [Endpoint](#endpoint) 506 | 507 | #### Endpoint 508 | 509 | | Type | Description | 510 | | -------- | ----------- | 511 | | SGString | IP Address | 512 | | SGString | Port | 513 | 514 | ### Disconnect 515 | 516 | **Message Type**: `0x2A` 517 | **Response**: None 518 | **Requests Ack**: `NO` 519 | 520 | Disconnect client from console. 521 | 522 | | Offset (hex) | Offset (dec) | Type | Description | 523 | | -----------: | -----------: | ------ | ----------- | 524 | | 0x00 | 0 | uint32 | Reason | 525 | | 0x04 | 4 | uint32 |  Error code | 526 | 527 | - **Reason**: See [Disconnect Reason](#disconnect-reason) 528 | - **Error code**: Error code 529 | 530 | #### Disconnect Reason 531 | 532 | | Reason | Value | 533 | | ----------- | ----- | 534 | | Unspecified | 0x00 | 535 | | Error | 0x01 | 536 | | Power Off | 0x02 | 537 | | Maintenance | 0x03 | 538 | | AppClose | 0x04 | 539 | | SignOut | 0x05 | 540 | | Reboot | 0x06 | 541 | | Disabled | 0x07 | 542 | | Low Power | 0x08 | 543 | 544 | ### Power Off 545 | 546 | **Message Type**: `0x39` 547 | **Response**: None 548 | **Requests Ack**: `NO` 549 | 550 | Send poweroff to the console. 551 | 552 | | Offset (hex) | Offset (dec) | Type | Description | 553 | | -----------: | -----------: | -------- | ----------- | 554 | | 0x00 | 0 | SGString | Live ID | 555 | 556 | - **Live ID**: Live ID of console to power off. This info is stored in the [Discovery Response Certificate](simple_message.md#certificate) 557 | 558 | ### Game DVR Record 559 | 560 | **Message Type**: `0x38` 561 | **Response**: None 562 | **Requests Ack**: `YES` 563 | 564 | Save a DVR clip 565 | 566 | | Offset (hex) | Offset (dec) | Type | Description | 567 | | -----------: | -----------: | ----- | ---------------- | 568 | | 0x00 | 0 | int32 | Start Time Delta | 569 | | 0x04 | 4 | int32 | End Time Delta | 570 | 571 | - **Start Time Delta**: Start time of recording in seconds (e.g. -60 for last minute) 572 | - **End Time Delta**: End time of recording in seconds (e.g. 0 for _now_) 573 | 574 | ### Unsnap 575 | 576 | **Message Type**: `0x37` 577 | **Response**: None 578 | **Requests Ack**: `NO` 579 | 580 | Unsnap currently snapped application. 581 | 582 | | Offset (hex) | Offset (dec) | Type | Description | 583 | | -----------: | -----------: | ---- | ----------- | 584 | | 0x00 | 0 | byte | Unknown | 585 | 586 | - **Unknown**: Unknown 587 | 588 | ### Gamepad 589 | 590 | **Message Type**: `0xF0A` 591 | **Response**: None 592 | **Requests Ack**: `NO` 593 | 594 | Send gamepad control (not for use with low latency _gamestreaming_). 595 | 596 | | Offset (hex) | Offset (dec) | Type | Description | 597 | | -----------: | -----------: | ------- | ------------------ | 598 | | 0x00 | 0 | uint64 | Timestamp | 599 | | 0x08 | 8 | uint16 | Buttons | 600 | | 0x0A | 10 | float32 | Left Trigger | 601 | | 0x0E | 14 | float32 | Right Trigger | 602 | | 0x12 | 18 | float32 | Left Thumbstick X | 603 | | 0x16 | 22 | float32 | Left Thumbstick Y | 604 | | 0x1A | 26 | float32 | Right Thumbstick X | 605 | | 0x1E | 30 | float32 | Right Thumbstick Y | 606 | 607 | - **Timestamp**: Timestamp 608 | - **Buttons**: See Flags [Gamepad Button](#gamepad-button) 609 | - **Left Trigger**: Left Trigger value 610 | - **Right Trigger**: Right trigger value 611 | - **Left Thumbstick X**: Left thumbstick x-axis value 612 | - **Left Thumbstick Y**: Left thumbstick y-axis value 613 | - **Right Thumbstick X**: Right thumbstick x-axis value 614 | - **Right Thumbstick Y**: Right thumbstick y-axis value 615 | 616 | #### Gamepad Button 617 | 618 | | Flag | Bits | Mask | 619 | | ---------------- | --------------------- | ------ | 620 | | Clear | `0000 0000 0000 0000` | 0x00 | 621 | | Enroll | `0000 0000 0000 0001` | 0x01 | 622 | | Nexus | `0000 0000 0000 0010` | 0x02 | 623 | | Menu | `0000 0000 0000 0100` | 0x04 | 624 | | View | `0000 0000 0000 1000` | 0x08 | 625 | | A | `0000 0000 0001 0000` | 0x10 | 626 | | B | `0000 0000 0010 0000` | 0x20 | 627 | | X | `0000 0000 0100 0000` | 0x40 | 628 | | Y | `0000 0000 1000 0000` | 0x080 | 629 | | D-Pad Up | `0000 0001 0000 0000` | 0x100 | 630 | | D-Pad Down | `0000 0010 0000 0000` | 0x200 | 631 | | D-Pad Left | `0000 0100 0000 0000` | 0x400 | 632 | | D-Pad Right | `0000 1000 0000 0000` | 0x800 | 633 | | Left Shoulder | `0001 0000 0000 0000` | 0x1000 | 634 | | Right Shoulder | `0010 0000 0000 0000` | 0x2000 | 635 | | Left Thumbstick | `0100 0000 0000 0000` | 0x4000 | 636 | | Right Thumbstick | `1000 0000 0000 0000` | 0x8000 | 637 | 638 | ### Paired Identity State Changed 639 | 640 | **Message Type**: `0x36` 641 | **Response**: None 642 | **Requests Ack**: `NO` 643 | 644 | Informs client about paired identity state change. 645 | 646 | | Offset (hex) | Offset (dec) | Type | Description | 647 | | -----------: | -----------: | ------ | ----------- | 648 | | 0x00 | 0 | uint16 | State | 649 | 650 | - **State**: See [Paired Identity State](#paired-identity-state) 651 | 652 | #### Paired Identity State 653 | 654 | | State | Value | 655 | | ---------- | ----- | 656 | | Not Paired | 0x00 | 657 | | Paired | 0x01 | 658 | 659 | ### Media State 660 | 661 | **Message Type**: `0xF03` 662 | **Response**: None 663 | **Requests Ack**: `YES` 664 | 665 | Informs client about media playback state. 666 | 667 | | Offset (hex) | Offset (dec) | Type | Description | 668 | | -----------: | -----------: | ------------------- | ---------------- | 669 | | 0x00 | 0 | uint32 | Title Id | 670 | | 0x08 | 8 | SGString | AUM Id | 671 | | 0x0A | 10 | SGString | Asset Id | 672 | | 0x0E | 14 | uint16 | Media Type | 673 | | 0x12 | 18 | uint16 | Sound Level | 674 | | 0x16 | 22 | uint32 | Enabled commands | 675 | | 0x1A | 26 | uint32 | Playback status | 676 | | 0x1E | 30 | float32 | Rate | 677 | | 0x22 | 34 | uint64 | Position | 678 | | 0x2A | 42 | uint64 | Media Start | 679 | | 0x32 | 50 | uint64 | Media End | 680 | | 0x38 | 58 | uint64 | Min Seek | 681 | | 0x42 | 66 | uint64 | Max Seek | 682 | | 0x4A | 74 | uint16 | Metadata Length | 683 | | 0x4C | 76 | MediaMetadata\[len] | Metadata | 684 | 685 | - **Title Id**: Title Id of media 686 | - **AUM Id**: Application User Model Id of media 687 | - **Asset Id**: Asset Id 688 | - **Media Type**: See [Media Type](#media-type) 689 | - **Sound Level**: See [Sound Level](#sound-level) 690 | - **Enabled commands**: See [Media Control Command](#media-control-command) 691 | - **Playback status**: See [Media Playback Status](#media-playback-status) 692 | - **Rate**: Playback rate 693 | - **Position**: Current media position (nanoseconds) 694 | - **Media Start**: Media start (nanoseconds) 695 | - **Media End**: Media end (nanoseconds) 696 | - **Min Seek**: Minimal seek position (nanoseconds) 697 | - **Max Seek**: Maximal seek position (nanoseconds) 698 | - **Metadata Length**: Length of Metadata array 699 | - **Metadata**: Array of [Media Metadata](#media-metadata) 700 | 701 | #### Sound Level 702 | 703 | | Level | Value | 704 | | ----- | ----- | 705 | | Muted | 0x00 | 706 | | Low | 0x01 | 707 | | Full | 0x02 | 708 | 709 | #### Media Playback Status 710 | 711 | | Status | Value | 712 | | -------- | ----- | 713 | | Closed | 0x00 | 714 | | Changing | 0x01 | 715 | | Stopped | 0x02 | 716 | | Playing | 0x03 | 717 | | Paused | 0x04 | 718 | 719 | #### Media Transport State 720 | 721 | | State | Value | 722 | | --------- | ----- | 723 | | Invalid | 0x00 | 724 | | Stopped | 0x01 | 725 | | Starting | 0x02 | 726 | | Playing | 0x03 | 727 | | Paused | 0x04 | 728 | | Buffering | 0x05 | 729 | 730 | #### Media Type 731 | 732 | | Type | Value | 733 | | ------------ | ----- | 734 | | NoMedia | 0x00 | 735 | | Music | 0x01 | 736 | | Video | 0x02 | 737 | | Image | 0x03 | 738 | | Conversation | 0x04 | 739 | | Game | 0x05 | 740 | 741 | #### Media Metadata 742 | 743 | | Offset (hex) | Offset (dec) | Type | Description | 744 | | -----------: | -----------: | -------- | ----------- | 745 | | 0x00 | 0 | SGString | Name | 746 | | 0x?? | ? | SGString | Value | 747 | 748 | ### Media Controller Removed 749 | 750 | **Message Type**: `0xF00` 751 | **Response**: None 752 | **Requests Ack**: `YES` 753 | 754 | Informs client about removed media controller. 755 | 756 | | Offset (hex) | Offset (dec) | Type | Description | 757 | | -----------: | -----------: | ------ | ----------- | 758 | | 0x00 | 0 | uint32 | Title Id | 759 | 760 | - **Title Id**: Title Id of removed media controller 761 | 762 | ### Media Command Result 763 | 764 | **Message Type**: `0xF02` 765 | **Response**: None 766 | **Requests Ack**: `YES` 767 | 768 | Informs client wether Media Command succeeded. 769 | 770 | | Offset (hex) | Offset (dec) | Type | Description | 771 | | -----------: | -----------: | ------ | ----------- | 772 | | 0x00 | 0 | uint64 | Request Id | 773 | | 0x08 | 8 | uint32 | Result | 774 | 775 | - **Request Id**: Match with request Id of [Media Command](#media-command) 776 | - **Result**: Result, `0` is success 777 | 778 | ### Media Command 779 | 780 | **Message Type**: `0xF01` 781 | **Response**: [Media Command Result](#media-command-result) 782 | **Requests Ack**: `NO` 783 | 784 | Sends media playback command to console. 785 | 786 | | Offset (hex) | Offset (dec) | Type | Description | 787 | | -----------: | -----------: | ------ | ----------- | 788 | | 0x00 | 0 | uint64 | Request Id | 789 | | 0x08 | 8 | uint32 | Title Id | 790 | | 0x0C | 12 | uint32 | Command | 791 | 792 | | If Command == Seek 793 | | 0x10 | 16 | uint32 | Seek position 794 | 795 | - **Request Id**: Request Id 796 | - **Title Id**: Title Id of media controller 797 | - **Command**: See [Media Control Command](#media-control-command) 798 | - **Seek position (optional)**: Only set if Command == Seek 799 | 800 | #### Media Control Command 801 | 802 | | Flag | Bits | Mask | 803 | | ----------------- | --------------------- | ------ | 804 | | Play | `0000 0000 0000 0010` | 0x02 | 805 | | Pause | `0000 0000 0000 0100` | 0x04 | 806 | | Play Pause Toggle | `0000 0000 0000 1000` | 0x08 | 807 | | Stop | `0000 0000 0001 0000` | 0x10 | 808 | | Record | `0000 0000 0010 0000` | 0x20 | 809 | | Next Track | `0000 0000 0100 0000` | 0x40 | 810 | | Previous Track | `0000 0000 1000 0000` | 0x080 | 811 | | Fast Forward | `0000 0001 0000 0000` | 0x100 | 812 | | Rewind | `0000 0010 0000 0000` | 0x200 | 813 | | Channel Up | `0000 0100 0000 0000` | 0x400 | 814 | | Channel Down | `0000 1000 0000 0000` | 0x800 | 815 | | Back | `0001 0000 0000 0000` | 0x1000 | 816 | | View | `0010 0000 0000 0000` | 0x2000 | 817 | | Menu | `0100 0000 0000 0000` | 0x4000 | 818 | | Seek | `1000 0000 0000 0000` | 0x8000 | 819 | 820 | ### Orientation 821 | 822 | **Message Type**: `0x33` 823 | **Response**: None 824 | **Requests Ack**: `NO` 825 | 826 | Send orientation message to console. 827 | 828 | | Offset (hex) | Offset (dec) | Type | Description | 829 | | -----------: | -----------: | ------- | --------------------- | 830 | | 0x00 | 0 | uint64 | Timestamp | 831 | | 0x08 | 8 | float32 | Rotation Matrix Value | 832 | | 0x0C | 12 | float32 | W | 833 | | 0x10 | 16 | float32 | X | 834 | | 0x14 | 20 | float32 | Y | 835 | | 0x18 | 24 | float32 | Z | 836 | 837 | - **Timestamp**: Timestamp 838 | - **Rotation Matrix Value**: Rotation Matrix Value 839 | - **W**: Value of W-axis 840 | - **X**: Value of X-axis 841 | - **Y**: Value of Y-axis 842 | - **Z**: Value of Z-axis 843 | 844 | ### Compass 845 | 846 | **Message Type**: `0x32` 847 | **Response**: None 848 | **Requests Ack**: `NO` 849 | 850 | Send compass message to console. 851 | 852 | | Offset (hex) | Offset (dec) | Type | Description | 853 | | -----------: | -----------: | ------- | -------------- | 854 | | 0x00 | 0 | uint64 | Timestamp | 855 | | 0x08 | 8 | float32 | Magnetic North | 856 | | 0x0C | 12 | float32 | True North | 857 | 858 | - **Timestamp**: Timestamp 859 | - **Magnetic North**: Magnetic North 860 | - **True North**: True North 861 | 862 | ### Inclinometer 863 | 864 | **Message Type**: `0x31` 865 | **Response**: None 866 | **Requests Ack**: `NO` 867 | 868 | Send inclinometer message to console. 869 | 870 | | Offset (hex) | Offset (dec) | Type | Description | 871 | | -----------: | -----------: | ------- | ----------- | 872 | | 0x00 | 0 | uint64 | Timestamp | 873 | | 0x08 | 8 | float32 | Pitch | 874 | | 0x0C | 12 | float32 | Roll | 875 | | 0x10 | 16 | float32 | Yaw | 876 | 877 | - **Timestamp**: Timestamp 878 | - **Pitch**: Pitch 879 | - **Roll**: Roll 880 | - **Yaw**: Yaw 881 | 882 | ### Gyrometer 883 | 884 | **Message Type**: `0x30` 885 | **Response**: None 886 | **Requests Ack**: `NO` 887 | 888 | Send gyrometer message to console. 889 | 890 | | Offset (hex) | Offset (dec) | Type | Description | 891 | | -----------: | -----------: | ------- | ------------------ | 892 | | 0x00 | 0 | uint64 | Timestamp | 893 | | 0x08 | 8 | float32 | Angular Velocity X | 894 | | 0x0C | 12 | float32 | Angular Velocity Y | 895 | | 0x10 | 16 | float32 | Angular Velocity Z | 896 | 897 | - **Timestamp**: Timestamp 898 | - **Angular Velocity X**: Angular Velocity X-axis 899 | - **Angular Velocity Y**: Angular Velocity Y-axis 900 | - **Angular Velocity Z**: Angular Velocity Z-axis 901 | 902 | ### Accelerometer 903 | 904 | **Message Type**: `0x` 905 | **Response**: None 906 | **Requests Ack**: `NO` 907 | 908 | Send accelerometer message to console. 909 | 910 | | Offset (hex) | Offset (dec) | Type | Description | 911 | | -----------: | -----------: | ------- | -------------- | 912 | | 0x00 | 0 | uint64 | Timestamp | 913 | | 0x08 | 8 | float32 | Acceleration X | 914 | | 0x0C | 12 | float32 | Acceleration Y | 915 | | 0x10 | 16 | float32 | Acceleration Z | 916 | 917 | - **Timestamp**: Timestamp 918 | - **Acceleration X**: Acceleration X-axis 919 | - **Acceleration Y**: Acceleration Y-axis 920 | - **Acceleration Z**: Acceleration Z-axis 921 | 922 | ### Touch 923 | 924 | **Message Type**: `0x2E` (Title) or `0xF2E` (System) 925 | **Response**: None 926 | **Requests Ack**: `NO` 927 | 928 | Send touch input message to console. 929 | 930 | | Offset (hex) | Offset (dec) | Type | Description | 931 | | ------------ | ------------ | ------------------ | ------------------- | 932 | | 0x00 | 0 | uint32 | Touch Msg Timestamp | 933 | | 0x04 | 4 | uint16 |  Touch Count | 934 | | 0x06 | 6 | Touchpoint\[count] | Touches | 935 | 936 | - **Touch Msg Timestamp**: Timestamp 937 | - **Touch Count**: Number of touchpoints 938 | - **Touches**: Array of [Touchpoint](#touchpoint) 939 | 940 | #### Touch Action 941 | 942 | | Action | Value | 943 | | ------ | ----- | 944 | | Down | 0x01 | 945 | | Move | 0x02 | 946 | | Up | 0x03 | 947 | | Cancel | 0x04 | 948 | 949 | #### Touchpoint 950 | 951 | | Offset (hex) | Offset (dec) | Type | Description | 952 | | -----------: | -----------: | ------ | ----------------- | 953 | | 0x00 | 0 | uint32 | Touchpoint Id | 954 | | 0x04 | 4 | uint16 | Touchpoint Action | 955 | | 0x06 | 6 | uint32 | Touchpoint X | 956 | | 0x0A | 10 | uint32 | Touchpoint Y | 957 | 958 | - **Touchpoint Id**: Id of Touchpoint 959 | - **Touchpoint Action**: See [Touch Action](#touch-action) 960 | - **Touchpoint X**: Touchpoint X-Axis 961 | - **Touchpoint Y**: Touchpoint X-Axis 962 | 963 | ### Title Launch 964 | 965 | **Message Type**: `0x23` 966 | **Response**: None 967 | **Requests Ack**: `YES` 968 | 969 | Launch a title / URL on the console. 970 | 971 | | Offset (hex) | Offset (dec) | Type | Description | 972 | | -----------: | -----------: | -------- | ----------- | 973 | | 0x00 | 0 | uint16 | Location | 974 | | 0x02 | 2 | SGString | Uri | 975 | 976 | - **Location**: Usually `0` 977 | - **Uri**: Uri to launch 978 | 979 | ### System Text Done 980 | 981 | **Message Type**: `0xF35` 982 | **Response**: None 983 | **Requests Ack**: `YES` 984 | 985 | | Offset (hex) | Offset (dec) | Type | Description | 986 | | -----------: | -----------: | ------ | --------------- | 987 | | 0x00 | 0 | uint32 | Text Session Id | 988 | | 0x04 | 4 | uint32 | Text Version | 989 | | 0x08 | 8 | uint32 | Flags | 990 | | 0x0C | 12 | uint32 | Result | 991 | 992 | - **Text Session Id**: Text session id 993 | - **Text Version**: Text version 994 | - **Flags**: Flags 995 | - **Result**: See [Text Result](#text-result) 996 | 997 | #### Text Result 998 | 999 | | Result | Value | 1000 | | ------ | ----- | 1001 | | Cancel | 0x00 | 1002 | | Accept | 0x01 | 1003 | 1004 | ### System Text Acknowledge 1005 | 1006 | **Message Type**: `0xF34` 1007 | **Response**: None 1008 | **Requests Ack**: `YES` 1009 | 1010 | | Offset (hex) | Offset (dec) | Type | Description | 1011 | | -----------: | -----------: | ------ | ---------------- | 1012 | | 0x00 | 0 | uint32 | Text Session Id | 1013 | | 0x04 | 4 | uint32 | Text Version Ack | 1014 | 1015 | - **Text Session Id**: Text session id 1016 | - **Text Version Ack**: Text version to acknowledge 1017 | 1018 | ### System Text Input 1019 | 1020 | **Message Type**: `0xF2C` 1021 | **Response**: [System Text Acknowledge](#system-text-acknowledge) 1022 | **Requests Ack**: `NO` 1023 | 1024 | | Offset (hex) | Offset (dec) | Type | Description | 1025 | | -----------: | -----------: | ----------- | --------------------- | 1026 | | 0x00 | 0 | uint32 | Text Session Id | 1027 | | 0x04 | 4 | uint32 | Base Version | 1028 | | 0x08 | 8 | uint32 | Submitted Version | 1029 | | 0x0C | 12 | uint32 | Total Text bytelength | 1030 | | 0x10 | 16 | uint32 | Selection Start | 1031 | | 0x14 | 20 | uint32 | Selection Length | 1032 | | 0x18 | 24 | uint16 | Flags | 1033 | | 0x1A | 26 | uint32 | Text Chunk bytestart | 1034 | | 0x1E | 30 | SGString | Text Chunk | 1035 | | 0x?? | ?? | uint16 | Delta Length | 1036 | | 0x?? | ?? | Delta\[len] | Text Delta | 1037 | 1038 | - **Text Session Id**: Text session id 1039 | - **Base Version**: Base version 1040 | - **Submitted Version**: Submitted version 1041 | - **Total Text bytelength**: Total bytelength of text 1042 | - **Selection Start**: Selection start 1043 | - **Selection Length**: Selection length 1044 | - **Flags**: Flags 1045 | - **Text Chunk bytestart**: Bytestart of textchunk 1046 | - **Text Chunk**: Actual text chunk to send 1047 | - **Delta Length**: Count of Text Delta 1048 | - **Text Delta**: See [Text Delta](#text-delta) 1049 | 1050 | #### Text Delta 1051 | 1052 | | Offset (hex) | Offset (dec) | Type | Description | 1053 | | -----------: | -----------: | -------- | -------------- | 1054 | | 0x00 | 0 | uint32 | Offset | 1055 | | 0x04 | 4 | uint32 | Delete Count | 1056 | | 0x08 | 8 | SGString | Insert Content | 1057 | 1058 | ### Title Text Selection 1059 | 1060 | **Message Type**: `0x21` 1061 | **Response**: None 1062 | **Requests Ack**: `YES` 1063 | 1064 | | Offset (hex) | Offset (dec) | Type | Description | 1065 | | -----------: | -----------: | ------ | ------------------- | 1066 | | 0x00 | 0 | uint64 | Text Session Id | 1067 | | 0x08 | 8 | uint32 | Text Buffer Version | 1068 | | 0x0C | 12 | uint32 | Start | 1069 | | 0x10 | 16 | uint32 | Length | 1070 | 1071 | - **Text Session Id**: Text session id 1072 | - **Text Buffer Version**: Text buffer version 1073 | - **Start**: Start 1074 | - **Length**: Length 1075 | 1076 | ### Title Text Input 1077 | 1078 | **Message Type**: `0x` 1079 | **Response**: None 1080 | **Requests Ack**: `YES` 1081 | 1082 | | Offset (hex) | Offset (dec) | Type | Description | 1083 | | -----------: | -----------: | -------- | ------------------- | 1084 | | 0x00 | 0 | uint64 | Text Session Id | 1085 | | 0x08 | 8 | uint32 | Text Buffer Version | 1086 | | 0x0C | 12 | uint16 | Result | 1087 | | 0x0E | 14 | SGString | Text | 1088 | 1089 | - **Text Session Id**: Text session id 1090 | - **Text Buffer Version**: Text buffer version 1091 | - **Result**: See [Text Result](#text-result) 1092 | - **Text**: Actual text 1093 | 1094 | ### Text Configuration 1095 | 1096 | **Message Type**: `0x1F` (Title) or `0xF2B` (System) 1097 | **Response**: None 1098 | **Requests Ack**: `YES` 1099 | 1100 | | Offset (hex) | Offset (dec) | Type | Description | 1101 | | -----------: | -----------: | -------- | ------------------- | 1102 | | 0x00 | 0 | uint64 | Text Session Id | 1103 | | 0x08 | 8 | uint32 | Text Buffer Version | 1104 | | 0x0C | 12 | uint32 | Text options | 1105 | | 0x10 | 16 | uint32 | Input Scope | 1106 | | 0x14 | 20 | uint32 | Max Text Length | 1107 | | 0x18 | 24 | SGString | Locale | 1108 | | 0x?? | ?? | SGString | Prompt | 1109 | 1110 | - **Text Session Id**: Text session id 1111 | - **Text Buffer Version**: Text buffer version 1112 | - **Text options**: See [Text Option Flags](#text-option-flags) 1113 | - **Input Scope**: See [Text Input Scope](#text-input-scope) 1114 | - **Max Text Length**: Maximal text length 1115 | - **Locale**: Locale to use 1116 | - **Prompt**: Text input prompt 1117 | 1118 | #### Text Input Scope 1119 | 1120 | | Scope | Value | 1121 | | ------------------------ | ----- | 1122 | | Default | 0x0 | 1123 | | Url | 0x1 | 1124 | | Full FilePath | 0x2 | 1125 | | File Name | 0x3 | 1126 | | Email UserName | 0x4 | 1127 | | Email SmtpAddress | 0x5 | 1128 | | LogOn Name | 0x6 | 1129 | | Personal FullName | 0x7 | 1130 | | Personal NamePrefix | 0x8 | 1131 | | Personal GivenName | 0x9 | 1132 | | Personal MiddleName | 0xA | 1133 | | Personal Surname | 0xB | 1134 | | Personal NameSuffix | 0xC | 1135 | | Postal Address | 0xD | 1136 | | Postal Code | 0xE | 1137 | | Address Street | 0xF | 1138 | | Address StateOrProvince | 0x10 | 1139 | | Address City | 0x11 | 1140 | | Address CountryName | 0x12 | 1141 | | Address CountryShortName | 0x13 | 1142 | | Currency AmountAndSymbol | 0x14 | 1143 | | Currency Amount | 0x15 | 1144 | | Date | 0x16 | 1145 | | Date Month | 0x17 | 1146 | | Date Day | 0x18 | 1147 | | Date Year | 0x19 | 1148 | | Date MonthName | 0x1A | 1149 | | Date DayName | 0x1B | 1150 | | Digits | 0x1C | 1151 | | Number | 0x1D | 1152 | | OneChar | 0x1E | 1153 | | Password | 0x1F | 1154 | | Telephone Number | 0x20 | 1155 | | Telephone CountryCode | 0x21 | 1156 | | Telephone AreaCode | 0x22 | 1157 | | Telephone LocalNumber | 0x23 | 1158 | | Time | 0x24 | 1159 | | Time Hour | 0x25 | 1160 | | Time MinorSec | 0x26 | 1161 | | Number FullWidth | 0x27 | 1162 | | Alphanumeric HalfWidth | 0x28 | 1163 | | Alphanumeric FullWidth | 0x29 | 1164 | | Currency Chinese | 0x2A | 1165 | | Bopomofo | 0x2B | 1166 | | Hiragana | 0x2C | 1167 | | Katakana HalfWidth | 0x2D | 1168 | | Katakana FullWidth | 0x2E | 1169 | | Hanja | 0x2F | 1170 | | Hangul HalfWidth | 0x30 | 1171 | | Hangul FullWidth | 0x31 | 1172 | | Search | 0x32 | 1173 | | Search TitleText | 0x33 | 1174 | | Search Incremental | 0x34 | 1175 | | Chinese HalfWidth | 0x35 | 1176 | | Chinese FullWidth | 0x36 | 1177 | | NativeScript | 0x37 | 1178 | 1179 | #### Text Option Flags 1180 | 1181 | | Flag | Bits | Mask | 1182 | | ------------------- | --------------------- | ------ | 1183 | | Default | `0000 0000 0000 0000` | 0x00 | 1184 | | Accepts Return | `0000 0000 0000 0001` | 0x01 | 1185 | | Password | `0000 0000 0000 0010` | 0x02 | 1186 | | Multi Line | `0000 0000 0000 0100` | 0x04 | 1187 | | Spell Check Enabled | `0000 0000 0000 1000` | 0x08 | 1188 | | Prediction Enabled | `0000 0000 0001 0000` | 0x10 | 1189 | | RTL | `0000 0000 0010 0000` | 0x20 | 1190 | | Dismiss | `0100 0000 0000 0000` | 0x4000 | 1191 | -------------------------------------------------------------------------------- /docs/nano.md: -------------------------------------------------------------------------------- 1 | # Nano Protocol 2 | 3 | Nano (aka. Nano RDP / Codename: Arcadia) is the protocol Gamestreaming is based on. 4 | You can see strong similarities when looking at the Windows IoT RDP server implementation, _NanoRDPServer.exe_. 5 | 6 | It's basically RTP over TCP (configuration / status of session) and UDP (data). 7 | 8 | - [Nano Protocol](#nano-protocol) 9 | - [How it works](#how-it-works) 10 | - [TCP vs. UDP](#tcp-vs-udp) 11 | - [Data layout](#data-layout) 12 | - [TCP Socket](#tcp-socket) 13 | - [Padding](#padding) 14 | - [Example](#example) 15 | - [RTP Header](#rtp-header) 16 | - [Channels](#channels) 17 | - [Payload Types](#payload-types) 18 | - [Streamer Protocol Version](#streamer-protocol-version) 19 | - [Channel Opening](#channel-opening) 20 | - [Streamer Handshaking](#streamer-handshaking) 21 | - [Packets](#packets) 22 | - [Packet Layout](#packet-layout) 23 | - [Control Handshake Packet](#control-handshake-packet) 24 | - [Channel Control Packet](#channel-control-packet) 25 | - [Channel Control Header](#channel-control-header) 26 | - [Channel Control Types](#channel-control-types) 27 | - [Channel Control Payloads](#channel-control-payloads) 28 | - [Create](#create) 29 | - [Open](#open) 30 | - [Close](#close) 31 | - [UDP Handshake Packet](#udp-handshake-packet) 32 | - [Streamer Packet](#streamer-packet) 33 | - [Streamer Flags](#streamer-flags) 34 | - [Streamer Header](#streamer-header) 35 | - [Audio Video Streamer Payload Type](#audio-video-streamer-payload-type) 36 | - [Input Payload Type](#input-payload-type) 37 | - [Streamer Payloads](#streamer-payloads) 38 | - [Reference Timestamp](#reference-timestamp) 39 | - [Timestamp of Data Packets](#timestamp-of-data-packets) 40 | - [Frame Id](#frame-id) 41 | - [Audio](#audio) 42 | - [Audio Format](#audio-format) 43 | - [Audio Codec](#audio-codec) 44 | - [Audio Server Handshake](#audio-server-handshake) 45 | - [Audio Client Handshake](#audio-client-handshake) 46 | - [Audio Control](#audio-control) 47 | - [Audio Control Flags](#audio-control-flags) 48 | - [Audio Data](#audio-data) 49 | - [Video](#video) 50 | - [Lost frame reporting](#lost-frame-reporting) 51 | - [Video Format](#video-format) 52 | - [Video Codec](#video-codec) 53 | - [Video Server Handshake](#video-server-handshake) 54 | - [Video Client Handshake](#video-client-handshake) 55 | - [Video Control](#video-control) 56 | - [Video Control Flags](#video-control-flags) 57 | - [Video Data](#video-data) 58 | - [Video Data Flags](#video-data-flags) 59 | - [Input](#input) 60 | - [Input Button Model](#input-button-model) 61 | - [Input Analog Model](#input-analog-model) 62 | - [Input Extension Model](#input-extension-model) 63 | - [Input Server Handshake](#input-server-handshake) 64 | - [Input Client Handshake](#input-client-handshake) 65 | - [Frame Ack](#frame-ack) 66 | - [Input Frame](#input-frame) 67 | - [Control Protocol](#control-protocol) 68 | - [Control Header](#control-header) 69 | - [Control Payload Type](#control-payload-type) 70 | - [Session Init](#session-init) 71 | - [Session Create](#session-create) 72 | - [Session Create Response](#session-create-response) 73 | - [Session Destroy](#session-destroy) 74 | - [Video Statistics](#video-statistics) 75 | - [Realtime Telemetry](#realtime-telemetry) 76 | - [Telemetry Field](#telemetry-field) 77 | - [Change Video Quality](#change-video-quality) 78 | - [Initiate Network Test](#initiate-network-test) 79 | - [Network Information](#network-information) 80 | - [Network Test Response](#network-test-response) 81 | - [Controller Event](#controller-event) 82 | - [Controller Event Type](#controller-event-type) 83 | 84 | ## How it works 85 | 86 | 1. Client opens [Broadcast Channel](channels.md#broadcast-channel) in SmartGlass Protocol and requests 87 | start of gamestreaming to initialize and receive the connection-data 88 | [`TCP Port`, `UDP Port` and `Session ID`](channels.md#initializing). 89 | 2. Client creates sockets for `TCP` and `UDP`. 90 | 3. Client sends [Control Handshake Packet](#control-handshake-packet) with 91 | own, randomly generated, `Connection Id`. 92 | 4. Host responds with [Control Handshake Packet](#control-handshake-packet) 93 | and it's own Connection Id. 94 | 5. Client starts a background loop, sending [UDP Handshake](#udp-handshake-packet) packets. 95 | 6. Host sends [`Channel Create` and `Channel Open`](#channel-control-packet) packets 96 | 7. Client replies with [Channel Open](#open) packets for the appropriate channels 97 | 8. Host sends [Server Handshake](#audio-video-streamer-payload-type) packets for 98 | [Video](#video) and [Audio](#audio), announcing it's available formats 99 | 9. Client replies with [Client Handshake](#audio-video-streamer-payload-type) packets, 100 | choosing the desired format. 101 | 10. Client sends [Audio / Video Control](#audio-video-streamer-payload-type) packets, 102 | starting the stream. 103 | 11. UDP Data comes flying in. 104 | 12. Client stops [UDP Handshake](#udp-handshake-packet) loop. 105 | 13. ... Client processes data ... 106 | 107 | ## TCP vs. UDP 108 | 109 | Nano uses these two IP protocols in the following ways: 110 | 111 | **TCP** 112 | 113 | - [Control Handshaking](#control-handshake-packet) 114 | - [Creating/Opening/Closing Channels](#channel-control-packet) 115 | - Streamer Client/Server **Handshaking** 116 | - Streamer **control** messaging 117 | - [Control Protocol messaging](#control-protocol) 118 | 119 | **UDP** 120 | 121 | - [UDP handshaking](#udp-handshake-packet) 122 | - Streamer **data** packets 123 | 124 | ## Data layout 125 | 126 | The RTP header is in network / big-endian byteorder, the rest of 127 | the packet is usually in little-endian byteorder. 128 | 129 | ### TCP Socket 130 | 131 | All TCP packets have a nano-packet size prepended 132 | (`uint32`, Little Endian), so several packets can be sent _stacked_. 133 | Example of a single TCP packet with two nano packets inside: 134 | 135 | ```text 136 | packet size #1: 0x123 137 | nano packet #1: byte[0x123] 138 | packet size #2: 0x321 139 | nano packet #2: byte[0x321] 140 | ``` 141 | 142 | Ideally the chunks are split directly when received from the socket 143 | and then forwarded to the unpacker/processor/parser. 144 | 145 | ## Padding 146 | 147 | Nano, different to SmartGlass core, uses padding of type 148 | [ANSI X.923](https://en.wikipedia.org/wiki/Padding_(cryptography)#ANSI_X.923) 149 | aka. padding with **zero**, the last byte defining the number of padding bytes. 150 | 151 | Aligment is **4 bytes**. 152 | 153 | ### Example 154 | 155 | **Plaintext (9 bytes)** 156 | `DE AD BE EF DE AD BE EF DE` 157 | 158 | **Padded Plaintext (12 bytes) - 3 bytes padding** 159 | `DE AD BE EF DE AD BE EF DE 00 00 03` 160 | 161 | ## RTP Header 162 | 163 | ![Rtp Header](rtpheader.png) 164 | 165 | Image Source: 166 | 167 | Image Copyright License: **CC-by-sa-3.0** 168 | 169 | - **Total size**: 0x0C (12) 170 | - **V**: Version (2 bits): Always 2 171 | - **P**: Padding (1 bit): If set, last byte of payload is padding size. 172 | - **X**: Extension (1 bit): If set, variable size header 173 | extension exists - **Not used by Nano** 174 | - **CSRC Count**: (4 bits) Should be 0 - **Not used by Nano** 175 | - **M**: Marker (1 bit): Indicates a _special packet_ in stream data 176 | - **Payload type**: (7 bits) See [Payload Types](#payload-types) 177 | - **Sequence number**: (16 bits) Incrementing number, indvidual to each 178 | channel, used to keep track of order and lost packets 179 | - **Timestamp**: (32 bits) Timestamp, unsure how it's actually calculated 180 | - **SSRC**: (32 bits) For Nano its split into 2 x 16 bits: 181 | 182 | > NOTE: Only UDP packets use `Connection Id` field, TCP sets it to `0` 183 | > 184 | > - Bits 0-15: Connection Id 185 | > - Bits 16-32: Channel Id 186 | 187 | ## Channels 188 | 189 | Nano communicates over the following channels + a core channel (`0`). 190 | Core Channel handles: 191 | 192 | - [Control Handshake](#control-handshake-packet) 193 | - [UDP Handshake](#udp-handshake-packet) 194 | - [Channel Control Packet](#channel-control-packet) 195 | 196 | The following listed channels only process messages of type `Streamer`. 197 | 198 | > Usage of `TcpBase` has not been spotted in the wild to date. 199 | 200 | | Channel | Id | 201 | | -------------- | ----------------------------------------------------- | 202 | | Video | `Microsoft::Rdp::Dct::Channel::Class::Video` | 203 | | Audio | `Microsoft::Rdp::Dct::Channel::Class::Audio` | 204 | | Chat Audio | `Microsoft::Rdp::Dct::Channel::Class::ChatAudio` | 205 | | Control | `Microsoft::Rdp::Dct::Channel::Class::Control` | 206 | | Input | `Microsoft::Rdp::Dct::Channel::Class::Input` | 207 | | Input Feedback | `Microsoft::Rdp::Dct::Channel::Class::Input Feedback` | 208 | | TCP Base | `Microsoft::Rdp::Dct::Channel::Class::TcpBase` | 209 | 210 | ## Payload Types 211 | 212 | Payload Type is encoded in the [RTP Header](#rtp-header) `Payload type` field. 213 | 214 | | Payload Type | Value | 215 | | --------------- | ----- | 216 | | Streamer | 0x23 | 217 | | Control | 0x60 | 218 | | Channel Control | 0x61 | 219 | | UDP Handshake | 0x64 | 220 | 221 | - **Streamer**: Sending encoded video/audio/input data, 222 | sending [Control Protocol packets](#control-protocol) 223 | - **Control**: Initial TCP handshake, informs each participants 224 | about the used Connection Id. 225 | - **Channel Control**: Creating / opening / closing [Channels](#channels). 226 | - **UDP Handshake**: Initial UDP handshake, used to inform 227 | the host about the used UDP port of the client side. 228 | 229 | ## Streamer Protocol Version 230 | 231 | Currently these are the used streamer protocol version: 232 | 233 | - **Video**: Version 5 234 | - **Audio**: Version 4 235 | - **Input**: Version 3 236 | 237 | ## Channel Opening 238 | 239 | By default, console [creates](#create) and [opens](#open) Channels for: 240 | 241 | - Audio 242 | - Video 243 | - ChatAudio 244 | - Control 245 | 246 | Client needs to respond with a [Channel Open](#open) packet. 247 | After that, handshaking needs to be done, see [Streamer Handshaking](#streamer-handshaking) 248 | 249 | > NOTE: `Input` and `Input Feedback` are somewhat special, see 250 | > [Input](#input) 251 | 252 | ## Streamer Handshaking 253 | 254 | Normally, a `Server Handshake` is sent by the console, client has to 255 | respond with a `Client Handshake`. 256 | 257 | > NOTE: [Input Feedback](#input) and [Chat Audio](#audio) are special! 258 | 259 | ## Packets 260 | 261 | ### Packet Layout 262 | 263 | ``` 264 | ├── RtpHeader 265 | └── Payload (RtpHeader.PayloadType) 266 | ├── Control Handshake 267 | ├── Channel Control 268 | │ ├── Channel Create 269 | │ ├── Channel Open 270 | │ └── Channel Close 271 | ├── UDP Handshake 272 | └── Streamer 273 | ├── TCP Header 274 | ├── UDP Header 275 | ├── Audio Payload 276 | │   ├── Server Handshake 277 | │   ├── Client Handshake 278 | │   ├── Control 279 | │   └── Data 280 | ├── Video Payload 281 | │   ├── Server Handshake 282 | │   ├── Client Handshake 283 | │   ├── Control 284 | │   └── Data 285 | ├── Input Payload 286 | │   ├── Server Handshake 287 | │   ├── Client Handshake 288 | │   ├── Frame Ack 289 | │   └── Frame 290 | └── Control Payload 291 | └── Control Header 292 | ├── Session Init 293 | ├── Session Create 294 | ├── Session Create Response 295 | ├── Session Destroy 296 | ├── Video Statistics 297 | ├── Realtime Telemetry 298 | ├── Change Video Quality 299 | ├── Initiate Network Test 300 | ├── Network Information 301 | ├── Network Test Response 302 | └── Controller Event 303 | ``` 304 | 305 | ### Control Handshake Packet 306 | 307 | | Offset (hex) | Offset (dec) | Type | Description | 308 | | -----------: | -----------: | ------ | ------------- | 309 | | 0x00 | 0 | byte | Type | 310 | | 0x01 | 1 | uint16 | Connection Id | 311 | 312 | **Total size**: 0x03 (3) 313 | 314 | - **Type**: Handshake type: SYN: 0 - Sent by client, ACK: 1 - Response from console 315 | - **Connection Id**: Client sends randomly generated connection 316 | Id, Host responds with it's own. 317 | 318 | ### Channel Control Packet 319 | 320 | #### Channel Control Header 321 | 322 | | Offset (hex) | Offset (dec) | Type | Description | 323 | | -----------: | -----------: | ------- | -------------------- | 324 | | 0x00 | 0 | uint32 | Channel Control Type | 325 | | 0x04 | 4 | byte\[] | Payload | 326 | 327 | **Total size**: _variable_ 328 | 329 | - **Channel Control Type**: See [Channel Control Types](#channel-control-types) 330 | - **Payload**: Depending on Type-field 331 | 332 | #### Channel Control Types 333 | 334 | | Channel Control | Value | 335 | | --------------- | ----- | 336 | | Create | 0x02 | 337 | | Open | 0x03 | 338 | | Close | 0x04 | 339 | 340 | #### Channel Control Payloads 341 | 342 | ##### Create 343 | 344 | **Channel Control Type**: `0x02` 345 | 346 | | Offset (hex) | Offset (dec) | Type | Description | 347 | | -----------: | -----------: | ----------- | ------------ | 348 | | 0x00 | 0 | uint16 | Name length | 349 | | 0x02 | 2 | uchar\[len] | Channel Name | 350 | | 0x02 + len | 2 + len | uint32 | Flags | 351 | 352 | **Total size**: _variable_ 353 | 354 | - **Channel Name**: See [Channels](#channels) 355 | - **Flags**: Unknown 356 | 357 | ##### Open 358 | 359 | **Channel Control Type**: `0x03` 360 | 361 | | Offset (hex) | Offset (dec) | Type | Description | 362 | | -----------: | -----------: | ---------- | ------------ | 363 | | 0x00 | 0 | uint32 | Flags length | 364 | | 0x04 | 4 | byte\[len] | Flags | 365 | 366 | **Total size**: _variable_ 367 | 368 | - **Flags (optional)**: Depending on prepended size. 369 | Are sent by the client _as-is_ in the responding [Channel Open](#open) packet. 370 | 371 | ##### Close 372 | 373 | **Channel Control Type**: `0x04` 374 | 375 | | Offset (hex) | Offset (dec) | Type | Description | 376 | | -----------: | -----------: | ------ | ----------- | 377 | | 0x00 | 0 | uint32 | Flags | 378 | 379 | **Total size**: 0x04 (4) 380 | 381 | - **Flags**: Same flags as sent with `Create` and `Open` packet 382 | 383 | ### UDP Handshake Packet 384 | 385 | | Offset (hex) | Offset (dec) | Type | Description | 386 | | -----------: | -----------: | ---- | ----------- | 387 | | 0x00 | 0 | byte | Type | 388 | 389 | **Total size**: 0x01 (1) 390 | 391 | - **Type**: Handshake Type 392 | 393 | ### Streamer Packet 394 | 395 | #### Streamer Flags 396 | 397 | | Flag | Value | 398 | | ---------------- | ----- | 399 | | Got Seq/Prev Seq | 0x01 | 400 | | Unknown | 0x02 | 401 | 402 | #### Streamer Header 403 | 404 | UDP usually uses flags: `0x0`. 405 | TCP usually uses flags: `0x3`. 406 | For FEC there are additional flags to process (TODO). 407 | 408 | | Offset (hex) | Offset (dec) | Type | Description | 409 | | -----------: | -----------: | ------------ | -------------------------- | 410 | | 0x00 | 0 | uint32 | Flags | 411 | | \*0x?? | \*?? | uint32 | \*Sequence Number | 412 | | \*0x?? | \*?? | uint32 | \*Previous Sequence Number | 413 | | 0x?? | ?? | uint32 | Streamer Payload Type | 414 | | \*0x?? | \*?? | uint32 | \*Streamer Payload Length | 415 | | 0x?? | ?? | byte\[\*len] | Streamer Payload | 416 | 417 | **Total size**: _variable_ 418 | 419 | > **Streamer Payload Length**: Only used if `Streamer Payload Type` 420 | > is **not** 0 (0: Control packet with own header) 421 | 422 | - **Flags**: Depending on used [Channel](#channels) 423 | - **Sequence Number**: If Flag `Got Seq/Prev Seq` is set -> incrementing number, 424 | specific for channel and participant side 425 | - **Previous Sequence Number**: If Flag `Got Seq/Prev Seq` is set -> previously 426 | sent `Sequence Number` 427 | - **Streamer Payload Type**: See [Audio / Video Payload Type](#audio-video-streamer-payload-type) 428 | and [Input Payload Type](#input-payload-type) 429 | - **Streamer Payload**: Depending on `Streamer Payload Type` 430 | 431 | #### Audio Video Streamer Payload Type 432 | 433 | | Type | Value | 434 | | ---------------- | ----- | 435 | | Server Handshake | 0x01 | 436 | | Client Handshake | 0x02 | 437 | | Control | 0x03 | 438 | | Data | 0x04 | 439 | 440 | #### Input Payload Type 441 | 442 | | Type | Value | 443 | | ---------------- | ----- | 444 | | Server Handshake | 0x01 | 445 | | Client Handshake | 0x02 | 446 | | Frame Ack | 0x03 | 447 | | Frame | 0x04 | 448 | 449 | ## Streamer Payloads 450 | 451 | ### Reference Timestamp 452 | 453 | Reference Timestamp is **milliseconds** since epoch. 454 | 455 | ### Timestamp of Data Packets 456 | 457 | The Timestamp used on each Data/Frame Packet is current time in 458 | **microseconds** -> **relative** to the [Reference Timestamp](#reference-timestamp) 459 | 460 | ### Frame Id 461 | 462 | The initial Frame Id is chosen randomly by the participant that's 463 | sending the handshake. 464 | Frame Ids of the data/frame packet will start with the initial 465 | value and increment on each packet. 466 | 467 | ### Audio 468 | 469 | > NOTE: For [ChatAudio Channel](#channels), the **client** needs to send the [Audio Server Handshake](#audio-server-handshake) 470 | > to the console after [opening](#open) the channel!!! 471 | > Console responds with [Audio Client Handshake](#audio-server-handshake) 472 | 473 | #### Audio Format 474 | 475 | | Offset (hex) | Offset (dec) | Type | Description | 476 | | -----------: | -----------: | ------ | --------------------- | 477 | | 0x00 | 0 | uint32 | Channels | 478 | | 0x04 | 4 | uint32 | Sample Rate | 479 | | 0x08 | 8 | uint32 | Audio Codec | 480 | | | | | If AudioCodec is PCM: | 481 | | 0x0C | 12 | uint32 | Bit Depth | 482 | | 0x10 | 16 | uint32 | Type | 483 | 484 | **Total size**: _variable_ 485 | 486 | - **Channels**: Audio Channels of encoded data 487 | - **Sample Rate**: Samplerate of encoded data 488 | - **Audio Codec**: See [Audio Codec](#audio-codec) 489 | - **Bit Depth (optional)**: If `Audio Codec` is `PCM` this field 490 | gives the bit depth of encoded data 491 | - **Type (optional)**: If `Audio Codec` is `PCM` this field 492 | gives the type (`Integer` or `Float`) of encoded data 493 | 494 | #### Audio Codec 495 | 496 | | Codec | Value | 497 | | ----- | ----- | 498 | | Opus | 0x00 | 499 | | AAC | 0x01 | 500 | | PCM | 0x02 | 501 | 502 | #### Audio Server Handshake 503 | 504 | Header: [Streamer Header](#streamer-header) 505 | 506 | | Offset (hex) | Offset (dec) | Type | Description | 507 | | -----------: | -----------: | ----------------- | ------------------- | 508 | | 0x00 | 0 | uint32 | Protocol Version | 509 | | 0x04 | 4 | uint64 | Reference Timestamp | 510 | | 0x0C | 12 | uint32 | Formats length | 511 | | 0x10 | 16 | AudioFormat\[len] | Audio Formats | 512 | 513 | **Total size**: _variable_ 514 | 515 | - **Protocol Version**: See [Streamer Protocol Version](#streamer-protocol-version) 516 | - **Reference Timestamp**: See [Reference Timestamp](#reference-timestamp) 517 | - **Audio Formats**: Available `Audio Formats`, Array of 518 | [Audio Format](#audio-format) 519 | 520 | #### Audio Client Handshake 521 | 522 | Header: [Streamer Header](#streamer-header) 523 | 524 | | Offset (hex) | Offset (dec) | Type | Description | 525 | | -----------: | -----------: | ----------- | ---------------- | 526 | | 0x00 | 0 | uint32 | Initial Frame Id | 527 | | 0x04 | 4 | AudioFormat | Audio Format | 528 | 529 | **Total size**: _variable_ 530 | 531 | - **Initial Frame Id**: See [Frame Id](#frame-id) 532 | - **Audio Format**: By client desired [Audio Format](#audio-format) 533 | 534 | #### Audio Control 535 | 536 | Header: [Streamer Header](#streamer-header) 537 | 538 | | Offset (hex) | Offset (dec) | Type | Description | 539 | | -----------: | -----------: | ------ | ------------------- | 540 | | 0x00 | 0 | uint32 | Audio Control Flags | 541 | 542 | **Total size**: 0x04 (4) 543 | 544 | - **Audio Control Flags**: See [Audio Control Flags](#audio-control-flags) 545 | 546 | ##### Audio Control Flags 547 | 548 | | Flag | Bits | Mask | 549 | | ------------ | --------------------- | ---- | 550 | | Reinitialize | `0000 0000 0000 0010` | 0x02 | 551 | | Start Stream | `0000 0000 0000 1000` | 0x08 | 552 | | Stop Stream | `0000 0000 0001 0000` | 0x10 | 553 | 554 | #### Audio Data 555 | 556 | Header: [Streamer Header](#streamer-header) 557 | 558 | | Offset (hex) | Offset (dec) | Type | Description | 559 | | -----------: | -----------: | ---------- | ----------- | 560 | | 0x00 | 0 | uint32 | Flags | 561 | | 0x04 | 4 | uint32 | Frame Id | 562 | | 0x08 | 8 | uint64 | Timestamp | 563 | | 0x10 | 16 | uint32 | Data length | 564 | | 0x14 | 20 | byte\[len] | Data | 565 | 566 | **Total size**: _variable_ 567 | 568 | - **Flags**: Unknown, for `AAC` it seems to always be `0x4` 569 | - **Frame Id**: See [Frame Id](#frame-id) 570 | - **Timestamp**: See [Timestamp of Data Packets](#timestamp-of-data-packets) 571 | - **Data**: Encoded data 572 | 573 | ### Video 574 | 575 | #### Lost frame reporting 576 | 577 | TODO 578 | 579 | #### Video Format 580 | 581 | | Offset (hex) | Offset (dec) | Type | Description | 582 | | -----------: | -----------: | ------ | --------------------- | 583 | | 0x00 | 0 | uint32 | FPS | 584 | | 0x04 | 4 | uint32 | Width | 585 | | 0x08 | 8 | uint32 | Height | 586 | | 0x0C | 12 | uint32 | Video Codec | 587 | | | | | If VideoCodec is RGB: | 588 | | 0x10 | 16 | uint32 | Bpp | 589 | | 0x14 | 20 | uint32 | Bytes | 590 | | 0x18 | 24 | uint64 | Red Mask | 591 | | 0x20 | 32 | uint64 | Green Mask | 592 | | 0x28 | 40 | uint64 | Blue Mask | 593 | 594 | **Total size**: _variable_ 595 | 596 | - **FPS**: Frames per second 597 | - **Width**: Video Frame Width 598 | - **Height**: Video Frame Height 599 | - **Video Codec**: See `Video Codec` 600 | - **Bpp (optional)**: If `Video Codec` is `RGB` this field gives 601 | the bits per pixel (color depth) 602 | - **Bytes (optional)**: If `Video Codec` is `RGB` this field gives 603 | the bytes per pixel 604 | - **Red Mask (optional)**: If `Video Codec` is `RGB` this field gives 605 | the `Red Mask` 606 | - **Green Mask (optional)**: If `Video Codec` is `RGB` this field gives 607 | the `Green Mask` 608 | - **Blue Mask (optional)**: If `Video Codec` is `RGB` this field gives 609 | the `Blue Mask` 610 | 611 | #### Video Codec 612 | 613 | | Codec | Value | 614 | | ----- | ----- | 615 | | H264 | 0x00 | 616 | | YUV | 0x01 | 617 | | RGB | 0x02 | 618 | 619 | #### Video Server Handshake 620 | 621 | Header: [Streamer Header](#streamer-header) 622 | 623 | | Offset (hex) | Offset (dec) | Type | Description | 624 | | -----------: | -----------: | ----------------- | ------------------- | 625 | | 0x00 | 0 | uint32 | Protocol Version | 626 | | 0x04 | 4 | uint32 | Width | 627 | | 0x08 | 8 | uint32 | Height | 628 | | 0x0C | 12 | uint32 | FPS | 629 | | 0x10 | 16 | uint64 | Reference Timestamp | 630 | | 0x18 | 24 | uint32 | Formats length | 631 | | 0x1C | 28 | VideoFormat\[len] | Video Formats | 632 | 633 | **Total size**: _variable_ 634 | 635 | - **Protocol Version**: See [Streamer Protocol Version](#streamer-protocol-version) 636 | - **Width**: Video Width 637 | - **Height**: Video Height 638 | - **FPS**: Frames per second 639 | - **Reference Timestamp**: [Reference Timestamp](#reference-timestamp) 640 | - **Video Formats**: Available `Video Formats`, Array of 641 | [Video Format](#video-format) 642 | 643 | #### Video Client Handshake 644 | 645 | Header: [Streamer Header](#streamer-header) 646 | 647 | | Offset (hex) | Offset (dec) | Type | Description | 648 | | -----------: | -----------: | ----------- | ---------------- | 649 | | 0x00 | 0 | uint32 | Initial Frame Id | 650 | | 0x04 | 4 | VideoFormat | Video Format | 651 | 652 | **Total size**: _variable_ 653 | 654 | - **Initial Frame Id**: See [Frame Id](#frame-id) 655 | - **Video Format**: By client desired [Video Format](#video-format) 656 | 657 | #### Video Control 658 | 659 | Header: [Streamer Header](#streamer-header) 660 | 661 | | Offset (hex) | Offset (dec) | Type | Description | 662 | | -----------: | -----------: | ------ | ------------------------------ | 663 | | 0x00 | 0 | uint32 | Video Control Flags | 664 | | | | | If "Last displayed frame" set: | 665 | | 0x?? | ? | uint32 | Last displayed Frame Id | 666 | | 0x?? | ? | uint64 | Timestamp | 667 | | | | | If "Queue Depth" set: | 668 | | 0x?? | ? | uint32 | Queue Depth | 669 | | | | | If "Lost frames" set: | 670 | | 0x?? | ? | uint32 | First lost Frame | 671 | | 0x?? | ? | uint32 | Last lost Frame | 672 | 673 | **Total size**: _variable_ 674 | 675 | - **Video Control Flags**: See [Video Control Flags](#video-control-flags) 676 | - **Last displayed Frame Id (optional)**: If flag `Last displayed frame` is set, this 677 | informs the host about the last displayed `Video Frame` by `Frame Id` 678 | - **Timestamp (optional)**: If flag `Last displayed frame` is set, this informs the 679 | host about the time the frame was displayed 680 | ([Timestamp of Data Packets](#timestamp-of-data-packets)) 681 | - **Queue Depth (optional)**: If flag `Queue Depth` is set: Informs the host about queue depth 682 | - **First lost Frame (optional)**: If flag `Lost frames` is set, this informs the host about 683 | the first last Frame, given the `Frame Id` 684 | - **Last lost Frame (optional)**: If flag `Lost frames` is set, this informs the host about 685 | the last last Frame, given the `Frame Id` 686 | 687 | ##### Video Control Flags 688 | 689 | | Flag | Bits | Mask | 690 | | -------------------- | --------------------- | ---- | 691 | | Request Keyframe | `0000 0000 0000 0100` | 0x04 | 692 | | Start Stream | `0000 0000 0000 1000` | 0x08 | 693 | | Stop Stream | `0000 0000 0001 0000` | 0x10 | 694 | | Queue Depth | `0000 0000 0010 0000` | 0x20 | 695 | | Lost frames | `0000 0000 0100 0000` | 0x40 | 696 | | Last displayed frame | `0000 0000 1000 0000` | 0x80 | 697 | 698 | #### Video Data 699 | 700 | Header: [Streamer Header](#streamer-header) 701 | 702 | Assembling video frames: 703 | 704 | - Frames are sequential by `Frame Id` 705 | - Each Frame aka. `Frame Id` has several chunks 706 | - Chunk size is provided in `Data length`, Position of Data chunk in Frame is provided with `Offset` 707 | - `Packet count` gives the amount of chunks in a frame 708 | - To assemble the full frame do the following: 709 | - Collect all packets for a specific `Frame Id` 710 | - Check if `Packet count` equals to collected packets/chunks 711 | - Sort the chunks by `Offset` 712 | - Concatenate the chunks `Data` section 713 | - The resulting final Frame blob should equal in size to `Total size` 714 | 715 | | Offset (hex) | Offset (dec) | Type | Description | 716 | | -----------: | -----------: | ---------- | ------------ | 717 | | 0x00 | 0 | uint32 | Flags | 718 | | 0x04 | 4 | uint32 | Frame Id | 719 | | 0x08 | 8 | uint64 | Timestamp | 720 | | 0x10 | 16 | uint32 | Total size | 721 | | 0x14 | 20 | uint32 | Packet count | 722 | | 0x18 | 24 | uint32 | Offset | 723 | | 0x1C | 28 | uint32 | Data length | 724 | | 0x20 | 32 | byte\[len] | Data | 725 | 726 | **Total size**: _variable_ 727 | 728 | - **Flags**: See [Video Data Flags](#video-data-flags) 729 | - **Frame Id**: See [Frame Id](#frame-id) 730 | - **Timestamp**: See [Timestamp of Data Packets](#timestamp-of-data-packets) 731 | - **Total size**: Total size of the Video Frame 732 | - **Packet count**: Count of total frame chunks 733 | - **Offset**: Beginning position of current frame chunk 734 | - **Data**: Encoded data 735 | 736 | ##### Video Data Flags 737 | 738 | | Flag | Bits | Mask | 739 | | -------- | --------------------- | ---- | 740 | | Keyframe | `0000 0000 0000 0010` | 0x02 | 741 | 742 | ### Input 743 | 744 | > NOTE: Input channel does not get created by default! 745 | > You need to send a [Controller Event](#controller-event) over 746 | > [Control Protocol](#control-protocol) to make the console 747 | > [Create](#create) the [Input Channel](#channels). 748 | > 749 | > After client sends the [Open Input Channel](#open) packet to the console, console 750 | > sends an [Open Input Channel](#open) packet packet too - then console 751 | > [creates](#create) and [opens](#open) the [Input Feedback Channel](#channels). 752 | > 753 | > Client [opens](#open) [Input Feedback Channel](#channels). 754 | > Client sends the [Input Server Handshake](#input-server-handshake) for 755 | > `Input Feedback` - console responds with [Input Client Handshake](#input-client-handshake). 756 | > 757 | > Console will further send a [Input Server Handshake](#input-server-handshake) 758 | > for `Input` which client needs to respond with [Input Client Handshake](#input-client-handshake) 759 | 760 | #### Input Button Model 761 | 762 | This structure starts with `all-zero`. 763 | When a button-state is changed, the appropriate field is incremented by one. 764 | 765 | **Advice**: Keep track of the button-state, don't increment the value if new button- 766 | state is the same as the old one - otherwise you will provoke a _stuck button_ 767 | in case an [Input Frame](#input-frame) packet is lost. 768 | 769 | **Example** 770 | 771 | ```C 772 | // Initial value - button is not pressed 773 | button_state.A = state.RELEASED 774 | button_model.A = 0 775 | 776 | SendPacket() 777 | 778 | while (getting_input_data) { 779 | // A button is pressed 780 | if (new_state != button_state.A) { 781 | button_model.A++ 782 | button_state.A = new_state 783 | 784 | SendPacket() 785 | } 786 | } 787 | 788 | // State can be checked from value like: 789 | // bool released = (button_model.A % 2) == 0 || button_model.A == 0 790 | ``` 791 | 792 | | Offset (hex) | Offset (dec) | Type | Description | 793 | | -----------: | -----------: | ---- | ---------------- | 794 | | 0x00 | 0 | byte | D-Pad Up | 795 | | 0x01 | 1 | byte | D-Pad Down | 796 | | 0x02 | 2 | byte | D-Pad Left | 797 | | 0x03 | 3 | byte | D-Pad Right | 798 | | 0x04 | 4 | byte | Start | 799 | | 0x05 | 5 | byte | Back | 800 | | 0x06 | 6 | byte | Left thumbstick | 801 | | 0x07 | 7 | byte | Right thumbstick | 802 | | 0x08 | 8 | byte | Left shoulder | 803 | | 0x09 | 9 | byte | Right shoulder | 804 | | 0x0A | 10 | byte | Guide | 805 | | 0x0B | 11 | byte | Unknown | 806 | | 0x0C | 12 | byte | A | 807 | | 0x0D | 13 | byte | B | 808 | | 0x0E | 14 | byte | X | 809 | | 0x0F | 15 | byte | Y | 810 | 811 | **Total size**: 0x10 (16) 812 | 813 | #### Input Analog Model 814 | 815 | | Offset (hex) | Offset (dec) | Type | Description | 816 | | -----------: | -----------: | ------ | -------------------- | 817 | | 0x00 | 0 | byte | Left Trigger | 818 | | 0x01 | 1 | byte | Right Trigger | 819 | | 0x02 | 2 | uint16 | Left thumbstick X | 820 | | 0x04 | 4 | uint16 | Left thumbstick Y | 821 | | 0x06 | 6 | uint16 | Right thumbstick X | 822 | | 0x08 | 8 | uint16 | Right thumbstick Y | 823 | | 0x0A | 10 | byte | Left Rumble trigger | 824 | | 0x0B | 11 | byte | Right Rumble trigger | 825 | | 0x0C | 12 | byte | Left Rumble handle | 826 | | 0x0D | 13 | byte | Right Rumble handle | 827 | 828 | **Total size**: 0x0E (14) 829 | 830 | #### Input Extension Model 831 | 832 | | Offset (hex) | Offset (dec) | Type | Description | 833 | | -----------: | -----------: | ---- | ---------------------- | 834 | | 0x00 | 0 | byte | Unknown 1 | 835 | | 0x01 | 1 | byte | Unknown 2 | 836 | | 0x02 | 2 | byte | Left Rumble trigger 2 | 837 | | 0x03 | 3 | byte | Right Rumble trigger 2 | 838 | | 0x04 | 4 | byte | Left Rumble handle 2 | 839 | | 0x05 | 5 | byte | Right Rumble handle 2 | 840 | | 0x06 | 6 | byte | Unknown 3 | 841 | | 0x07 | 7 | byte | Unknown 4 | 842 | | 0x08 | 8 | byte | Unknown 5 | 843 | 844 | **Total size**: 0x09 (9) 845 | 846 | - **Unknown 1**: Set to 1 for gamepad 847 | 848 | #### Input Server Handshake 849 | 850 | Header: [Streamer Header](#streamer-header) 851 | 852 | | Offset (hex) | Offset (dec) | Type | Description | 853 | | -----------: | -----------: | ------ | ---------------- | 854 | | 0x00 | 0 | uint32 | Protocol Version | 855 | | 0x04 | 4 | uint32 | Desktop Width | 856 | | 0x08 | 8 | uint32 | Desktop Height | 857 | | 0x0C | 12 | uint32 | Max Touches | 858 | | 0x10 | 16 | uint32 | Initial frame Id | 859 | 860 | **Total size**: 0x14 (20) 861 | 862 | - **Protocol Version**: See [Streamer Protocol Version](#streamer-protocol-version) 863 | - **Desktop Width**: Host's display width 864 | - **Desktop Height**: Host's display height 865 | - **Max Touches**: `0` 866 | - **Initial Frame Id**: [Frame Id](#frame-id) 867 | 868 | #### Input Client Handshake 869 | 870 | Header: [Streamer Header](#streamer-header) 871 | 872 | | Offset (hex) | Offset (dec) | Type | Description | 873 | | -----------: | -----------: | ------ | ------------------- | 874 | | 0x00 | 0 | uint32 | Max Touches | 875 | | 0x04 | 4 | uint64 | Reference Timestamp | 876 | 877 | **Total size**: 0x0C (12) 878 | 879 | - **Max Touches**: Input: `10`, Input Feedback: `0` 880 | - **Reference Timestamp**: See [Reference Timestamp](#reference-timestamp) 881 | 882 | #### Frame Ack 883 | 884 | Header: [Streamer Header](#streamer-header) 885 | 886 | | Offset (hex) | Offset (dec) | Type | Description | 887 | | -----------: | -----------: | ------ | ----------- | 888 | | 0x00 | 0 | uint32 | Acked frame | 889 | 890 | **Total size**: 0x04 (4) 891 | 892 | - **Acked frame**: `Frame Id` of [Input Frame](#input-frame) to acknowledge 893 | 894 | #### Input Frame 895 | 896 | Header: [Streamer Header](#streamer-header) 897 | 898 | | Offset (hex) | Offset (dec) | Type | Description | 899 | | -----------: | -----------: | --------- | --------------------- | 900 | | 0x00 | 0 | uint32 | Frame Id | 901 | | 0x04 | 4 | uint64 | Timestamp | 902 | | 0x0C | 12 | uint64 | Created Timestamp | 903 | | 0x14 | 20 | byte\[16] | Input Button Model | 904 | | 0x24 | 36 | byte\[14] | Input Analog Model | 905 | | | | | If remaining data: | 906 | | 0x32 | 50 | byte\[9] | Input Extension Model | 907 | 908 | **Total size**: 0x32 (50) or 0x3B (59) 909 | 910 | - **Frame Id**: See [Frame Id](#frame-id) 911 | - **Timestamp**: See [Timestamp of Data Packets](#timestamp-of-data-packets) 912 | - **Created Timestamp**: See [Timestamp of Data Packets](#timestamp-of-data-packets) 913 | - **Input Button Model**: See [Input Button Model](#input-button-model) 914 | - **Input Analog Model**: See [Input Analog Model](#input-analog-model) 915 | - **Input Extension Model (optional)**: [Input Extension Model](#input-extension-model) 916 | 917 | ### Control Protocol 918 | 919 | Header: [Streamer Header](#streamer-header) 920 | Control Protocol packets have `Streamer Payload Type` set to `0` 921 | 922 | #### Control Header 923 | 924 | | Offset (hex) | Offset (dec) | Type | Description | 925 | | -----------: | -----------: | ------- | ------------------------ | 926 | | 0x00 | 0 | uint32 | Previous Sequence (DUPE) | 927 | | 0x04 | 4 | uint16 | Unknown 1 | 928 | | 0x06 | 6 | uint16 | Unknown 2 | 929 | | 0x08 | 8 | uint16 | Control Payload Type | 930 | | 0x0A | 10 | byte\[] | Control Payload | 931 | 932 | **Total size**: _variable_ 933 | 934 | - **Previous Sequence**: Previous Sequence Number (DUPE) 935 | - **Unknown 1**: TODO: `1` 936 | - **Unknown 2**: TODO: `1406` 937 | - **Control Payload Type**: See [Control Payload Type](#control-payload-type) 938 | - **Control Payload**: Depends on [Control Payload Type](#control-payload-type) 939 | 940 | #### Control Payload Type 941 | 942 | | Type | Value | 943 | | ----------------------- | ----- | 944 | | Session Init | 0x01 | 945 | | Session Create | 0x02 | 946 | | Session Create Response | 0x03 | 947 | | Session Destroy | 0x04 | 948 | | Video Statistics | 0x05 | 949 | | Realtime Telemetry | 0x06 | 950 | | Change Video Quality | 0x07 | 951 | | Initiate Network Test | 0x08 | 952 | | Network Information | 0x09 | 953 | | Network Test Response | 0x0A | 954 | | Controller Event | 0x0B | 955 | 956 | #### Session Init 957 | 958 | | Offset (hex) | Offset (dec) | Type | Description | 959 | | -----------: | -----------: | ------- | ----------- | 960 | | 0x00 | 0 | byte\[] | Unknown | 961 | 962 | #### Session Create 963 | 964 | | Offset (hex) | Offset (dec) | Type | Description | 965 | | -----------: | -----------: | ------------- | ----------- | 966 | | 0x00 | 0 | uint32 | Length | 967 | | 0x04 | 4 | byte\[length] | Unknown | 968 | 969 | #### Session Create Response 970 | 971 | | Offset (hex) | Offset (dec) | Type | Description | 972 | | -----------: | -----------: | ------- | ----------- | 973 | | 0x00 | 0 | byte\[] | Unknown | 974 | 975 | #### Session Destroy 976 | 977 | | Offset (hex) | Offset (dec) | Type | Description | 978 | | -----------: | -----------: | ------ | ----------- | 979 | | 0x00 | 0 | uint32 | Unknown 1 | 980 | | 0x04 | 4 | uint32 | Length | 981 | | 0x08 | 8 | uint32 | Unknown 2 | 982 | 983 | #### Video Statistics 984 | 985 | | Offset (hex) | Offset (dec) | Type | Description | 986 | | -----------: | -----------: | ------ | ----------- | 987 | | 0x00 | 0 | uint32 | Unknown 1 | 988 | | 0x04 | 4 | uint32 | Unknown 2 | 989 | | 0x08 | 8 | uint32 | Unknown 3 | 990 | | 0x0C | 12 | uint32 | Unknown 4 | 991 | | 0x10 | 16 | uint32 | Unknown 5 | 992 | | 0x14 | 20 | uint32 | Unknown 6 | 993 | 994 | #### Realtime Telemetry 995 | 996 | | Offset (hex) | Offset (dec) | Type | Description | 997 | | -----------: | -----------: | ---------------------- | ---------------- | 998 | | 0x00 | 0 | uint16 | Field count | 999 | | 0x02 | 2 | TelemetryField\[count] | Telemetry Fields | 1000 | 1001 | ##### Telemetry Field 1002 | 1003 | | Offset (hex) | Offset (dec) | Type | Description | 1004 | | -----------: | -----------: | ------ | ----------- | 1005 | | 0x00 | 0 | uint16 | Key | 1006 | | 0x02 | 2 | uint64 | Value | 1007 | 1008 | #### Change Video Quality 1009 | 1010 | | Offset (hex) | Offset (dec) | Type | Description | 1011 | | -----------: | -----------: | ------ | ----------- | 1012 | | 0x00 | 0 | uint32 | Unknown 1 | 1013 | | 0x04 | 4 | uint32 | Unknown 2 | 1014 | | 0x08 | 8 | uint32 | Unknown 3 | 1015 | | 0x0C | 12 | uint32 | Unknown 4 | 1016 | | 0x10 | 16 | uint32 | Unknown 5 | 1017 | | 0x14 | 20 | uint32 | Unknown 6 | 1018 | 1019 | #### Initiate Network Test 1020 | 1021 | | Offset (hex) | Offset (dec) | Type | Description | 1022 | | -----------: | -----------: | ------- | ----------- | 1023 | | 0x00 | 0 | byte\[] | Unknown | 1024 | 1025 | #### Network Information 1026 | 1027 | | Offset (hex) | Offset (dec) | Type | Description | 1028 | | -----------: | -----------: | ------ | ----------- | 1029 | | 0x00 | 0 | uint64 | Unknown 1 | 1030 | | 0x08 | 8 | byte | Unknown 2 | 1031 | | 0x09 | 9 | uint32 | Unknown 3 | 1032 | 1033 | #### Network Test Response 1034 | 1035 | | Offset (hex) | Offset (dec) | Type | Description | 1036 | | -----------: | -----------: | ------ | ----------- | 1037 | | 0x00 | 0 | uint32 | Unknown 1 | 1038 | | 0x04 | 4 | uint32 | Unknown 2 | 1039 | | 0x08 | 8 | uint32 | Unknown 3 | 1040 | | 0x0C | 12 | uint32 | Unknown 4 | 1041 | | 0x10 | 16 | uint32 | Unknown 5 | 1042 | | 0x14 | 20 | uint64 | Unknown 6 | 1043 | | 0x1C | 28 | uint64 | Unknown 7 | 1044 | | 0x24 | 36 | uint32 | Unknown 8 | 1045 | 1046 | #### Controller Event 1047 | 1048 | By sending a `Controller Event` packet to the console, the console 1049 | will open the `Input`/`Input Feedback` channels and send the 1050 | appropriate `Channel Open` packets to the client. 1051 | 1052 | | Offset (hex) | Offset (dec) | Type | Description | 1053 | | -----------: | -----------: | ---- | ----------------- | 1054 | | 0x00 | 0 | byte | Event | 1055 | | 0x01 | 1 | byte | Controller Number | 1056 | 1057 | - **Event**: [Controller Event Type](#controller-event-type) 1058 | - **Controller Number**: Null-indexed controller number 1059 | 1060 | ##### Controller Event Type 1061 | 1062 | | Event | Value | 1063 | | ------- | ----- | 1064 | | Removed | 0x00 | 1065 | | Added | 0x01 | 1066 | -------------------------------------------------------------------------------- /docs/rtpheader.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenXbox/smartglass-documentation/c6a3e523e3fc6037835d44b37b64bf3fa2d21470/docs/rtpheader.png -------------------------------------------------------------------------------- /docs/simple_message.md: -------------------------------------------------------------------------------- 1 | # Simple Message 2 | 3 | - [Simple Message](#simple-message) 4 | - [General](#general) 5 | - [Header](#header) 6 | - [Packet Type](#packet-type) 7 | - [Power On Request](#power-on-request) 8 | - [Discovery Packet](#discovery-packet) 9 | - [Discovery Request](#discovery-request) 10 | - [Client Type](#client-type) 11 | - [Discovery Response](#discovery-response) 12 | - [Primary Device Flags](#primary-device-flags) 13 | - [Certificate](#certificate) 14 | - [Connect Packet](#connect-packet) 15 | - [Connect Request](#connect-request) 16 | - [Public Key Type](#public-key-type) 17 | - [Fragmenting Request](#fragmenting-request) 18 | - [Request Group explaination](#request-group-explaination) 19 | - [Connect Response](#connect-response) 20 | - [Connect Result](#connect-result) 21 | 22 | ## General 23 | 24 | Simple Messages are used for the most basic tasks in SmartGlass 25 | 26 | - Discovering consoles on the network 27 | - Waking up consoles (Power On) 28 | - Connecting to consoles 29 | 30 | ## Header 31 | 32 | | Offset (hex) | Offset (dec) | Type | Description | 33 | | -----------: | -----------: | ------ | -------------------------- | 34 | | 0x00 | 0 | uint16 | Packet Type | 35 | | 0x02 | 2 | uint16 | Unprotected Payload Length | 36 | | 0x04 | 4 | uint16 | \*Protected Payload Length | 37 | | 0x06 or 0x08 | 6 or 8 | uint16 | Version | 38 | 39 | > NOTE: The header only contains the protected 40 | > payload length-field if packet actually has such 41 | > payload: 42 | > 43 | > SimpleMessage containing protected payload: 44 | > 45 | > - [Connect Request](#connect-request) 46 | > - [Connect Response](#connect-response) 47 | 48 | - **Packet Type**: See [Packet Type](#packet-type) 49 | - **Version**: Usually 2, [Discovery](#discovery-packet) has set this to 0 50 | 51 | ### Packet Type 52 | 53 | | Value | Description | 54 | | :----- | ----------------------------------------- | 55 | | 0xCC00 | [Connect Request](#connect-request) | 56 | | 0xCC01 | [Connect Response](#connect-response) | 57 | | 0xDD00 | [Discovery Request](#discovery-request) | 58 | | 0xDD01 | [Discovery Response](#discovery-response) | 59 | | 0xDD02 |  [Power On Request](#power-on-request) | 60 | 61 | > [Message](message.md) packets (type: `0xD00D`) have a different 62 | > header and are not considered `SimpleMessage` 63 | 64 | ## Power On Request 65 | 66 | **Packet Type**: `0xDD02` 67 | **Response**: None 68 | 69 | This packet is sent over broadcast / multicast and directly 70 | to the console's IP address. 71 | 72 | - Broadcast (255.255.255.255) 73 | - Multicast (239.255.255.250) 74 | - Specific IP of the console (if available) 75 | 76 | If the console is in standby-mode, it will process that 77 | packet and power on. 78 | 79 | > NOTE: You may need to send the packet multiple times to make 80 | > sure it is actually received by the console. 81 | > 82 | > Power On request packets only have an `Unprotected payload` 83 | 84 | | Offset (hex) | Offset (dec) | Type | Description | 85 | | -----------: | -----------: | -------- | ----------- | 86 | | 0x00 | 0 | SGString | Live ID | 87 | 88 | ## Discovery Packet 89 | 90 | Discovery packet, as the name suggests, is used for finding 91 | active consoles on the network. 92 | 93 | Same as [Power On Request](#power-on-request), the packet is sent to 3 remote endpoints: 94 | 95 | - Broadcast (255.255.255.255) 96 | - Multicast (239.255.255.250) 97 | - Specific IP of the console (if available) 98 | 99 | The console will respond by sending a [Discovery Response](#discovery-response) to the sending IP. 100 | 101 | > Discovery packets set Header->Version: 0 102 | > 103 | > Discovery packets only have an `Unprotected payload` 104 | 105 | ### Discovery Request 106 | 107 | **Packet Type**: `0xDD00` 108 | **Response**: [Discovery Response](#discovery-response) 109 | 110 | Discovers active consoles on the network 111 | 112 | | Offset (hex) | Offset (dec) | Type | Description | 113 | | -----------: | -----------: | ------ | --------------- | 114 | | 0x00 | 0 | uint32 | Flags | 115 | | 0x04 | 4 | uint16 | Client Type | 116 | | 0x06 | 6 | uint16 | Minimum Version | 117 | | 0x08 | 8 | uint16 | Maximum Version | 118 | 119 | - **Flags**: Unknown, always 0 120 | - **Client Type**: See [Client Type](#client-type) 121 | - **Minimum Version**: Minimum version to discover: 0 122 | - **Maximum Version**: Maximum version to discover: 2 123 | 124 | #### Client Type 125 | 126 | | Type | Value | 127 | | --------------- | ----- | 128 | | Xbox One | 0x01 | 129 | | Xbox 360 | 0x02 | 130 | | Windows Desktop | 0x03 | 131 | | Windows Store | 0x04 | 132 | | Windows Phone | 0x05 | 133 | | iPhone | 0x06 | 134 | | iPad | 0x07 | 135 | | Android | 0x08 | 136 | 137 | ### Discovery Response 138 | 139 | **Packet Type**: `0xDD01` 140 | **Response**: None 141 | 142 | Console info for the client 143 | 144 | | Offset (hex) | Offset (dec) | Type | Description | 145 | | -----------: | -----------: | ---------- | -------------------- | 146 | | 0x00 | 0 | uint32 | Primary Device Flags | 147 | | 0x04 | 4 | uint16 | Type | 148 | | 0x08 | 8 | SGString | ConsoleName | 149 | | 0x?? | ? | SGString | UUID | 150 | | 0x?? | ? | uint32 | Last Error | 151 | | 0x?? | ? | uint16 | Certificate Length | 152 | | 0x?? | ? | char\[len] | Certificate | 153 | 154 | - **Primary Device Flags**: See [Primary Device Flags](#primary-device-flags) 155 | - **Type**: See [Client Type](#client-type) 156 | - **ConsoleName**: Console Name 157 | - **UUID**: Console UUID aka Hardware Id 158 | - **Last Error**: Last error 159 | - **Certificate**: See [Certificate](#certificate) 160 | 161 | #### Primary Device Flags 162 | 163 | | Flag | Bits | Mask | 164 | | ------------------------- | --------------------- | ---- | 165 | | Allow Console Users | `0000 0000 0000 0001` | 0x01 | 166 | | Allow Authenticated Users | `0000 0000 0000 0010` | 0x02 | 167 | | Allow Anonymous Users | `0000 0000 0000 0100` | 0x04 | 168 | | Certificate Pending | `0000 0000 0000 1000` | 0x08 | 169 | 170 | #### Certificate 171 | 172 | Certificate is X509 ASN.1 / DER formatted. 173 | 174 | **Fields**: 175 | 176 | > - **Version**: 3 177 | > - **Serial Number**: Console serial number 178 | > - **Issuer**: `CN=XBL Smart Glass Issuing CA` 179 | > - **Validity**: Validity timespan 180 | > - **Subject**: Console's LiveId (`CN=`) 181 | > - **Subject Public Key Info**: Console's public key 182 | > 183 | > TIP: You can get a human-readable output for the cert 184 | > via openssl: 185 | > 186 | > `openssl x509 -inform der -in -text -noout` 187 | 188 | ## Connect Packet 189 | 190 | ### Connect Request 191 | 192 | **Packet Type**: `0xCC00` 193 | **Response**: [Connect Response](#connect-response) 194 | 195 | The request is sent from client to console. 196 | 197 | Based on the previously received [Discovery Response](#discovery-response), a shared secret is 198 | calculated and the request is encrypted using that (See [Cryptography](cryptography.md)). 199 | 200 | ##### Public Key Type 201 | 202 | Public Key identifier, sent with [Connect Request](#connect-request) packet. 203 | 204 | | Type | Value | Size | 205 | | ---------- | ----- | ---- | 206 | | EC DH P256 | 0x00 | 65 | 207 | | EC DH P384 | 0x01 | 97 | 208 | | EC DH P521 | 0x02 | 133 | 209 | 210 | #### Fragmenting Request 211 | 212 | When connecting authenticated, the `Authentication data` 213 | (userhash, authorization token) won't fit inside a single packet, 214 | which means the packet will need to be fragmented. 215 | 216 | 1. Calculate payload length (plaintext unprotected + plaintext 217 | protected) of assembled [Connect Request](#connect-request) using 218 | empty `Userhash`("") and `Auth Token`(""). 219 | 2. Calculate available space by `available space = (1024 - payload length)` 220 | 3. Assemble first fragment by copying whole `Userhash` and filling up 221 | the rest of available space with `Auth Token`. 222 | 4. Assemble rest of fragments by just filling them with `Auth Token` chunks. 223 | 224 | > NOTE: Make sure to use correct Request (Group) numbering, see 225 | > [Request Group explaination](#request-group-explaination) 226 | 227 | **Unprotected Payload** 228 | 229 | | Offset (hex) | Offset (dec) | Type | Description | 230 | | -----------: | -----------: | --------- | --------------- | 231 | | 0x00 | 0 | byte\[16] | SmartGlass UUID | 232 | | 0x10 | 16 | uint16 | Public Key Type | 233 | | 0x12 | 18 | byte\[??] | Public Key | 234 | | 0x52 | 82 | byte\[16] | IV | 235 | 236 | - **SmartGlass UUID**: Random UUID generated by the client 237 | - **Public Key Type**: See [Public Key Type](#public-key-type) 238 | - **Public Key**: Public Key, size depends on [Public Key Type](#public-key-type) 239 | - **IV**: Initialization Vector of the encrypted packet 240 | 241 | **Protected Payload** 242 | 243 | > Because strings are of variable length it's not 244 | > possible to give absolute offsets here 245 | 246 | | Offset (hex) | Offset (dec) | Type | Description | 247 | | -----------: | -----------: | -------- | ------------------- | 248 | | 0x0 | 0 | SGString | Userhash | 249 | | 0x? | ? | SGString | Auth Token | 250 | | 0x? | ? | uint32 | Request Number | 251 | | 0x? | ? | uint32 | Request Group Start | 252 | | 0x? | ? | uint32 | Request Group End | 253 | 254 | - **Userhash**: Xbox Live Userhash (`uhs`) 255 | - **Auth Token**: Xbox Live Auth token (`XSTS`) 256 | - **Request Number**: Current request number 257 | - **Request Group Start**: First request number of this group 258 | - **Request Group End**: Last request number of that group **+ 1** 259 | 260 | #### Request Group explaination 261 | 262 | > **Example**: If you just have a single connect fragment, assume 263 | > request number 0: 264 | > 265 | > Request Number: 0, Request Group Start: 0, Request Group End: 1 266 | > 267 | > Next connect attempt, if needed, would be: 268 | > 269 | > Request Number: 1, Request Group Start: 1, Request Group End: 2 270 | > 271 | > **Example 2**: You have 3 connect fragments, starting again with request number 0: 272 | > 273 | > Fragment #1: Request Number: 0, Request Group Start: 0, Request Group End: 3 274 | > 275 | > Fragment #2: Request Number: 1, Request Group Start: 0, Request Group End: 3 276 | > 277 | > Fragment #3: Request Number: 2, Request Group Start: 0, Request Group End: 3 278 | 279 | ### Connect Response 280 | 281 | **Packet Type**: `0xCC01` 282 | **Response**: None 283 | 284 | If the request was formatted and encrypted properly, the console responds 285 | with the following packet: 286 | 287 | **Unprotected Payload** 288 | 289 | | Offset (hex) | Offset (dec) | Type | Description | 290 | | -----------: | -----------: | --------- | ----------- | 291 | | 0x00 | 0 | byte\[16] | IV | 292 | 293 | - **IV**: Initialization Vector of the encrypted 294 | packet 295 | 296 | **Protected Payload** 297 | 298 | | Offset (hex) | Offset (dec) | Type | Description | 299 | | -----------: | -----------: | ------ | -------------- | 300 | | 0x0 | 0 | uint16 | Connect Result | 301 | | 0x2 | 2 | uint16 | Pairing State | 302 | | 0x4 | 4 | uint32 | Participant ID | 303 | 304 | - **Connect Result**: Indicates status of the connect 305 | attempt, see [Connect Result](#connect-result) 306 | - **Pairing State**: Pairing state of the client 307 | - **Participant Id**: Auto-incremented index given to 308 | the client by the console. Received [Messages](message.md) in the 309 | active session have to match that Id in their `Target Participant Id` 310 | field. 311 | 312 | #### Connect Result 313 | 314 | | Result | Value | 315 | | ------- | ----- | 316 | | Success | 0x00 | 317 | | Pending | 0x01 | 318 | 319 | | Error | Value | 320 | | ----------------------------- | ----- | 321 | | Unknown | 0x02 | 322 | | Anonymous Connection Disabled | 0x03 | 323 | | Device Limit Exceeded | 0x04 | 324 | | SmartGlass disabled | 0x05 | 325 | | User Auth failed | 0x06 | 326 | | User SignIn failed | 0x07 | 327 | | User SignIn timeout | 0x08 | 328 | | User SignIn required | 0x09 | 329 | -------------------------------------------------------------------------------- /mkdocs.yml: -------------------------------------------------------------------------------- 1 | site_name: Xbox SmartGlass protocol 2 | nav: 3 | - Home: index.md 4 | - Basics: basics.md 5 | - Cryptography: cryptography.md 6 | - Simple Message: simple_message.md 7 | - Channels: channels.md 8 | - Messages: message.md 9 | - Gamestream (Nano): nano.md 10 | theme: material 11 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | mkdocs-material 2 | --------------------------------------------------------------------------------