├── LICENSE ├── Makefile ├── README.md ├── docs ├── draft-ftl-specification-00.html ├── draft-ftl-specification-00.pdf ├── draft-ftl-specification-00.txt └── index.html └── draft-ftl-specification-00.xml /LICENSE: -------------------------------------------------------------------------------- 1 | Creative Commons Legal Code 2 | 3 | CC0 1.0 Universal 4 | 5 | CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE 6 | LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN 7 | ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS 8 | INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES 9 | REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS 10 | PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM 11 | THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED 12 | HEREUNDER. 13 | 14 | Statement of Purpose 15 | 16 | The laws of most jurisdictions throughout the world automatically confer 17 | exclusive Copyright and Related Rights (defined below) upon the creator 18 | and subsequent owner(s) (each and all, an "owner") of an original work of 19 | authorship and/or a database (each, a "Work"). 20 | 21 | Certain owners wish to permanently relinquish those rights to a Work for 22 | the purpose of contributing to a commons of creative, cultural and 23 | scientific works ("Commons") that the public can reliably and without fear 24 | of later claims of infringement build upon, modify, incorporate in other 25 | works, reuse and redistribute as freely as possible in any form whatsoever 26 | and for any purposes, including without limitation commercial purposes. 27 | These owners may contribute to the Commons to promote the ideal of a free 28 | culture and the further production of creative, cultural and scientific 29 | works, or to gain reputation or greater distribution for their Work in 30 | part through the use and efforts of others. 31 | 32 | For these and/or other purposes and motivations, and without any 33 | expectation of additional consideration or compensation, the person 34 | associating CC0 with a Work (the "Affirmer"), to the extent that he or she 35 | is an owner of Copyright and Related Rights in the Work, voluntarily 36 | elects to apply CC0 to the Work and publicly distribute the Work under its 37 | terms, with knowledge of his or her Copyright and Related Rights in the 38 | Work and the meaning and intended legal effect of CC0 on those rights. 39 | 40 | 1. Copyright and Related Rights. A Work made available under CC0 may be 41 | protected by copyright and related or neighboring rights ("Copyright and 42 | Related Rights"). Copyright and Related Rights include, but are not 43 | limited to, the following: 44 | 45 | i. the right to reproduce, adapt, distribute, perform, display, 46 | communicate, and translate a Work; 47 | ii. moral rights retained by the original author(s) and/or performer(s); 48 | iii. publicity and privacy rights pertaining to a person's image or 49 | likeness depicted in a Work; 50 | iv. rights protecting against unfair competition in regards to a Work, 51 | subject to the limitations in paragraph 4(a), below; 52 | v. rights protecting the extraction, dissemination, use and reuse of data 53 | in a Work; 54 | vi. database rights (such as those arising under Directive 96/9/EC of the 55 | European Parliament and of the Council of 11 March 1996 on the legal 56 | protection of databases, and under any national implementation 57 | thereof, including any amended or successor version of such 58 | directive); and 59 | vii. other similar, equivalent or corresponding rights throughout the 60 | world based on applicable law or treaty, and any national 61 | implementations thereof. 62 | 63 | 2. Waiver. To the greatest extent permitted by, but not in contravention 64 | of, applicable law, Affirmer hereby overtly, fully, permanently, 65 | irrevocably and unconditionally waives, abandons, and surrenders all of 66 | Affirmer's Copyright and Related Rights and associated claims and causes 67 | of action, whether now known or unknown (including existing as well as 68 | future claims and causes of action), in the Work (i) in all territories 69 | worldwide, (ii) for the maximum duration provided by applicable law or 70 | treaty (including future time extensions), (iii) in any current or future 71 | medium and for any number of copies, and (iv) for any purpose whatsoever, 72 | including without limitation commercial, advertising or promotional 73 | purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each 74 | member of the public at large and to the detriment of Affirmer's heirs and 75 | successors, fully intending that such Waiver shall not be subject to 76 | revocation, rescission, cancellation, termination, or any other legal or 77 | equitable action to disrupt the quiet enjoyment of the Work by the public 78 | as contemplated by Affirmer's express Statement of Purpose. 79 | 80 | 3. Public License Fallback. Should any part of the Waiver for any reason 81 | be judged legally invalid or ineffective under applicable law, then the 82 | Waiver shall be preserved to the maximum extent permitted taking into 83 | account Affirmer's express Statement of Purpose. In addition, to the 84 | extent the Waiver is so judged Affirmer hereby grants to each affected 85 | person a royalty-free, non transferable, non sublicensable, non exclusive, 86 | irrevocable and unconditional license to exercise Affirmer's Copyright and 87 | Related Rights in the Work (i) in all territories worldwide, (ii) for the 88 | maximum duration provided by applicable law or treaty (including future 89 | time extensions), (iii) in any current or future medium and for any number 90 | of copies, and (iv) for any purpose whatsoever, including without 91 | limitation commercial, advertising or promotional purposes (the 92 | "License"). The License shall be deemed effective as of the date CC0 was 93 | applied by Affirmer to the Work. Should any part of the License for any 94 | reason be judged legally invalid or ineffective under applicable law, such 95 | partial invalidity or ineffectiveness shall not invalidate the remainder 96 | of the License, and in such case Affirmer hereby affirms that he or she 97 | will not (i) exercise any of his or her remaining Copyright and Related 98 | Rights in the Work or (ii) assert any associated claims and causes of 99 | action with respect to the Work, in either case contrary to Affirmer's 100 | express Statement of Purpose. 101 | 102 | 4. Limitations and Disclaimers. 103 | 104 | a. No trademark or patent rights held by Affirmer are waived, abandoned, 105 | surrendered, licensed or otherwise affected by this document. 106 | b. Affirmer offers the Work as-is and makes no representations or 107 | warranties of any kind concerning the Work, express, implied, 108 | statutory or otherwise, including without limitation warranties of 109 | title, merchantability, fitness for a particular purpose, non 110 | infringement, or the absence of latent or other defects, accuracy, or 111 | the present or absence of errors, whether or not discoverable, all to 112 | the greatest extent permissible under applicable law. 113 | c. Affirmer disclaims responsibility for clearing rights of other persons 114 | that may apply to the Work or any use thereof, including without 115 | limitation any person's Copyright and Related Rights in the Work. 116 | Further, Affirmer disclaims responsibility for obtaining any necessary 117 | consents, permissions or other rights required for any use of the 118 | Work. 119 | d. Affirmer understands and acknowledges that Creative Commons is not a 120 | party to this document and has no duty or obligation with respect to 121 | this CC0 or use of the Work. 122 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: mkdir all 2 | 3 | all: mkdir docs/index.html docs/draft-ftl-specification-00.html docs/draft-ftl-specification-00.txt docs/draft-ftl-specification-00.pdf 4 | 5 | mkdir: 6 | @-mkdir docs 2> /dev/null 7 | 8 | docs/draft-ftl-specification-00.html: draft-ftl-specification-00.xml 9 | xml2rfc --html draft-ftl-specification-00.xml -p docs 10 | 11 | docs/draft-ftl-specification-00.pdf: draft-ftl-specification-00.xml 12 | xml2rfc --pdf draft-ftl-specification-00.xml -p docs 13 | 14 | docs/draft-ftl-specification-00.txt: draft-ftl-specification-00.xml 15 | xml2rfc -P draft-ftl-specification-00.xml -p docs 16 | 17 | docs/index.html: 18 | ln -s draft-ftl-specification-00.html docs/index.html 19 | 20 | clean: 21 | rm -r docs 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ftl-specification 2 | The specification for the FTL streaming protocol 3 | 4 | A rendered version of the specification can be found here: https://ftl-protocol-dev.github.io/ftl-specification/ 5 | -------------------------------------------------------------------------------- /docs/draft-ftl-specification-00.html: -------------------------------------------------------------------------------- 1 | 3 | 4 | 5 | 6 | 7 | 8 | Faster-Than-Light Streaming Protocl Specification 9 | 10 | 375 | 376 | 377 | 378 | 379 | 380 | 381 | 382 | 383 | 384 | 385 | 386 | 387 | 388 | 389 | 390 | 391 | 392 | 393 | 394 | 395 | 396 | 397 | 398 | 399 | 400 | 401 | 402 | 403 | 404 | 405 | 406 | 407 | 408 | 409 | 410 | 411 | 412 | 413 | 414 | 415 | 416 | 417 | 418 | 419 | 420 | 421 | 422 | 423 | 424 | 425 | 426 | 427 | 428 | 429 | 430 | 431 | 432 | 433 | 434 | 435 | 436 | 437 | 438 | 439 | 440 | 441 | 442 | 443 | 444 |
Faster-Than-Light Standardization EffortM. Casadevall, Ed.
Internet-DraftIndepedent
Intended status: InformationalJanuary 5, 2021
Expires: July 9, 2021
445 | 446 |

Faster-Than-Light Streaming Protocl Specification
447 | draft-ftl-specification-00

448 | 449 |

Abstract

450 |

With the demise of Microsoft's Mixer, the future of the Faster-Than-Light (FTL) streaming protocol has been left in doubt. As the Internet's first practical subsecond streaming protocol, several successors to Mixer have decided to re-implement FTL from the original SDK and notes. While Mixer's original FTL specification had a de-facto specification in the form of ftl-sdk, the source code was in-complete, and several aspects of the FTL were left undocumented.

451 |

In an effort to keep FTL viable and cross-service compatible, this specification denotes a canonical implementation of FTL, handshake protocols, WebRTC notes, and all relevant information as relating to FTL with the hope that FTL may still be continued as a vechile for low latency video streaming over the Internet.

452 |

Status of This Memo

453 |

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

454 |

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

455 |

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

456 |

This Internet-Draft will expire on July 9, 2021.

457 |

Copyright Notice

458 |

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.

459 |

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document.

460 | 461 | 462 |
463 |

Table of Contents

464 | 529 | 530 |

531 | 1. Introduction and Overview

532 |

This specification covers several components of the FTL protocol stream, and is primarily derieved from the implementation used at Mixer and the freely available source code in ftl-sdk. This document details the handshaking protocol known as Charon, the SRTP ingest behaviors, and defines a recommended ingest->endpoint streaming protocol, as well as notes in regards to implementation of the last mile WebRTC connections.

533 |

FTL was specifically designed with the following objectives in mind which is must handle at all times.

534 |

535 | 536 | 545 | 546 |

547 |

As originally designed, the following criticera were also kept in mind, although not realized at least during the initial implementation phases.

548 |

549 | 550 | 552 | 553 |

554 |

555 | 2. Design Tradeoffs/Limitations

556 |

TBD

557 |

558 | 3. One-to-Many WebRTC

559 |

TBD

560 |

561 | 4. A Note on SSRCs

562 |

TBD

563 |

564 | 5. Charon Negotiation Protocol

565 |

Charon handles negotiation of RTP streams to Styx, and acts as an out of band signaling method for FTL stream behavior. The Charon connection MUST be kept-alive at all times as a TCP/IP connection on port 8084 (MC: should apply for IANA number). Upon establishment of a CONNECT, a client must send PING messages once every five seconds, or the server MAY time-out the connection.

566 |

It is RECOMMENDED that the server wait 10 seconds before assuming that a client has hung and hanging up. PING messages must be responded to with a 201 respond code. The server should send 5XX Timeout message before disconnecting. (Ed: fix this and actually put the client behavior)

567 |

If the TCP/IP connection is reset, the broadcaster MUST assume that the ingest point of presence has become unavailable, and begin clean-up and teardown. Likewise, the ingest daemon MUST begin teardown and disregard any UDP traffic from the streamer upon Charon connection loss.

568 |

The Charon protocol is built upon ASCII verbs with optional arguments. The lifecycle of this connection under normal circumstances is as follows.

569 |

570 | 571 | 582 | 583 |

584 |

585 | 5.1. Message Format

586 |

Charon is directly modeled on SMTP commands and response codes. As such, commands take the form of a verb, followed by a three digital response. An example exchange looks as such:

587 | 
588 |             Client: PING\r\n\r\n
589 |             Server: 201 Ping OK!\r\n\r\n
590 |
591 |

For legacy reasons, linebreaks in Charon are encoded as '\r\n\r\n' (hex 0x0D0A0D0A), with an unusually complex implementation detail. See the section "On Linebreaks" for more details.

592 |

For commands that do not need additional meta-data, the server SHALL process them immediately upon receiving a linebreak, otherwise, a multiline format using RFC822 headers is defined, with the message ended with a single period. Unknown headers MUST be disregarded by the server. This message exchange looks as follows:

593 | 
594 |                 Client: CONNECT 1234-myhashhere\r\n\r\n
595 |                      C: Video: true\r\n\r\n
596 |                      C: Audio: true\r\n\r\n
597 |                      C: .\r\n\r\n
598 |                 Server: 200 Send UDP!\r\n\r\n
599 |
600 |

The Charon protocol does not allow for transmission of binary data without encoding. If necessary, it is RECOMMENDED that binary data base encoded in Base64.

601 |

602 | 5.1.1. On Linebreaks

603 |

The reference implementation of FTL uses the message signature '\r\n\r\n' as an end of line marker. This is an intentional implementation detail due to an unintentional similarity between FTL's CONNECT message, and the HTTP CONNECT proxy command, and Microsoft discovered that some commercial firewalls would intercept Charon's messages. The original justification was left as this comment in ftl_helpers.c

604 | 
605 | /*
606 |   Please note that throughout the code, we send "\r\n\r\n", where a 
607 |   normal newline ("\n") would suffice. This is done due to some 
608 |   firewalls /  anti-malware systems not passing our packets 
609 |   through when we don't send those double-windows-newlines. They
610 |   seem to incorrectly detect our protocol as HTTP.
611 | */
612 |
613 |

A problem however arises that earlier implementations of FTL used "\n" as a newline indicator. While it is unlikely that any of these legacy FTL clients are still in use, clients and ingests must use the following behaviors

614 |

Charon servers MUST disregard any empty newline, and treat "\r\n" and "\n" as identical for the purposes of message parsing. In effect, all the following are the same.

615 | 
616 | CONNECT 1234-hash\n
617 | Option1: 1234\n
618 | Option2: 1234\n
619 | .\n
620 | 
621 | CONNECT 1234-hash\r\n
622 | Option1: 1234\r\n
623 | Option2: 1234\r\n
624 | .\r\n
625 | 
626 | CONNECT 1234-hash\r\n
627 | Option1: 1234\r\n\r\n
628 | Option2: 1234\r\n\r\n
629 | .\r\n\
630 |
631 |

Charon clients are MUST to use '\r\n\r\n' when transmitting commands over clear text.

632 |

633 | 5.2. Response Codes

634 |

635 | 5.3. Charon Verbs

636 |

637 | 5.3.1. HMAC

638 |

TDB

639 |

640 | 5.3.2. CONNECT

641 |

TDB

642 |

643 | 5.3.2.1. ProtocolVersion

644 |

TBD

645 |

646 | 5.3.2.2. VendorName

647 |

TBD

648 |

649 | 5.3.2.3. VendorVersion

650 |

TBD

651 |

652 | 5.3.2.4. Video

653 |

TBD

654 |

655 | 5.3.2.5. VideoCodex

656 |

TBD

657 |

658 | 5.3.2.6. VideoHeight

659 |

TBD

660 |

661 | 5.3.2.7. VideoWidth

662 |

TBD

663 |

664 | 5.3.2.8. VideoPayloadType

665 |

TBD

666 |

667 | 5.3.2.9. VideoIngestSSRC

668 |

TBD

669 |

670 | 5.3.2.10. Audio

671 |

TBD

672 |

673 | 5.3.2.11. AudioCodec

674 |

TBD

675 |

676 | 5.3.2.12. AudioPayloadType

677 |

TBD

678 |

679 | 5.3.2.13. AudioIngestSSRC

680 |

TBD

681 |

682 | 5.3.3. DISCONNECT

683 |

TBD

684 |

685 | 5.3.4. PING

686 |

TBD

687 |

688 | 6. Styx Protocol Ingest Behavior

689 |

TBD

690 |

691 | 7. Babel Transcoding Behavior

692 |

TBD

693 |

694 | 8. WebRTC Last Mile Negotations

695 |

TBD

696 |

Author's Address

697 |
698 |
699 | 700 | Michael Casadevall (editor) 701 | 704 | 705 | Indepedent 706 | 707 | 708 | 709 | Jersey City, 710 | 711 | 712 | 713 | United States 714 | 715 | EMail: michael@casadevall.pro 716 | 717 |
718 |
719 | 720 | 721 | 722 | -------------------------------------------------------------------------------- /docs/draft-ftl-specification-00.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ftl-protocol-dev/ftl-specification/07fc3f90d8e8b653a0ade67c2231a7dfb3e9b7fd/docs/draft-ftl-specification-00.pdf -------------------------------------------------------------------------------- /docs/draft-ftl-specification-00.txt: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | Faster-Than-Light Standardization Effort M. Casadevall, Ed. 6 | Internet-Draft Indepedent 7 | Intended status: Informational January 5, 2021 8 | Expires: July 9, 2021 9 | 10 | 11 | Faster-Than-Light Streaming Protocl Specification 12 | draft-ftl-specification-00 13 | 14 | Abstract 15 | 16 | With the demise of Microsoft's Mixer, the future of the Faster-Than- 17 | Light (FTL) streaming protocol has been left in doubt. As the 18 | Internet's first practical subsecond streaming protocol, several 19 | successors to Mixer have decided to re-implement FTL from the 20 | original SDK and notes. While Mixer's original FTL specification had 21 | a de-facto specification in the form of ftl-sdk, the source code was 22 | in-complete, and several aspects of the FTL were left undocumented. 23 | 24 | In an effort to keep FTL viable and cross-service compatible, this 25 | specification denotes a canonical implementation of FTL, handshake 26 | protocols, WebRTC notes, and all relevant information as relating to 27 | FTL with the hope that FTL may still be continued as a vechile for 28 | low latency video streaming over the Internet. 29 | 30 | Status of This Memo 31 | 32 | This Internet-Draft is submitted in full conformance with the 33 | provisions of BCP 78 and BCP 79. 34 | 35 | Internet-Drafts are working documents of the Internet Engineering 36 | Task Force (IETF). Note that other groups may also distribute 37 | working documents as Internet-Drafts. The list of current Internet- 38 | Drafts is at https://datatracker.ietf.org/drafts/current/. 39 | 40 | Internet-Drafts are draft documents valid for a maximum of six months 41 | and may be updated, replaced, or obsoleted by other documents at any 42 | time. It is inappropriate to use Internet-Drafts as reference 43 | material or to cite them other than as "work in progress." 44 | 45 | This Internet-Draft will expire on July 9, 2021. 46 | 47 | Copyright Notice 48 | 49 | Copyright (c) 2021 IETF Trust and the persons identified as the 50 | document authors. All rights reserved. 51 | 52 | 53 | 54 | 55 | 56 | Casadevall Expires July 9, 2021 [Page 1] 57 | 58 | Internet-Draft Abbreviated Title January 2021 59 | 60 | 61 | This document is subject to BCP 78 and the IETF Trust's Legal 62 | Provisions Relating to IETF Documents 63 | (https://trustee.ietf.org/license-info) in effect on the date of 64 | publication of this document. Please review these documents 65 | carefully, as they describe your rights and restrictions with respect 66 | to this document. 67 | 68 | Table of Contents 69 | 70 | 1. Introduction and Overview . . . . . . . . . . . . . . . . . . 2 71 | 2. Design Tradeoffs/Limitations . . . . . . . . . . . . . . . . 3 72 | 3. One-to-Many WebRTC . . . . . . . . . . . . . . . . . . . . . 4 73 | 4. A Note on SSRCs . . . . . . . . . . . . . . . . . . . . . . . 4 74 | 5. Charon Negotiation Protocol . . . . . . . . . . . . . . . . . 4 75 | 5.1. Message Format . . . . . . . . . . . . . . . . . . . . . 5 76 | 5.1.1. On Linebreaks . . . . . . . . . . . . . . . . . . . . 5 77 | 5.2. Response Codes . . . . . . . . . . . . . . . . . . . . . 6 78 | 5.3. Charon Verbs . . . . . . . . . . . . . . . . . . . . . . 6 79 | 5.3.1. HMAC . . . . . . . . . . . . . . . . . . . . . . . . 6 80 | 5.3.2. CONNECT . . . . . . . . . . . . . . . . . . . . . . . 6 81 | 5.3.2.1. ProtocolVersion . . . . . . . . . . . . . . . . . 7 82 | 5.3.2.2. VendorName . . . . . . . . . . . . . . . . . . . 7 83 | 5.3.2.3. VendorVersion . . . . . . . . . . . . . . . . . . 7 84 | 5.3.2.4. Video . . . . . . . . . . . . . . . . . . . . . . 7 85 | 5.3.2.5. VideoCodex . . . . . . . . . . . . . . . . . . . 7 86 | 5.3.2.6. VideoHeight . . . . . . . . . . . . . . . . . . . 7 87 | 5.3.2.7. VideoWidth . . . . . . . . . . . . . . . . . . . 7 88 | 5.3.2.8. VideoPayloadType . . . . . . . . . . . . . . . . 7 89 | 5.3.2.9. VideoIngestSSRC . . . . . . . . . . . . . . . . . 7 90 | 5.3.2.10. Audio . . . . . . . . . . . . . . . . . . . . . . 7 91 | 5.3.2.11. AudioCodec . . . . . . . . . . . . . . . . . . . 7 92 | 5.3.2.12. AudioPayloadType . . . . . . . . . . . . . . . . 8 93 | 5.3.2.13. AudioIngestSSRC . . . . . . . . . . . . . . . . . 8 94 | 5.3.3. DISCONNECT . . . . . . . . . . . . . . . . . . . . . 8 95 | 5.3.4. PING . . . . . . . . . . . . . . . . . . . . . . . . 8 96 | 6. Styx Protocol Ingest Behavior . . . . . . . . . . . . . . . . 8 97 | 7. Babel Transcoding Behavior . . . . . . . . . . . . . . . . . 8 98 | 8. WebRTC Last Mile Negotations . . . . . . . . . . . . . . . . 8 99 | Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 8 100 | 101 | 1. Introduction and Overview 102 | 103 | This specification covers several components of the FTL protocol 104 | stream, and is primarily derieved from the implementation used at 105 | Mixer and the freely available source code in ftl-sdk. This document 106 | details the handshaking protocol known as Charon, the SRTP ingest 107 | behaviors, and defines a recommended ingest->endpoint streaming 108 | 109 | 110 | 111 | 112 | Casadevall Expires July 9, 2021 [Page 2] 113 | 114 | Internet-Draft Abbreviated Title January 2021 115 | 116 | 117 | protocol, as well as notes in regards to implementation of the last 118 | mile WebRTC connections. 119 | 120 | FTL was specifically designed with the following objectives in mind 121 | which is must handle at all times. 122 | 123 | o Real-world 500 millsecond delay for streamer to receiver broadcast 124 | under normal cirmstances at 30 FPS or more 125 | 126 | * At the time of it's implementation, 720p streaming was 127 | considered the best possible for most users, however, 128 | advancements in technology have allowed for 1080p streaming. 129 | 130 | o That FTL has be technology neutral; it should be able to use any 131 | video or audio codec as supported by WebRTC (or another last mile 132 | technology) independently. The original FTL implemntation used 133 | VP8 and Opus, later ones used H.264, keeping with the original 134 | intent. 135 | 136 | o It is expected that to reduce latency, an FTL deployment has 137 | multiple ingest and points of presenses for client connections. A 138 | stream connects to one ingest point, and then data is routed to 139 | points of presence as necessary. 140 | 141 | o FTL end-points must support being behind anycast, as well as use 142 | of STUN, and TURN if necessary 143 | 144 | o Use of standards based technology for use in a web browser with no 145 | additional software or downloads for viewers 146 | 147 | As originally designed, the following criticera were also kept in 148 | mind, although not realized at least during the initial 149 | implementation phases. 150 | 151 | o Use of IPv6 152 | 153 | * IPv6 deployment and usage was considered highly desirable for 154 | multiple reasons, primarily to simplify routing and mandated 155 | multicast support; as implemented, there should be no known 156 | IPv6 problems, but the protocol never tested either. 157 | 158 | 2. Design Tradeoffs/Limitations 159 | 160 | TBD 161 | 162 | 163 | 164 | 165 | 166 | 167 | 168 | Casadevall Expires July 9, 2021 [Page 3] 169 | 170 | Internet-Draft Abbreviated Title January 2021 171 | 172 | 173 | 3. One-to-Many WebRTC 174 | 175 | TBD 176 | 177 | 4. A Note on SSRCs 178 | 179 | TBD 180 | 181 | 5. Charon Negotiation Protocol 182 | 183 | Charon handles negotiation of RTP streams to Styx, and acts as an out 184 | of band signaling method for FTL stream behavior. The Charon 185 | connection MUST be kept-alive at all times as a TCP/IP connection on 186 | port 8084 (MC: should apply for IANA number). Upon establishment of 187 | a CONNECT, a client must send PING messages once every five seconds, 188 | or the server MAY time-out the connection. 189 | 190 | It is RECOMMENDED that the server wait 10 seconds before assuming 191 | that a client has hung and hanging up. PING messages must be 192 | responded to with a 201 respond code. The server should send 5XX 193 | Timeout message before disconnecting. (Ed: fix this and actually put 194 | the client behavior) 195 | 196 | If the TCP/IP connection is reset, the broadcaster MUST assume that 197 | the ingest point of presence has become unavailable, and begin clean- 198 | up and teardown. Likewise, the ingest daemon MUST begin teardown and 199 | disregard any UDP traffic from the streamer upon Charon connection 200 | loss. 201 | 202 | The Charon protocol is built upon ASCII verbs with optional 203 | arguments. The lifecycle of this connection under normal 204 | circumstances is as follows. 205 | 206 | o Broadcaster connects to ingest on TCP/IP 8084 207 | 208 | o Broadcaster gets HMAC authentication 209 | 210 | o Broadcaster generates HMAC authentication based off streamer 211 | connection ID and channel idea 212 | 213 | o Broadcaster sends CONNECT, combined with stream paramters. 214 | 215 | o Ingest sends FTL_OK or FTL_REJECT based on settings 216 | 217 | o If FLT_OK, Broadcaster sends RTP streams to the media port 218 | indicated by the ingest 219 | 220 | o Broadcaster starts RTP streams per Ingests response 221 | 222 | 223 | 224 | Casadevall Expires July 9, 2021 [Page 4] 225 | 226 | Internet-Draft Abbreviated Title January 2021 227 | 228 | 229 | o Every five seconds, a PING response is sent, ingest replies with 230 | 201 231 | 232 | o Broadcaster sends DISCONNECT for orderly shutdown 233 | 234 | 5.1. Message Format 235 | 236 | Charon is directly modeled on SMTP commands and response codes. As 237 | such, commands take the form of a verb, followed by a three digital 238 | response. An example exchange looks as such: 239 | 240 | Client: PING\r\n\r\n 241 | Server: 201 Ping OK!\r\n\r\n 242 | 243 | For legacy reasons, linebreaks in Charon are encoded as '\r\n\r\n' 244 | (hex 0x0D0A0D0A), with an unusually complex implementation detail. 245 | See the section "On Linebreaks" for more details. 246 | 247 | For commands that do not need additional meta-data, the server SHALL 248 | process them immediately upon receiving a linebreak, otherwise, a 249 | multiline format using RFC822 headers is defined, with the message 250 | ended with a single period. Unknown headers MUST be disregarded by 251 | the server. This message exchange looks as follows: 252 | 253 | Client: CONNECT 1234-myhashhere\r\n\r\n 254 | C: Video: true\r\n\r\n 255 | C: Audio: true\r\n\r\n 256 | C: .\r\n\r\n 257 | Server: 200 Send UDP!\r\n\r\n 258 | 259 | The Charon protocol does not allow for transmission of binary data 260 | without encoding. If necessary, it is RECOMMENDED that binary data 261 | base encoded in Base64. 262 | 263 | 5.1.1. On Linebreaks 264 | 265 | The reference implementation of FTL uses the message signature 266 | '\r\n\r\n' as an end of line marker. This is an intentional 267 | implementation detail due to an unintentional similarity between 268 | FTL's CONNECT message, and the HTTP CONNECT proxy command, and 269 | Microsoft discovered that some commercial firewalls would intercept 270 | Charon's messages. The original justification was left as this 271 | comment in ftl_helpers.c 272 | 273 | 274 | 275 | 276 | 277 | 278 | 279 | 280 | Casadevall Expires July 9, 2021 [Page 5] 281 | 282 | Internet-Draft Abbreviated Title January 2021 283 | 284 | 285 | /* 286 | Please note that throughout the code, we send "\r\n\r\n", where a 287 | normal newline ("\n") would suffice. This is done due to some 288 | firewalls / anti-malware systems not passing our packets 289 | through when we don't send those double-windows-newlines. They 290 | seem to incorrectly detect our protocol as HTTP. 291 | */ 292 | 293 | A problem however arises that earlier implementations of FTL used 294 | "\n" as a newline indicator. While it is unlikely that any of these 295 | legacy FTL clients are still in use, clients and ingests must use the 296 | following behaviors 297 | 298 | Charon servers MUST disregard any empty newline, and treat "\r\n" and 299 | "\n" as identical for the purposes of message parsing. In effect, 300 | all the following are the same. 301 | 302 | CONNECT 1234-hash\n 303 | Option1: 1234\n 304 | Option2: 1234\n 305 | .\n 306 | 307 | CONNECT 1234-hash\r\n 308 | Option1: 1234\r\n 309 | Option2: 1234\r\n 310 | .\r\n 311 | 312 | CONNECT 1234-hash\r\n 313 | Option1: 1234\r\n\r\n 314 | Option2: 1234\r\n\r\n 315 | .\r\n\ 316 | 317 | Charon clients are MUST to use '\r\n\r\n' when transmitting commands 318 | over clear text. 319 | 320 | 5.2. Response Codes 321 | 322 | 5.3. Charon Verbs 323 | 324 | 5.3.1. HMAC 325 | 326 | TDB 327 | 328 | 5.3.2. CONNECT 329 | 330 | TDB 331 | 332 | 333 | 334 | 335 | 336 | Casadevall Expires July 9, 2021 [Page 6] 337 | 338 | Internet-Draft Abbreviated Title January 2021 339 | 340 | 341 | 5.3.2.1. ProtocolVersion 342 | 343 | TBD 344 | 345 | 5.3.2.2. VendorName 346 | 347 | TBD 348 | 349 | 5.3.2.3. VendorVersion 350 | 351 | TBD 352 | 353 | 5.3.2.4. Video 354 | 355 | TBD 356 | 357 | 5.3.2.5. VideoCodex 358 | 359 | TBD 360 | 361 | 5.3.2.6. VideoHeight 362 | 363 | TBD 364 | 365 | 5.3.2.7. VideoWidth 366 | 367 | TBD 368 | 369 | 5.3.2.8. VideoPayloadType 370 | 371 | TBD 372 | 373 | 5.3.2.9. VideoIngestSSRC 374 | 375 | TBD 376 | 377 | 5.3.2.10. Audio 378 | 379 | TBD 380 | 381 | 5.3.2.11. AudioCodec 382 | 383 | TBD 384 | 385 | 386 | 387 | 388 | 389 | 390 | 391 | 392 | Casadevall Expires July 9, 2021 [Page 7] 393 | 394 | Internet-Draft Abbreviated Title January 2021 395 | 396 | 397 | 5.3.2.12. AudioPayloadType 398 | 399 | TBD 400 | 401 | 5.3.2.13. AudioIngestSSRC 402 | 403 | TBD 404 | 405 | 5.3.3. DISCONNECT 406 | 407 | TBD 408 | 409 | 5.3.4. PING 410 | 411 | TBD 412 | 413 | 6. Styx Protocol Ingest Behavior 414 | 415 | TBD 416 | 417 | 7. Babel Transcoding Behavior 418 | 419 | TBD 420 | 421 | 8. WebRTC Last Mile Negotations 422 | 423 | TBD 424 | 425 | Author's Address 426 | 427 | Michael Casadevall (editor) 428 | Indepedent 429 | Jersey City 430 | United States 431 | 432 | Email: michael@casadevall.pro 433 | 434 | 435 | 436 | 437 | 438 | 439 | 440 | 441 | 442 | 443 | 444 | 445 | 446 | 447 | 448 | Casadevall Expires July 9, 2021 [Page 8] 449 | -------------------------------------------------------------------------------- /docs/index.html: -------------------------------------------------------------------------------- 1 | draft-ftl-specification-00.html -------------------------------------------------------------------------------- /draft-ftl-specification-00.xml: -------------------------------------------------------------------------------- 1 | 2 | 4 | 8 | 9 | 10 | 11 | 12 | ]> 13 | 14 | 15 | 17 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 43 | 44 | 45 | 46 | 47 | 49 | 50 | Faster-Than-Light Streaming Protocl Specification 51 | 52 | 53 | 54 | 55 | 56 | 57 | Indepedent 58 |
59 | 60 | 61 | 62 | 63 | Jersey City 64 | 65 | 66 | United States 67 | 68 | michael@casadevall.pro 69 | 70 |
71 | 72 | 73 | 74 | 80 | 81 | 82 | 83 | General 84 | Faster-Than-Light Standardization Effort 85 | template 86 | 87 | 88 | With the demise of Microsoft's Mixer, the future of the Faster-Than-Light (FTL) streaming protocol has been left in doubt. 89 | As the Internet's first practical subsecond streaming protocol, several successors to Mixer have decided to re-implement 90 | FTL from the original SDK and notes. While Mixer's original FTL specification had a de-facto specification in the form of 91 | ftl-sdk, the source code was in-complete, and several aspects of the FTL were left undocumented. 92 | 93 | In an effort to keep FTL viable and cross-service compatible, this specification denotes a canonical implementation of FTL, 94 | handshake protocols, WebRTC notes, and all relevant information as relating to FTL with the hope that FTL may still be continued 95 | as a vechile for low latency video streaming over the Internet. 96 | 97 | 98 | 99 | 100 |
101 | This specification covers several components of the FTL protocol stream, and is primarily derieved from the implementation used 102 | at Mixer and the freely available source code in ftl-sdk. This document details the handshaking protocol known as Charon, the 103 | SRTP ingest behaviors, and defines a recommended ingest->endpoint streaming protocol, as well as notes in regards to implementation 104 | of the last mile WebRTC connections. 105 | 106 | FTL was specifically designed with the following objectives in mind which is must handle at all times. 107 | 108 | 109 | 110 | Real-world 500 millsecond delay for streamer to receiver broadcast under normal cirmstances at 30 FPS or more 111 | 112 | At the time of it's implementation, 720p streaming was considered the best possible for most users, however, 113 | advancements in technology have allowed for 1080p streaming. 114 | 115 | 116 | That FTL has be technology neutral; it should be able to use any video or audio codec as supported by WebRTC 117 | (or another last mile technology) independently. The original FTL implemntation used VP8 and Opus, later ones 118 | used H.264, keeping with the original intent. 119 | It is expected that to reduce latency, an FTL deployment has multiple ingest and points of presenses for client connections. 120 | A stream connects to one ingest point, and then data is routed to points of presence as necessary. 121 | FTL end-points must support being behind anycast, as well as use of STUN, and TURN if necessary 122 | Use of standards based technology for use in a web browser with no additional software or downloads for viewers 123 | 124 | 125 | 126 | As originally designed, the following criticera were also kept in mind, although not realized at least during the initial 127 | implementation phases. 128 | 129 | 130 | Use of IPv6 131 | 132 | IPv6 deployment and usage was considered highly desirable for multiple reasons, primarily to simplify 133 | routing and mandated multicast support; as implemented, there should be no known IPv6 problems, but 134 | the protocol never tested either. 135 | 136 | 137 | 138 |
139 | 140 |
141 | TBD 142 |
143 | 144 |
145 | TBD 146 |
147 | 148 |
149 | TBD 150 |
151 | 152 |
153 | Charon handles negotiation of RTP streams to Styx, and acts as an out of band signaling 154 | method for FTL stream behavior. The Charon connection MUST be kept-alive at all times as 155 | a TCP/IP connection on port 8084 (MC: should apply for IANA number). Upon establishment of 156 | a CONNECT, a client MUST send PING messages once every five seconds, or the server MAY time-out 157 | the connection. 158 | 159 | It is RECOMMENDED that the server wait 10 seconds before assuming that a client has hung and 160 | hanging up. PING messages must be responded to with a 201 respond code. The server SHOULD send 161 | 5XX Timeout message before disconnecting. (Ed: fix this and actually put the client behavior) 162 | 163 | If the TCP/IP connection is reset, the broadcaster MUST assume that the ingest point of presence 164 | has become unavailable, and begin clean-up and teardown. Likewise, the ingest daemon MUST begin 165 | teardown and disregard any UDP traffic from the streamer upon Charon connection loss. 166 | 167 | The Charon protocol is built upon ASCII verbs with optional arguments. The lifecycle of this 168 | connection under normal circumstances is as follows. 169 | 170 | 171 | 172 | Broadcaster connects to ingest on TCP/IP 8084 173 | Broadcaster gets HMAC authentication 174 | Broadcaster generates HMAC authentication based off streamer connection ID and channel idea 175 | Broadcaster sends CONNECT, combined with stream paramters. 176 | Ingest sends FTL_OK or FTL_REJECT based on settings 177 | If FLT_OK, Broadcaster sends RTP streams to the media port indicated by the ingest 178 | Broadcaster starts RTP streams per Ingests response 179 | Every five seconds, a PING response is sent, ingest replies with 201 180 | Broadcaster sends DISCONNECT for orderly shutdown 181 | 182 | 183 |
184 | Charon is directly modeled on SMTP commands and response codes. As such, commands take the form of a verb, 185 | followed by a three digital response. An example exchange looks as such: 186 | 187 |
188 | 192 |
193 | 194 | For legacy reasons, linebreaks in Charon are encoded as '\r\n\r\n' (hex 0x0D0A0D0A), with an unusually complex 195 | implementation detail. See the section "On Linebreaks" for more details. 196 | 197 | For commands that do not need additional meta-data, the server SHALL process them immediately upon receiving a 198 | linebreak, otherwise, a multiline format using RFC822 headers is defined, with the message ended with a single 199 | period. Unknown headers MUST be disregarded by the server. This message exchange looks as follows: 200 | 201 |
202 | 209 |
210 | 211 | The Charon protocol does not allow for transmission of binary data without encoding. If necessary, it is RECOMMENDED 212 | that binary data base encoded in Base64. 213 | 214 |
215 | The reference implementation of FTL uses the message signature '\r\n\r\n' as an end of line marker. 216 | This is an intentional implementation detail due to an unintentional similarity between FTL's CONNECT message, 217 | and the HTTP CONNECT proxy command, and Microsoft discovered that some commercial firewalls would intercept 218 | Charon's messages. The original justification was left as this comment in ftl_helpers.c 219 | 220 |
221 | 230 |
231 | A problem however arises that earlier implementations of FTL used "\n" as a newline indicator. 232 | While it is unlikely that any of these legacy FTL clients are still in use, clients and ingests 233 | must use the following behaviors 234 | 235 | Charon servers MUST disregard any empty newline, and treat "\r\n" and "\n" as identical for the purposes 236 | of message parsing. In effect, all the following are the same. 237 |
238 | 254 |
255 | Charon clients MUST to use '\r\n\r\n' when transmitting commands over clear text. 256 |
257 |
258 |
259 |
260 |
261 | TDB 262 |
263 |
264 | TDB 265 |
266 | TBD 267 |
268 |
269 | TBD 270 |
271 |
272 | TBD 273 |
274 |
275 | TBD 276 |
277 |
278 | TBD 279 |
280 |
281 | TBD 282 |
283 |
284 | TBD 285 |
286 |
287 | TBD 288 |
289 |
290 | TBD 291 |
292 |
293 | TBD 294 |
295 |
296 | TBD 297 |
298 |
299 | TBD 300 |
301 |
302 | TBD 303 |
304 |
305 |
306 | TBD 307 |
308 |
309 | TBD 310 |
311 |
312 |
313 | 314 |
315 | TBD 316 |
317 | 318 |
319 | TBD 320 |
321 |
322 | TBD 323 |
324 | 325 | 326 | 327 | 328 | 329 | 330 | 334 | 335 | --------------------------------------------------------------------------------