Scalability Mode Dependency Diagrams
982 |983 | Dependency diagrams for the scability modes defined in this specification 984 | are provided below. 985 |
986 |13 | This document defines a set of ECMAScript APIs in WebIDL to extend the WebRTC 14 | specification to enable configuration of encoding parameters for Scalable 15 | Video Coding (SVC). Discovery of SVC encoder and decoder capabilities is 16 | handled by the Media Capabilities specification. 17 |
18 |21 | The API is based on preliminary work done in the W3C ORTC Community Group. 22 |
23 |27 | This specification extends the WebRTC specification [[WEBRTC]] to 28 | enable configuration of encoding parameters for Scalable Video 29 | Coding (SVC). Discovery of SVC encoder and decoder capabilities is 30 | handled by the Media Capabilities API [[?Media-Capabilities]]. 31 |
32 |33 | This specification does not change the behavior of WebRTC objects and 34 | methods. Therefore, restrictions relating to Offer/Answer negotiation and 35 | encoding parameters remain, as described in [[WEBRTC]] Section 5.2: 36 | "{{RTCRtpSender/setParameters()}} does not cause SDP renegotiation and can 37 | only be used to change what the media stack is sending or receiving within the 38 | envelope negotiated by Offer/Answer." 39 |
40 |41 | As a result, this specification can configure encoding parameters for 42 | codecs that do not negotiate SVC support in Offer/Answer, such as the 43 | VP8 [[?RFC6386]], VP9 [[?VP9]] and AV1 [[?AV1]] codecs. 44 | Configuraton of temporal scalability can also be supported 45 | for the H.264 [[?ITU-T-REC-H.264]] and H.265 [[?ITU-T-REC-H.265]] codecs. 46 |
47 |50 | This specification defines conformance criteria that apply to a single 51 | product: the user agent that implements the interfaces that it 52 | contains. 53 |
54 |55 | Conformance requirements phrased as algorithms or specific steps may be 56 | implemented in any manner, so long as the end result is equivalent. In 57 | particular, the algorithms defined in this specification are intended to be 58 | easy to follow, and not intended to be performant. 59 |
60 |61 | Implementations that use ECMAScript to implement the APIs defined in 62 | this specification MUST implement them in a manner consistent with the 63 | ECMAScript Bindings defined in the Web IDL specification [[WEBIDL]], as 64 | this specification uses that specification and terminology. 65 |
66 |70 | The term "simulcast envelope" is defined in [[!WEBRTC]] Section 5.4.1. 71 |
72 |73 | This specification references objects, methods, internal slots 74 | and dictionaries defined in [[!WEBRTC]]. 75 |
76 |77 | For Scalable Video Coding (SVC), the terms "single-session transmission" 78 | (SST) and "multi-session transmission" (MST) are 79 | defined in [[?RFC6190]]. This specification only supports SST but 80 | not MST. 81 |
82 |83 | The term "Single Real-time Transport Protocol stream Single Transport" 84 | (SRST), defined in [[RFC7656]] Section 3.7, refers to SVC 85 | implementations that transmit all layers within a single transport, 86 | using a single Real-time Transport Protocol (RTP) stream and 87 | synchronization source (SSRC). The term "Multiple RTP stream Single 88 | Transport" (MRST), also defined in [[RFC7656]] Section 3.7, 89 | refers to implementations that transmit all layers within a single 90 | transport, using multiple RTP streams with a distinct SSRC for each layer. 91 | This specification only supports SRST, not MRST. Codecs 92 | with RTP payload specifications supporting SRST include VP8 93 | [[?RFC7741]], VP9 [[?VP9-PAYLOAD]], AV1 [[?AV1-RTP-SPEC]], H.264 94 | [[?RFC6184]] and H.265 [[?RFC7798]]. 95 |
96 |97 | The term "S mode" refers to a scalability mode in which multiple 98 | encodings are sent on the same SSRC. This includes the [="S2T1"=], [="S2T1h"=], 99 | [="S2T2"=], [="S2T2h"=], [="S2T3"=], [="S2T3h"=], [="S3T1"=], [="S3T1h"=], [="S3T2"=], [="S3T2h"=], 100 | [="S3T3"=] and [="S3T3h"=] {{RTCRtpEncodingParameters/scalabilityMode}} values. 101 |
102 |103 | The term Selective Forwarding Middlebox 104 | (SFM) is defined in Section 3.7 of [[RFC7667]]. 105 |
106 |110 | This specification enables the configuration of encoding parameters for SVC 111 | by extending the {{RTCRtpEncodingParameters}} dictionary. 112 |
113 |partial dictionary RTCRtpEncodingParameters {
117 | DOMString scalabilityMode;
118 | };
119 |
126 | This member can only be [=map/exists|present=] if the sender's
127 | kind is "video".
128 | When [=map/exists|present=], indicates the case-sensitive
129 | identifier of the scalability mode to be
130 | used for this stream. Scalability modes are defined in
131 | Section 5.
132 |
141 | [[!WEBRTC]] describes error handling in {{RTCPeerConnection/addTransceiver()}} 142 | (Section 5.1) and {{RTCRtpSender/setParameters()}} (Section 5.2), 143 | including use of {{RTCError}} to indicate a 144 | {{RTCErrorDetailType/"hardware-encoder-error"}} 145 | due to an unsupported encoding parameter, as well as other errors. 146 | Implementations utilize {{RTCError}} and other errors 147 | in the prescribed manner when an invalid 148 | {{RTCRtpEncodingParameters/scalabilityMode}} value is provided to 149 | {{RTCRtpSender/setParameters()}} or {{RTCPeerConnection/addTransceiver()}}. 150 |
151 |154 | [[!WEBRTC]] Section 5.1 describes validation of 155 | {{RTCRtpTransceiverInit/sendEncodings}} within 156 | {{RTCPeerConnection/addTransceiver()}}. To validate 157 | {{RTCRtpEncodingParameters/scalabilityMode}}, add 158 | the following steps after step 3 of 159 | [=RTCPeerConnection/addTransceiver sendEncodings validation steps=]: 160 |
true and
177 | sendEncodings [=list/contains=] any encoding whose
178 | {{RTCRtpEncodingParameters/scalabilityMode}} value
179 | represents an "S mode" and whose
180 | {{RTCRtpEncodingParameters/active}} member has a
181 | value of true, [= exception/throw =] an
182 | {{OperationError}}.
183 | 187 | When the {{RTCPeerConnection/addTransceiver()}} and 188 | {{RTCRtpTransceiver/setCodecPreferences()}} methods are called 189 | prior to conclusion of the Offer/Answer negotiation, the negotiated 190 | codec and its capabilities may not be known. In this situation the 191 | {{RTCRtpEncodingParameters/scalabilityMode}} values configured 192 | in {{RTCRtpTransceiverInit/sendEncodings}} may not be supported 193 | by the eventually negotiated codec. However, an error will only result 194 | if the requested {{RTCRtpEncodingParameters/scalabilityMode}} value is 195 | invalid for any supported codec, or if mixed simulcast transport is requested. 196 |
197 |198 | So as to ensure that the desired {{RTCRtpEncodingParameters/scalabilityMode}} 199 | values can be applied, {{RTCRtpTransceiver/setCodecPreferences()}} can be 200 | used to prefer or only include codecs supporting the desired configuration. 201 | For example, if temporal scalability is desired along with spatial simulcast, 202 | when {{RTCPeerConnection/addTransceiver()}} is called, 203 | {{RTCRtpTransceiverInit/sendEncodings}} can be configured to send multiple 204 | simulcast streams with different resolutions, with each stream 205 | utilizing temporal scalability. If only the VP8, VP9 and AV1 codec implementations 206 | support temporal scalability, {{RTCRtpTransceiver/setCodecPreferences()}} 207 | can be used to remove the H.264/AVC codec from the Offer, improving the 208 | chances that a codec supporting temporal scalability is negotiated. 209 |
210 |211 | When {{RTCRtpTransceiverInit/sendEncodings}} is used to 212 | request the sending of multiple simulcast streams using 213 | {{RTCPeerConnection/addTransceiver()}}, an "S mode" cannot 214 | be requested. The browser may only be configured to send 215 | simulcast encodings with multiple SSRCs and RIDs, or 216 | alternatively, to send all simulcast encodings on a single 217 | RTP stream. Simultaneously using both simulcast transport 218 | techniques is not permitted. 219 |
220 |224 | [[!WEBRTC]] Section 5.2 describes validation of parameters 225 | within {{RTCRtpSender/setParameters()}}. Insert the following 226 | conditions under which the operation causes a promise rejected 227 | with an {{InvalidModificationError}} (step 4) of 228 | [=RTCRtpSender/setParameters validation steps=]: 229 |
true and
253 | encodings contains any encoding whose
254 | {{RTCRtpEncodingParameters/scalabilityMode}} value
255 | represents an "S mode" and whose
256 | {{RTCRtpEncodingParameters/active}} member has a
257 | value of true.
258 | 262 | The [="L1T1"=] scalability mode enables SVC encoding to be turned off 263 | using {{RTCRtpSender/setParameters()}}. If [="L1T1"=] is set using 264 | {{RTCRtpSender/setParameters()}} then it will be returned in response 265 | to {{RTCRtpSender/getParameters()}}. 266 |
267 |271 | Before the initial negotiation has completed, 272 | {{RTCRtpSender/getParameters()}} returns the 273 | {{RTCRtpEncodingParameters/scalabilityMode}} value for each 274 | encoding in encodings, as last set by 275 | {{RTCPeerConnection/addTransceiver()}} or 276 | {{RTCRtpSender/setParameters()}}. If no 277 | {{RTCRtpEncodingParameters/scalabilityMode}} value 278 | was provided for an encoding in encodings, 279 | or if a value was not successfully set, then 280 | {{RTCRtpSender/getParameters()}} will not return a 281 | {{RTCRtpEncodingParameters/scalabilityMode}} value 282 | for that encoding. 283 |
284 |285 | After the initial negotiation has completed, {{RTCRtpSender/getParameters()}} 286 | returns the currently configured {{RTCRtpEncodingParameters/scalabilityMode}} 287 | value for each encoding in encodings which had a value before the 288 | initial negotiation. This MAY be different from the values requested in 289 | {{RTCPeerConnection/addTransceiver()}} or {{RTCRtpSender/setParameters()}}. 290 | For example, if the codecs selected during negotiation do not include an 291 | encoder supporting the desired {{RTCRtpEncodingParameters/scalabilityMode}} 292 | value, the user agent MAY select another value. If the configuration is not 293 | satisfactory, {{RTCRtpSender/setParameters()}} can be used to change it. 294 |
295 |296 | If {{RTCPeerConnection/addTransceiver()}} or {{RTCRtpSender/setParameters()}} 297 | did not provide a {{RTCRtpEncodingParameters/scalabilityMode}} value for 298 | an encoding in encodings, then after the initial negotiation 299 | has completed, {{RTCRtpSender/getParameters()}} will not return a 300 | {{RTCRtpEncodingParameters/scalabilityMode}} value and the encoder will use 301 | the default {{RTCRtpEncodingParameters/scalabilityMode}} of the codec for 302 | that encoding's RTP stream. The default {{RTCRtpEncodingParameters/scalabilityMode}} 303 | for each codec is implementation dependent. The default 304 | {{RTCRtpEncodingParameters/scalabilityMode}} SHOULD be one of the 305 | temporal scalability modes (e.g. [="L1T1"=],[="L1T2"=],[="L1T3"=], etc.). 306 |
307 |311 | SVC is most often employed in video conferencing, where a conferencing 312 | server such as a SFM 313 | selectively forwards layers to participants based on their device 314 | characteristics and available bandwidth. In such an environment, 315 | the application will negotiate codecs to be sent and received with 316 | the conferencing server. However, since scalability modes are not 317 | negotiated in Offer/Answer, the application needs to determine 318 | which scalability modes the browser and conference server support 319 | by other means. 320 |
321 |322 | The [[?Media-Capabilities]] API enables discovery of encoder and decoder 323 | support for scalable video coding. {{VideoConfiguration//scalabilityMode}} 324 | is used to query whether an encoder supports a 325 | {{RTCRtpEncodingParameters/scalabilityMode}} value, indicating whether 326 | it is "supported", "smooth" and "power efficient". 327 |
328 |
329 | The [[?Media-Capabilities]] API also provides information on decoder support
330 | for spatial scalablity modes. {{VideoConfiguration/spatialScalability}}
331 | indicates whether a decoder has the ability to support spatial prediction,
332 | which requires the ability to use frames of a resolution different than
333 | the current resolution as a dependency. If {{VideoConfiguration/spatialScalability}}
334 | is set to true, the decoder can decode any
335 | {{RTCRtpEncodingParameters/scalabilityMode}} value supported by the encoder.
336 | If {{VideoConfiguration/spatialScalability}} is set to false
337 | or is absent, the decoder cannot decode spatial scalability modes, but can
338 | can decode all other {{RTCRtpEncodingParameters/scalabilityMode}} values
339 | supported by the encoder.
340 |
342 | Once an application has determined which combination of codecs and 343 | {{RTCRtpEncodingParameters/scalabilityMode}} values it has available to use, 344 | it needs to determine which of these combinations are supported by the SFM. 345 | One way to handle this is for the SFM to indicate the combination of codecs 346 | and scalability modes it can forward in the form of receiver capabilities 347 | After receiving the SFM's capabilities, the application can compute the 348 | intersection of codec and {{RTCRtpEncodingParameters/scalabilityMode}} values 349 | supported by the browser's {{RTCRtpSender}} and the 350 | SFM's receiver. This 351 | can be used to determine the arguments passed to the browser's 352 | {{RTCPeerConnection/addTransceiver()}} and {{RTCRtpSender/setParameters()}} 353 | methods. 354 |
355 |356 | Here are some examples: 357 |
381 | There are situations in which computing the intersection of the browser and 382 | SFM capabilities needs to take RTP header extensions into account. If the 383 | SFM cannot parse codec payloads 384 | (either because it is not designed to do so, or because the payloads are encrypted), 385 | then negotiation of an RTP header extension (such as the AV1 Dependency Descriptor 386 | defined in Appendix A of [[?AV1-RTP-SPEC]]) could be required for the 387 | SFM to forward a particular codec. 388 | To take this into account, the RTP header extensions required for forwarding of a 389 | codec could be added to the SFM's receiver capabilities. The application could 390 | then compute the intersection of codec, header extension and 391 | {{RTCRtpEncodingParameters/scalabilityMode}} values supported 392 | by the browser's {{RTCRtpSender}} and the 393 | SFM's receiver. 394 |
395 |401 | The {{RTCRtpEncodingParameters/scalabilityMode}} values supported in this specification, 402 | as well as their associated identifiers and characteristics, are provided in the table 403 | below. The names of the {{RTCRtpEncodingParameters/scalabilityMode}} values (which are 404 | case sensitive) are provided, along with the scalability mode identifiers assigned in 405 | [[?AV1]] Section 6.7.5, and links to dependency diagrams provided in Section 9. 406 |
407 |408 | While the [[?AV1]] and VP9 [[?VP9]] specifications support all the 409 | {{RTCRtpEncodingParameters/scalabilityMode}} values defined in the table, other codec 410 | specifications do not. For example, VP8 [[?RFC6386]], H.264 [[?RFC6184]] and 411 | H.265 [[?RFC7798]] only support temporal scalability (e.g. [="L1T2"=], [="L1T3"=]). 412 | Also VP8 [[?RFC6386]], H.264 [[?RFC6184]] and H.265 [[?RFC7798]] only permit transport 413 | of simulcast on distinct SSRCs, so that "S" modes (where multiple encodings are 414 | transported on a single RTP stream) are not supported. 415 |
416 || Scalability Mode Identifier | 420 |Spatial Layers | 421 |Resolution Ratio | 422 |Temporal Layers | 423 |Inter-layer dependency | 424 |AV1 scalability_mode_idc | 425 |
|---|---|---|---|---|---|
| "L1T1" | 430 |1 | 431 |432 | | 1 | 433 |434 | | N/A | 435 |
| "L1T2" | 438 |1 | 439 |440 | | 2 | 441 |442 | | SCALABILITY_L1T2 | 443 |
| "L1T3" | 446 |1 | 447 |448 | | 3 | 449 |450 | | SCALABILITY_L1T3 | 451 |
| "L2T1" | 454 |2 | 455 |2:1 | 456 |1 | 457 |Yes | 458 |SCALABILITY_L2T1 | 459 |
| "L2T2" | 462 |2 | 463 |2:1 | 464 |2 | 465 |Yes | 466 |SCALABILITY_L2T2 | 467 |
| "L2T3" | 470 |2 | 471 |2:1 | 472 |3 | 473 |Yes | 474 |SCALABILITY_L2T3 | 475 |
| "L3T1" | 478 |3 | 479 |2:1 | 480 |1 | 481 |Yes | 482 |SCALABILITY_L3T1 | 483 |
| "L3T2" | 486 |3 | 487 |2:1 | 488 |2 | 489 |Yes | 490 |SCALABILITY_L3T2 | 491 |
| "L3T3" | 494 |3 | 495 |2:1 | 496 |3 | 497 |Yes | 498 |SCALABILITY_L3T3 | 499 |
| "L2T1h" | 502 |2 | 503 |1.5:1 | 504 |1 | 505 |Yes | 506 |SCALABILITY_L2T1h | 507 |
| "L2T2h" | 510 |2 | 511 |1.5:1 | 512 |2 | 513 |Yes | 514 |SCALABILITY_L2T2h | 515 |
| "L2T3h" | 518 |2 | 519 |1.5:1 | 520 |3 | 521 |Yes | 522 |SCALABILITY_L2T3h | 523 |
| "L3T1h" | 526 |3 | 527 |1.5:1 | 528 |1 | 529 |Yes | 530 |531 | |
| "L3T2h" | 534 |3 | 535 |1.5:1 | 536 |2 | 537 |Yes | 538 |539 | |
| "L3T3h" | 542 |3 | 543 |1.5:1 | 544 |3 | 545 |Yes | 546 |547 | |
| "S2T1" | 550 |2 | 551 |2:1 | 552 |1 | 553 |No | 554 |SCALABILITY_S2T1 | 555 |
| "S2T2" | 558 |2 | 559 |2:1 | 560 |2 | 561 |No | 562 |SCALABILITY_S2T2 | 563 |
| "S2T3" | 566 |2 | 567 |2:1 | 568 |3 | 569 |No | 570 |SCALABILITY_S2T3 | 571 |
| "S2T1h" | 574 |2 | 575 |1.5:1 | 576 |1 | 577 |No | 578 |SCALABILITY_S2T1h | 579 |
| "S2T2h" | 582 |2 | 583 |1.5:1 | 584 |2 | 585 |No | 586 |SCALABILITY_S2T2h | 587 |
| "S2T3h" | 590 |2 | 591 |1.5:1 | 592 |3 | 593 |No | 594 |SCALABILITY_S2T3h | 595 |
| "S3T1" | 598 |3 | 599 |2:1 | 600 |1 | 601 |No | 602 |SCALABILITY_S3T1 | 603 |
| "S3T2" | 606 |3 | 607 |2:1 | 608 |2 | 609 |No | 610 |SCALABILITY_S3T2 | 611 |
| "S3T3" | 614 |3 | 615 |2:1 | 616 |3 | 617 |No | 618 |SCALABILITY_S3T3 | 619 |
| "S3T1h" | 622 |3 | 623 |1.5:1 | 624 |1 | 625 |No | 626 |SCALABILITY_S3T1h | 627 |
| "S3T2h" | 630 |3 | 631 |1.5:1 | 632 |2 | 633 |No | 634 |SCALABILITY_S3T2h | 635 |
| "S3T3h" | 638 |3 | 639 |1.5:1 | 640 |3 | 641 |No | 642 |SCALABILITY_S3T3h | 643 |
| "L2T2_KEY" | 646 |2 | 647 |2:1 | 648 |2 | 649 |Yes | 650 |SCALABILITY_L3T2_KEY | 651 |
| "L2T2_KEY_SHIFT" | 654 |2 | 655 |2:1 | 656 |2 | 657 |Yes | 658 |SCALABILITY_L3T2_KEY_SHIFT | 659 |
| "L2T3_KEY" | 662 |2 | 663 |2:1 | 664 |3 | 665 |Yes | 666 |SCALABILITY_L3T3_KEY | 667 |
| "L2T3_KEY_SHIFT" | 670 |2 | 671 |2:1 | 672 |3 | 673 |Yes | 674 |SCALABILITY_L3T3_KEY_SHIFT | 675 |
| "L3T1_KEY" | 678 |3 | 679 |2:1 | 680 |1 | 681 |Yes | 682 |683 | |
| "L3T2_KEY" | 686 |3 | 687 |2:1 | 688 |2 | 689 |Yes | 690 |SCALABILITY_L4T5_KEY | 691 |
| "L3T2_KEY_SHIFT" | 694 |3 | 695 |2:1 | 696 |2 | 697 |Yes | 698 |SCALABILITY_L4T5_KEY_SHIFT | 699 |
| "L3T3_KEY" | 702 |3 | 703 |2:1 | 704 |3 | 705 |Yes | 706 |SCALABILITY_L4T7_KEY | 707 |
| "L3T3_KEY_SHIFT" | 710 |3 | 711 |2:1 | 712 |3 | 713 |Yes | 714 |SCALABILITY_L4T7_KEY_SHIFT | 715 |
When proposing a {{RTCRtpEncodingParameters/scalabilityMode}} value, the following principles should be followed:
721 |LxTy to denote a {{RTCRtpEncodingParameters/scalabilityMode}} with x
731 | spatial layers using a 2:1 resolution ratio and y temporal layers.
732 | LxTyh denotes x spatial layers with a 1.5:1 resolution ratio and
733 | y temporal layers. SxTy denotes a {{RTCRtpEncodingParameters/scalabilityMode}}
734 | with x simulcast encodings with a 2:1 resolution ratio, with each
735 | simulcast encoding containing y temporal layers. SxTyh denotes
736 | a 1.5:1 resolution ratio. LxTy_KEY denotes a {{RTCRtpEncodingParameters/scalabilityMode}}
737 | with x spatial layers using a 2:1 resolution ratio and y temporal layers in which spatial layers only
738 | depend on lower spatial layers at a key frame. LxTy_KEY_SHIFT modes denotes a
739 | {{RTCRtpEncodingParameters/scalabilityMode}} with x spatial layers using a 2:1 resolution ratio and
740 | y temporal layers in which spatial layers only depend on lower spatial layers at a key frame and subsequent
741 | frames have their temporal identifier shifted upward.
742 | 754 | This example extends [[WEBRTC]] Section 7.1 (Example 1) to demonstrate sending three spatial 755 | simulcast layers each with three temporal layers, using an SSRC and RID for each simulcast 756 | layer. Only the "sendEncodings" attribute is changed from the original example. 757 |
758 |
759 | const signaling = new SignalingChannel(); // handles JSON.stringify/parse
760 | const constraints = {audio: true, video: true};
761 | const configuration = {'iceServers': [{'urls': 'stun:stun.example.org'}]};
762 | let pc;
763 |
764 | // call start() to initiate
765 | async function start() {
766 | pc = new RTCPeerConnection(configuration);
767 |
768 | // let the "negotiationneeded" event trigger offer generation
769 | pc.onnegotiationneeded = async () => {
770 | try {
771 | await pc.setLocalDescription();
772 | // send the offer to the other peer
773 | signaling.send({description: pc.localDescription});
774 | } catch (err) {
775 | console.error(err);
776 | }
777 | };
778 |
779 | try {
780 | // get a local stream, show it in a self-view and add it to be sent
781 | const stream = await navigator.mediaDevices.getUserMedia(constraints);
782 | selfView.srcObject = stream;
783 | pc.addTransceiver(stream.getAudioTracks()[0], {direction: 'sendonly'});
784 | pc.addTransceiver(stream.getVideoTracks()[0], {
785 | direction: 'sendonly',
786 | sendEncodings: [
787 | {rid: 'q', scaleResolutionDownBy: 4.0, scalabilityMode: 'L1T3'},
788 | {rid: 'h', scaleResolutionDownBy: 2.0, scalabilityMode: 'L1T3'},
789 | {rid: 'f', scalabilityMode: 'L1T3'}
790 | ]
791 | });
792 | } catch (err) {
793 | console.error(err);
794 | }
795 | }
796 |
797 | signaling.onmessage = async ({data: {description, candidate}}) => {
798 | try {
799 | if (description) {
800 | await pc.setRemoteDescription(description);
801 | // if we got an offer, we need to reply with an answer
802 | if (description.type == 'offer') {
803 | await pc.setLocalDescription();
804 | signaling.send({description: pc.localDescription});
805 | }
806 | } else if (candidate) {
807 | await pc.addIceCandidate(candidate);
808 | }
809 | } catch (err) {
810 | console.error(err);
811 | }
812 | };
813 |
814 | 815 | This is an example with two spatial layers (with a 2:1 ratio) and three temporal layers. 816 |
817 |
818 | let sendEncodings = [
819 | {scalabilityMode: 'L2T3'}
820 | ];
821 |
822 | 823 | This is an example of mixed codec simulcast, with each simulcast layer having 3 temporal layers. 824 |
825 |
826 | let sendEncodings = [
827 | {rid: 'q', codec: {clockRate: 90000, mimeType: 'video/AV1'}, scaleResolutionDownBy: 4.0, scalabilityMode: 'L1T3'},
828 | {rid: 'h', codec: {clockRate: 90000, mimeType: 'video/VP8'}, scaleResolutionDownBy: 2.0, scalabilityMode: 'L1T3'},
829 | {rid: 'f', codec: {clockRate: 90000, mimeType: 'video/VP8'}, scalabilityMode: 'L1T3'}
830 | ];
831 |
832 | 833 | This is an example with three spatial simulcast layers each with three temporal layers on a single SSRC. 834 |
835 |
836 | let sendEncodings = [
837 | {scalabilityMode: 'S3T3'}
838 | ]
839 |
840 | 844 | This is an example of {{MediaCapabilities/encodingInfo(configuration)}} 845 | returned by a browser implementing [[WEBRTC]] and [[?Media-Capabilities]]. 846 |
847 |
848 | const contentType = 'video/VP9';
849 |
850 | const configuration = {
851 | type: 'webrtc',
852 | video: {
853 | contentType,
854 | width: 640,
855 | height: 480,
856 | bitrate: 10000,
857 | framerate: 29.97,
858 | scalabilityMode: 'L3T3_KEY'
859 | }
860 | };
861 |
862 | try {
863 | const info = await navigator.mediaCapabilities.encodingInfo(configuration);
864 |
865 | if (!info.supported) {
866 | console.log(`${contentType} is unsupported.`);
867 | return;
868 | }
869 | console.log(`${contentType} is ${info.smooth || 'NOT '}smooth, and ` +
870 | `${info.powerEfficient || 'NOT '}power efficient`);
871 | } catch (err) {
872 | console.error(err, ' caused encodingInfo to fail');
873 | }
874 |
875 | 879 | This is an example of receiver capabilities returned by an 880 | SFM that only 881 | supports forwarding of VP8, VP9 and AV1 temporal scalability modes. 882 |
883 |
884 | "codecs": [
885 | {
886 | "clockRate": 90000,
887 | "mimeType": "video/VP8",
888 | "scalabilityModes": [
889 | "L1T1",
890 | "L1T2",
891 | "L1T3"
892 | ]
893 | },
894 | {
895 | "clockRate": 90000,
896 | "mimeType": "video/VP9",
897 | "scalabilityModes": [
898 | "L1T1",
899 | "L1T2",
900 | "L1T3"
901 | ]
902 | },
903 | {
904 | "clockRate": 90000,
905 | "mimeType": "video/AV1",
906 | "scalabilityModes": [
907 | "L1T1",
908 | "L1T2",
909 | "L1T3"
910 | ]
911 | }
912 | ]
913 |
914 | 919 | This section is non-normative; it specifies no new behaviour, but 920 | instead summarizes information already present in other parts of the 921 | specification. The privacy considerations 922 | for the WebRTC APIs are described in [[WEBRTC]] Section 13. 923 |
924 |927 | In WebRTC, the use of scalable coding tools is not 928 | negotiated between peers, so neither supported 929 | {{RTCRtpEncodingParameters/scalabilityMode}} values nor 930 | decoder support for spatial prediction is exposed in SDP. 931 |
932 |933 | By attempting to set 934 | {{RTCRtpEncodingParameters/scalabilityMode}} values for 935 | each codec using the {{RTCRtpSender/setParameters()}} API, 936 | an application can determine the values supported by the 937 | encoder, by noting which configuration attempts succeed 938 | and which ones fail. However, this does not indicate 939 | whether a {{RTCRtpEncodingParameters/scalabilityMode}} 940 | value is supported by a hardware or software encoder 941 | (or both). Since {{RTCRtpSender/setParameters()}} 942 | is not supported for the {{RTCRtpReceiver}}, equivalent 943 | experiments cannot be run to determine decoder support. 944 |
945 |946 | Since the {{RTCRtpEncodingParameters/scalabilityMode}} 947 | values supported by software encoders are typically 948 | a superset of those supported in hardware, the information 949 | available from these experiments has a high correlation 950 | with the browser in use, which is already available to 951 | web pages. Once media is flowing, information on 952 | performance characteristics or whether a 953 | {{RTCRtpEncodingParameters/scalabilityMode}} value 954 | is decodable for the codec in use can be obtained, 955 | which provides more information on hardware capabilities. 956 |
957 |958 | As noted in [[?Media-Capabilities]] Section 3.1, the Media 959 | Capabilities API "will likely provide more accurate and 960 | consistent information" than is available from this 961 | specification. As noted in 962 | [[?Media-Capabilities]] Section 3.1, "This information is 963 | expected to have a high correlation with other information 964 | already available to the web pages as a given class of 965 | device is expected to have very similar decoding/encoding 966 | capabilities." 967 |
968 |973 | This section is non-normative; it specifies no new behaviour, but 974 | instead summarizes information already present in other parts of the 975 | specification. WebRTC protocol security considerations are described 976 | in [[RFC8827]] and the security and privacy considerations 977 | for the WebRTC APIs are described in [[WEBRTC]] Section 13. 978 |
979 |983 | Dependency diagrams for the scability modes defined in this specification 984 | are provided below. 985 |
986 |1240 | The editors wish to thank Robin Raymond, Michael Horowitz, Harald Alvestrand, 1241 | Chris Cunningham, Danil Chapovalov, Florent Castelli, Erik Språng and Henrik Boström 1242 | for their contributions to this specification, which evolved from the ORTC API 1243 | developed in the W3C ORTC CG. 1244 |
1245 |