├── .gitignore
├── README.md
├── _config.yml
├── demo-multicast-receive-api.html
└── explainer.md
/.gitignore:
--------------------------------------------------------------------------------
1 | .*.sw?
2 | *~
3 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # MulticastReceiver
2 |
3 | An API that allows web applications to join a multicast (S,G) and receive
4 | authenticated, congestion-safe multicast IP traffic from services that
5 | offer it.
6 |
7 | It provides:
8 |
9 | * per-packet authentication
10 | * an origin-based security model
11 | * protection against over-subscription
12 |
13 | This lets browsers join in subscribing to popular live media events or
14 | file downloads (software or pre-recorded media) that make use of multicast
15 | IP to enable the efficient use of network and server resources.
16 |
17 | See the [explainer](explainer.md) for more info.
18 |
19 | Visit the [demo page](https://grumpyoldtroll.github.io/wicg-multicast-receiver-api/demo-multicast-receive-api.html) with a [browser that implements the API](https://github.com/GrumpyOldTroll/chromium_fork) to experiment with trying to receive traffic. (You may need some extra steps to receive external traffic, see the [getting started](https://github.com/w3c/multicast-cg/blob/main/primers/01-getting-started.md) primer for help, and please open issues if you encounter trouble.)
20 |
21 |
--------------------------------------------------------------------------------
/_config.yml:
--------------------------------------------------------------------------------
1 | title: Multicast Receive API Proposal
2 | description: proposal to add a generic multicast receive API to web browsers.
3 | remote_theme: pmarsceill/just-the-docs
4 | color_scheme: dark
5 |
--------------------------------------------------------------------------------
/demo-multicast-receive-api.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 | MulticastReceive API Demo
10 |
11 |
12 |
13 |
14 |
15 |
65 |
66 |
67 |
68 |
69 |
70 |
71 |
314 |
315 |
316 |
317 |
--------------------------------------------------------------------------------
/explainer.md:
--------------------------------------------------------------------------------
1 | # MulticastReceiver Explainer
2 |
3 | ## Problem and Motivation
4 |
5 | The capacity of CDNs and others to deliver popular content at
6 | scale is not keeping up with demand.
7 |
8 | ### Scaling Problem Examples
9 |
10 | A few different presentations have addressed the scaling problem that can be solved by multiast, but we'll refer here to some real-world numbers presented at NANOG 79:
11 |
12 | * [video](https://www.youtube.com/watch?v=2aihLUb1elg&t=4m56s)
13 | * [slides](https://storage.googleapis.com/site-media-prod/meetings/NANOG79/2209/20200530_Holland_Ip_Multicast_Next_v1.pdf#page=5)
14 |
15 | NB: The presentation used [mistaken numbers](https://github.com/GrumpyOldTroll/wicg-multicast-receiver-api/issues/2) for video bitrate estimates, corrected below, but the overall point still holds. The arithmetic is simple.
16 |
17 | * `167 tbps`: new record peak traffic delivery to end users, announced [April 2020](https://news.mit.edu/2020/3-questions-tom-leighton-managing-covid-19-internet-traffic-surge-0427) by Akamai.
18 |
19 | #### Linear Media Delivery
20 |
21 | Using these values:
22 |
23 | * **20 mbps**: bit-rate for 4k video
24 | * **5 mbps**: bit-rate for 1080p video
25 |
26 | Example reachable audience sizes:
27 |
28 | * **8.35m users** = 167 tbps / (20mbps / end user)
29 | * **33.5m users** = 167 tbps / (5mbps / end user)
30 |
31 | For comparison, audience sizes of popular events:
32 |
33 | Overall viewership:
34 |
35 | * 500m = Fifa World Cup Finals, 2018
36 | * 200m-300m = Cricket World Cup 2015 (whenever India is playing)
37 | * 100m = Super Bowl 2019
38 |
39 | Online Viewership:
40 |
41 | * 6m viewers = Twitch concurrent viewers [all time peak](https://twitchtracker.com/statistics) in June 2020 (as of December 2020)
42 | * 9m viewers = 2018 World Cup [peak concurrent online viewers](https://www.conviva.com/peak-concurrent-plays-broke-records-world-cup-finals/)
43 |
44 | What *is* achievable in 4k as a maximum upper bound?:
45 |
46 | * Most popular 2017-2018 Nielsen show with \<8.3m viewers is ranked **#43** ("Scorpion")
47 |
48 | When one of the largest providers on the Internet can only handle the
49 | 43rd-most popular TV show with no extra capacity before setting a
50 | new record for traffic because of the Internet's unicast delivery model,
51 | the Internet is really not living up to its potential.
52 |
53 | #### Downloads
54 |
55 | Consider a 2GB OS upgrade that has to be delivered to 1b devices. What's
56 | the average delivery time at a high-end peak delivery rate?
57 |
58 | * **1.1 days** = 2GB * 1b devices / 167 Tbps
59 |
60 | Consider also that some OS or app updates are over **10GB**, and therefore
61 | would take over **11 days** to deliver to 1b users, at a record-setting
62 | aggregate delivery rate, doing nothing else. (Sure hope there's no urgent
63 | security updates!)
64 |
65 | ### Conclusion
66 |
67 | We think these problems of scale are addressable with IP Multicast.
68 |
69 | If web pages were able to subscribe to this kind of popular content and
70 | have traffic replicated to many end users as part of delivery through the
71 | network, it would have a transformative effect on the capability of the
72 | Internet to handle these scaling issues, and would provide dramatic network
73 | efficiency gains, even for much less popular content than the extreme
74 | cases.
75 |
76 | (The extreme cases simply can't be done satisfactorily on the
77 | Internet by any means today, regardless of the real-world need to do
78 | them sometimes.)
79 |
80 | ## Goals
81 |
82 | * Provide a way to subscribe to one-to-many multicast streams.
83 |
84 | * Provide an API that can be used by many use cases. There are lots
85 | of protocols, some of them proprietary. As long as they can live
86 | within the safety rules, they're all welcome.
87 |
88 | * Authenticate the traffic cryptographically. An app can only receive
89 | data from a packet with cryptographic proof that it was sent by the
90 | correct sender, with a server-controlled origin policy.
91 |
92 | * Ensure network safety. Using the API cannot blow out the network
93 | by signing up for too much traffic, because that will get noticed
94 | and the group will be pruned at the browser (and also probably in
95 | the network, if it succeeds at the browser).
96 |
97 | ## Non-goals
98 |
99 | * Sending. This API does not do any outbound data traffic. The functions in this API invoke a few outbound packets with IGMP or MLD, plus some other signaling protocols defined in the IETF's RFC series, but provides no capability to create app-controllable outbound traffic, outside of the specific narrow signaling to join multicast groups and process their traffic.
100 |
101 | * ASM (Any-Source Multicast). This proposal only permits use of
102 | [SSM](https://tools.ietf.org/html/rfc4607) (Source Specific Multicast).
103 | (NB: [RFC 8815](https://tools.ietf.org/html/rfc8815) deprecated interdomain multicast using ASM)
104 |
105 | * Privacy and Secrecy within your local network.
106 |
107 | As a receiver, your IGMP or MLD membership report goes on the LAN, to a local
108 | network that can see your machine's IP and MAC address.
109 |
110 | The network is replicating the packets, and they know the global IP
111 | for source and group. If they want to, they can probably find out
112 | what content it contains.
113 |
114 | This API is mainly for content that's so popular it's expected that
115 | lots of people will consume it, but if you can't let your local
116 | network know you're consuming it, you'll have to tunnel it in as
117 | secured unicast from somewhere else that can know. It's NOT just you
118 | and the sender, and it cannot be, because the network is involved in
119 | replicating the packets. You need to have a trust relationship with
120 | someone downstream of that packet replication, because they can find
121 | out what you're consuming.
122 |
123 | Upstream of your local network it gets more private--they'll only
124 | know that at least one machine inside your network is subscribed,
125 | without knowing who it was. But you can assume your next-hop router
126 | will know that your machine has asked to receive a specific piece of
127 | content.
128 |
129 | * Encrypting the data.
130 |
131 | Although the sender MAY encrypt the data, it is not required,
132 | nor is it effective at providing privacy or secrecy in the intended
133 | use case of a scalable one-to-many distribution. The contents of the
134 | packets are sent to many receivers, and some of them will not be
135 | trustworthy. Any key that allows a receiver to decrypt it will also be
136 | available to the receiver's adversaries, perhaps because they are a
137 | legitimate customer to the sender and paid for it.
138 |
139 | That said, the data could for example be built out of encrypted
140 | segments, and the receiver could construct those segments and feed them
141 | into a player that provides its own guarantees about key control, as many existing DRM systems do.
142 |
143 | Nothing prevents the data from being encrypted, it is entirely possible and likely for some use cases.
144 | But nothing in this architecture provides or requires encryption for the transported data.
145 | This spec provides authenticated UDP payloads to the web app, and a different layer would have to understand the content of those payloads (including whether they hold encrypted data).
146 |
147 | ## Proposed Solution
148 |
149 | ### Overview
150 |
151 | The API subscribes to a Source Specific Multicast (S,G) with a UDP port
152 | number and a domain name for the authentication metadata. After that,
153 | it receives UDP payloads if the join is successful (and may of course
154 | leave).
155 |
156 | If the loss rate is high enough or if the total bandwidth would go above
157 | a system-managed threshold, the join will fail with an error.
158 |
159 | If the loss rate is above a safety threshold but below an emergency
160 | disconnect threshold, there's a signal and grace period for the application
161 | to gracefully transition to a state that maintains a lower loss level.
162 |
163 | ### Specifications
164 |
165 | This API will build upon an ongoing set of efforts within the IETF. The
166 | basic idea is to provide a standardized and secure method for the browser
167 | to discover and process metadata about the traffic streams, which it can
168 | use to ensure safe operation. (The network MAY also use the same
169 | metadata to ensure network safety.)
170 |
171 | * DORMS (Discovery Of RESTCONF Metadata for SSM):
172 |
173 | [draft-ietf-mboned-dorms](https://datatracker.ietf.org/doc/draft-ietf-mboned-dorms/)
174 |
175 | This defines a mechanism for the metadata to get discovered and
176 | delivered to the browser and the network. (It uses
177 | [RESTCONF](https://tools.ietf.org/html/rfc8040) and a
178 | [YANG](https://tools.ietf.org/html/rfc7950) model.) The other 2
179 | (AMBI and CBACC, below), extend that data with the info they need to
180 | operate.
181 |
182 | * AMBI (Asymmetric Manifest-Based Integrity):
183 |
184 | [draft-ietf-mboned-ambi](https://datatracker.ietf.org/doc/draft-ietf-mboned-ambi/)
185 |
186 | This is how the browser verifies the integrity of the data received
187 | and gets the information to monitor packet loss, without needing to
188 | understand the payload of the protocol being delivered. It sends
189 | packet hashes out of band in an authenticated stream, so the
190 | receiver knows whether it received what the sender sent.
191 |
192 | * CBACC (Circuit Breaker Assisted Congestion Control):
193 |
194 | [draft-ietf-mboned-cbacc](https://datatracker.ietf.org/doc/draft-ietf-mboned-cbacc/)
195 |
196 | This is how the browser (and the network) ensure that capacity
197 | limits are not exceeded. This defines the behavior both for a
198 | policy-based limit, and for responding to loss by using the
199 | loss detection provided by AMBI. It notices and cuts off
200 | multicast streams whenever someone has over-subscribed.
201 |
202 | Note: other relevant extensions may be forthcoming. In particular, some
203 | work is ongoing to define a transport for AMBI that can live inside the
204 | same multicast channel as the data using
205 | [ALTA](https://datatracker.ietf.org/doc/draft-krose-mboned-alta/) to
206 | authenticate the packet manifests. (We hope this will make for
207 | some improvements in achievable latency targets.)
208 | However, the components listed here are intended to be sufficient to
209 | provide a safe core architecture, and to be adequate on their own for
210 | solutions to scalability problems where network support for multicast is
211 | enabled, and for use cases where the achievable latency is acceptable.
212 |
213 | ### JavaScript API
214 |
215 | Here is an example that joins a multicast flow for 20 seconds.
216 |
217 | ```javascript
218 | // Multicast flow to join:
219 |
220 | let multicastFlow = {
221 | source: '198.51.100.10',
222 | group: '232.10.10.1',
223 | port: 5001,
224 | dorms: 'dorms.example.com'
225 | };
226 |
227 | // Construct MulticastReceiver and subscribe to the multicast flow on the
228 | // network:
229 |
230 | let multicastReceiver=new MulticastReceiver(multicastFlow);
231 |
232 | // Read multicast UDP packets:
233 |
234 | let multicastReader=multicastReceiver.readable.getReader();
235 |
236 | async function readMulticastData() {
237 | for(;;) {
238 | let { done, value } = await multicastReader.read();
239 | if(done) {
240 | return;
241 | } else {
242 | // value is an UInt8Array with the payload of one UDP packet.
243 | console.log("Got multicast packet with size "+value.length);
244 | }
245 | }
246 | }
247 |
248 | readMulticastData().then( () => {
249 | console.log("Cancel was called.");
250 | }).catch( error => {
251 | console.log(error,"Error. Closecode is "+multicastReader.closecode);
252 | });
253 |
254 | // Cancel after 20 seconds:
255 |
256 | setTimeout( () => {
257 | console.log("Canceling multicast");
258 | multicastReader.cancel();
259 | },20000);
260 | ```
261 |
262 | The browser will try to join the multicast flow on the network as soon as MulticastReceiver is constructed. The browser will leave the multicast flow on the network whenever there is an event that will trigger either a) a promise returned by `read()` to be rejected or b) the `done` value to become true. After such an event, no more multicast data can be read from that MulticastReceiver. So if JavaScript wants to receive the same or another multicast flow later, it will need to construct a new MulticastReceiver.
263 |
264 | In case the promise returned by `read()` is rejected, it will be rejected with an informative text which may be useful for debugging and which is logged to the console in the example above. In addition to this, the closecode will be set to one of several well-known values such as:
265 |
266 | * `MulticastReceiver.FLOW_PROBLEM`: There was a problem with the multicast flow itself. For example:
267 | * DORMS could not identify a multicast flow on the given source and group.
268 | * The multicast flow had a higher bandwidth than announced by DORMS.
269 | * Multicast packets were received but could not be authenticated.
270 | * `MulticastReceiver.LOCAL_PROBLEM`: There was a problem on the local device that prevented reception of the multicast flow. For example:
271 | * A socket to receive multicast on could not be created or was closed.
272 | * `MulticastReceiver.OVERSUBSCRIBED`: A CBACC oversubscription threshold (see below) was reached so the left the multicast flow or did not join it at all.
273 |
274 | ### Getting DORMS metadata
275 |
276 | Consider the subscription to the multicast flow from the example in the previous section:
277 |
278 | ```javascript
279 | {
280 | source: '198.51.100.10',
281 | group: '232.10.10.1',
282 | port: 5001,
283 | dorms: 'dorms.example.com'
284 | };
285 | ```
286 |
287 | Here `dorms.example.com` is a domain name and an optional port (default is 443) where the browser will try to get metadata for the multicast flow with HTTPS as described in the DORMS specification. Among other things, this metadata must include the trust anchor used by AMBI to perform integrity checks on the multicast flow. When downloading the metadata, the browser will verify that the server presents a valid certificate for `dorms.example.com`. This ensures that a chain of trust is established from the JavaScript that subscribes to a multicast flow and to the integrity checks the browser performs using AMBI.
288 |
289 | We also want to ensure that the browser only subscribes to a multicast flow which wants to be subscribed to by a browser. This is to prevent for example:
290 |
291 | * A malicious website to subscribe to a multicast flow private to a network.
292 | * A pirate website to subscribe to a multicast flow from a normal website.
293 |
294 | There are two mechanisms in place to prevent this:
295 |
296 | * When downloading DORMS metadata with HTTPS, the browser will perform [CORS](https://fetch.spec.whatwg.org/#http-cors-protocol) checks.
297 | * The DORMS server provided by JavaScript needs to be the same identified by the DORMS reverse-DNS scheme described in the DORMS specification (see next paragraph).
298 |
299 | The second bullet ensures that JavaScript can only subscribe to a multicast flow from a given source IP if the reverse DNS of that source IP has been setup to allow it. More precisely, when receiving the multicast subscription in our example, the browser will make a reverse-DNS lookup for a `SRV` record for the domain `_dorms._tcp.10.100.51.198.in-addr.arpa`. In order for the subscription to succeed, this must give a `SRV` record with the hostname `dorms.example.com` and port 443. A different port can be used if provided in the JavaScript API as well. Obviously, DNS is usually not considered secure, so in some cases it may be possible to hack DNS to circumvent this check. This problem may need further considerations.
300 |
301 | ### TBDs
302 |
303 | TBD: In the current API, one packet is delivered at a time to JavaScript. This is similar to how datagrams work in [WebTransport](https://github.com/WICG/web-transport). We may want to deliver multiple packets at a time for performance reasons. We are currently working on profiling this in Chromium.
304 |
305 | TBD: It would be nice if JavaScript had access to precise timestamps for when packets are received by the OS or at least by the browser from the OS. This could allow JavaScript to implement protocols for detecting the amount of available bandwidth using mechanisms such as [pathchirp](http://www.spin.rice.edu/Software/pathChirp/) and/or [pathrate](https://www.cc.gatech.edu/~dovrolis/bw-est/pathrate.html). Do we want to support this? And if we do, what should the API look like? It seems like an extra field in the object returned by `read()` would be an obvious place. But this may not be compatible with the ReadableStream system. So we may need to wrap the UInt8Array with the packet content in an extra object?
306 |
307 | TBD: Define the CBACC oversubscription threshold model for the browser.
308 | The rough initial idea is to start with a threshold at ~1mbps, grow if it's
309 | succeeding without loss, and shrink if loss goes persistently high.
310 |
311 | TBD: add api extensions for loss rate reporting ahead of cbacc+ambi-based cutoff. need some kind of early warning from stats.
312 |
313 | TBD: We may want to extend the API such that the browser can inform JavaScript that a CBACC oversubscription threshold is about to be reached. This may allow JavaScript to decide which multicast flows to unsubscribe from rather than relying on the browser to select one or more flows and close them with an `MulticastReceiver.OVERSUBSCRIBED` closecode.
314 |
315 | TBD: If JavaScript is not reading any packets, should the browser close the multicast flow with a specific closecode? Related to this, should there be an API where JavaScript can set the maximum number of packets that can be queued up internally in the ReadableStream?
316 |
317 | TBD: Should the browser unsubscribe automatically from a multicast flow on which no data is received from the network? Most likely the answer is *no* as subscription to a multicast flow without data can be used to signal to the network that it may consider creating the flow.
318 |
319 | TBD: AMBI may give information about dropped packets. Do we want that information to be available in the API? Perhaps by the reading of a special packet that indicates one or more dropped packets.
320 |
321 | TBD: Detailed design. This early draft is intended to solicit community
322 | feedback about whether this general approach is doomed for reasons not
323 | addressed in the IETF drafts, or whether attempting an implementation and
324 | filling out all the details will actually be deployable, once it all works.
325 |
326 | ## Alternative designs considered
327 |
328 | ### Specific Protocols
329 |
330 | It's reasonable to question why we wouldn't solve the scaling problem by
331 | defining an API for a specific transport protocol, such as
332 | [FLUTE](https://tools.ietf.org/html/rfc6726) or
333 | [NORM](https://tools.ietf.org/html/rfc5740),
334 | or by adding support for [RTP](https://tools.ietf.org/html/rfc3550) with [RAMS](https://tools.ietf.org/html/rfc6285) or other
335 | [multicast-enabling features](https://tools.ietf.org/html/rfc5760).
336 |
337 | Part of the problem is that many existing applications (such as IPTV
338 | systems currently deployed by some ISPs) don't use these protocols,
339 | and we hope not to exclude their use cases, even though some of the
340 | deployments use proprietary protocols with no publicly available
341 | implementations.
342 |
343 | Multicast IP also is an area of ongoing research with new protocols
344 | still being developed, for example
345 | [draft-pardue-quic-http-mcast](https://datatracker.ietf.org/doc/draft-pardue-quic-http-mcast/). We believe enabling this kind of experimentation
346 | with web apps will be extremely valuable.
347 |
348 | Additionally, a number of gaming or communication protocols exist
349 | that are built directly on top of unreliable UDP transport, and are
350 | specific to the application itself. These applications are also often
351 | proprietary or least non-standard (but still sometimes deployed and
352 | useful and could benefit from scalability).
353 |
354 | The intent of the proposed architecture is to provide all the necessary
355 | guard rails to ensure safety for the network and receiver so that
356 | interdomain multicast can be delivered approximately as safely as other
357 | kinds of traffic, while permitting experimentation that can realize the
358 | untapped potential for a wide variety of uni-directional one-to-many
359 | protocol designs.
360 |
361 | ### WebTransport
362 |
363 | It has been suggested that this might fit as a unidirectional protocol
364 | definition within [WebTransport](https://github.com/WICG/web-transport).
365 |
366 | As it stands today, this proposal is not compatible with the WebTransport
367 | goal of "Ensure the same security properties as WebSockets", because the
368 | one-to-many nature of this proposal makes it impossible to provide the
369 | same security guarantees as TLS.
370 |
371 | After discussion with some of the WebTransport contributors, the advice
372 | was to open this as an independent proposal.
373 |
374 | If there's enough consensus that it's worthwhile, perhaps it's possible
375 | to make the goals compatible in some way and to fold this proposal into
376 | a subset of WebTransport. But as of today the idea has been (very
377 | tentatively) considered, and sits somewhere between 'rejected' and
378 | 'deferred'.
379 |
380 |
--------------------------------------------------------------------------------