├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── LICENSE.md
├── README.md
├── audio_splice.png
├── byte-stream-format-registry-cr.html
├── byte-stream-format-registry-respec.html
├── byte-stream-format-registry.html
├── cr-heartbeat-2
├── audio_splice.png
├── byte-stream-format-registry.html
├── index.html
├── isobmff-byte-stream-format.html
├── mp2t-byte-stream-format.html
├── mpeg-audio-byte-stream-format.html
├── mse.css
├── pipeline_model.png
└── webm-byte-stream-format.html
├── index.html
├── isobmff-byte-stream-format-cr.html
├── isobmff-byte-stream-format-respec.html
├── isobmff-byte-stream-format.html
├── media-source-cr.html
├── media-source-fpwd.html
├── media-source-lc.html
├── media-source-pr.html
├── media-source-respec.html
├── media-source-wd.html
├── media-source.html
├── media-source.js
├── mp2t-byte-stream-format-respec.html
├── mp2t-byte-stream-format.html
├── mpeg-audio-byte-stream-format-cr.html
├── mpeg-audio-byte-stream-format-respec.html
├── mpeg-audio-byte-stream-format.html
├── mse-1-errata.html
├── mse.css
├── pipeline_model.svg
├── pipeline_model_description.html
├── respec-w3c-common.js
├── w3c.json
├── webm-byte-stream-format-respec.html
└── webm-byte-stream-format.html
/CODE_OF_CONDUCT.md:
--------------------------------------------------------------------------------
1 | # Code of Conduct
2 |
3 | All documentation, code and communication under this repository are covered by the [W3C Code of Ethics and Professional Conduct](https://www.w3.org/Consortium/cepc/).
4 |
--------------------------------------------------------------------------------
/CONTRIBUTING.md:
--------------------------------------------------------------------------------
1 | # HTML Working Group
2 |
3 | Contributions to this repository are intended to become part of Recommendation-track documents
4 | governed by the [W3C Patent Policy](http://www.w3.org/Consortium/Patent-Policy-20040205/) and
5 | [Document License](http://www.w3.org/Consortium/Legal/copyright-documents). To contribute, you must
6 | either participate in the relevant W3C Working Group or make a non-member patent licensing
7 | commitment.
8 |
9 | If you are not the sole contributor to a contribution (pull request), please identify all
10 | contributors in the pull request's body or in subsequent comments.
11 |
12 | To add a contributor (other than yourself, that's automatic), mark them one per line as follows:
13 |
14 | ```
15 | +@github_username
16 | ```
17 |
18 | If you added a contributor by mistake, you can remove them in a comment with:
19 |
20 | ```
21 | -@github_username
22 | ```
23 |
24 | If you are making a pull request on behalf of someone else but you had no part in designing the
25 | feature, you can remove yourself with the above syntax.
26 |
--------------------------------------------------------------------------------
/LICENSE.md:
--------------------------------------------------------------------------------
1 | All documents in this Repository are licensed by contributors under the [W3C Document
2 | License](http://www.w3.org/Consortium/Legal/copyright-documents).
3 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Incubation of vNext 'media-source' Specification Features
2 |
3 |
4 | # May 2021: See instead the upstream w3c media-source repository now for new feature incubations (this WICG repo was useful before MSE v2 work upstream began; current work is upstream). Moving incubated feature explainers, etc into the main spec is being done there currently (work-in-progress).
5 |
6 |
7 | This is [the repository forked off the W3C REC repository for
8 | media-source](https://github.com/w3c/media-source), and is intended to track
9 | incubation of new media-source features.
10 |
11 | You're welcome to contribute! Let's continue to make the Web rock our socks off!
12 |
13 | The admins will update this repository's gh-pages branch to align with
14 | the upstream w3c repository.
15 |
16 | **All** specification updates that are part of incubation will be done on a
17 | per-feature branch off of gh-pages in this repository. Please base your
18 | pull-requests accordingly.
19 |
20 | We will use the [upstream w3c/media-source repository's issue tracker](https://github.com/w3c/media-source/issues)
21 | for feature discussion beyond pull-requests.
22 |
23 | The following is the list of current feature incubation branches:
24 |
25 | * [codec-switching](https://github.com/WICG/media-source/tree/codec-switching)
26 | * Tracking issue:
27 | [w3c/media-source/issues/155](https://github.com/w3c/media-source/issues/155)
28 | * [Feature
29 | explainer](https://github.com/wicg/media-source/blob/codec-switching/codec-switching-explainer.md)
30 | * [HTML Diff versus REC
31 | spec](https://services.w3.org/htmldiff?doc1=https%3A%2F%2Fwww.w3.org%2FTR%2Fmedia-source%2F&doc2=https%3A%2F%2Frawgit.com%2FWICG%2Fmedia-source%2Fcodec-switching%2Findex.html)
32 | * [HTML Diff versus upstream W3C Editor's
33 | Draft](https://services.w3.org/htmldiff?doc1=https%3A%2F%2Frawgit.com%2FW3C%2Fmedia-source%2Fgh-pages%2Findex.html&doc2=https%3A%2F%2Frawgit.com%2FWICG%2Fmedia-source%2Fcodec-switching%2Findex.html)
34 |
35 | * [mse-in-workers-using-handle](https://github.com/WICG/media-source/tree/mse-in-workers-using-handle)
36 | * Tracking issue:
37 | [w3c/media-source/issues/175](https://github.com/w3c/media-source/issues/175)
38 | * [Feature
39 | explainer](https://github.com/wicg/media-source/blob/mse-in-workers-using-handle/mse-in-workers-using-handle-explainer.md)
40 |
41 | * [eviction
42 | policies](https://github.com/WICG/media-source/tree/mse-eviction-policies)
43 | * Tracking issue:
44 | [w3c/media-source/issues/232](https://github.com/w3c/media-source/issues/232)
45 | * [Feature
46 | explainer](https://github.com/wicg/media-source/blob/mse-eviction-policies/mse-eviction-policies-explainer.md)
47 |
--------------------------------------------------------------------------------
/audio_splice.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/WICG/media-source/8f8b4205a038a8e02a525841fd83f272077d71ce/audio_splice.png
--------------------------------------------------------------------------------
/byte-stream-format-registry-cr.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 | Media Source Extensions Byte Stream Format Registry
6 |
7 |
8 |
9 |
10 |
11 |
12 |
13 |
14 |
15 |
16 |
17 |
24 |
140 |
141 |
142 |
143 |
144 |
145 |
146 |
Media Source Extensions Byte Stream Format Registry
This registry is intended to enhance interoperability among implementations and users of
200 | SourceBuffer objects described in the
201 | Media Source Extensions(MSE) specification. In particular, this registry provides the means (1) to identify
202 | and avoid MIME-type collisions among byte stream formats, and (2) to disclose information about byte stream formats accepted by MSE
203 | implementations to promote interoperability.
204 |
205 |
206 |
207 |
2. Organization
208 |
The registry maintains a mapping between MIME-type/subtype pairs and byte stream format specifications. The byte stream format specifications describe the
209 | structure and semantics of byte streams accepted by SourceBuffer objects
210 | created with the associated MIME-type/subtype pair.
211 |
This registry is not intended to include any information on whether a byte stream format is encumbered by intellectual property claims. Implementors and users
212 | are advised to seek appropriate legal counsel in this matter if they intend to implement or use a specific byte stream format.
213 |
214 |
215 |
216 |
3. Registration Entry Requirements
217 |
218 |
Each entry must include a unique MIME-type/subtype pair. If the byte stream format is derived-from an existing file format, then it should use the
219 | MIME-type/subtype pairs typically used for the file format.
Each entry must include a link that references a publically available specification. It is recommended that such a specification be made available
223 | without cost (other than reasonable shipping and handling if not available by online means).
Candidate entries must be announced on public-html-media@w3.org(subscribe,
227 | archives) so they can be discussed and evaluated for compliance before being added to the
228 | registry.
229 |
230 |
If a registration fails to satisfy a mandatory requirement specified above, then it may be removed from the registry (without prejudice).
231 |
Implementors should be aware that this specification is not stable. Implementors who are not taking part in the discussions are likely to find the specification changing out from under them in incompatible ways. Vendors interested in implementing this specification before it eventually reaches the Candidate Recommendation stage should join the mailing list mentioned below and take part in the discussions.
119 |
120 |
121 |
122 |
Purpose
123 |
This registry is intended to enhance interoperability among implementations and users of
124 | SourceBuffer objects described in the
125 | (MSE) specification. In particular, this registry provides the means (1) to identify
126 | and avoid MIME-type collisions among byte stream formats, and (2) to disclose information about byte stream formats accepted by MSE
127 | implementations to promote interoperability.
128 |
129 |
130 |
131 |
Organization
132 |
The registry maintains a mapping between MIME-type/subtype pairs and byte stream format specifications. The byte stream format specifications describe the
133 | structure and semantics of byte streams accepted by SourceBuffer objects
134 | created with the associated MIME-type/subtype pair.
135 |
This registry is not intended to include any information on whether a byte stream format is encumbered by intellectual property claims. Implementors and users
136 | are advised to seek appropriate legal counsel in this matter if they intend to implement or use a specific byte stream format.
137 |
138 |
139 |
140 |
Registration Entry Requirements
141 |
142 |
Each entry MUST include a unique MIME-type/subtype pair. If the byte stream format is derived-from an existing file format, then it SHOULD use the
143 | MIME-type/subtype pairs typically used for the file format.
144 |
Each entry MUST include a value that MUST be used by
145 | SourceBuffer when handling the byte stream format.
146 |
Each entry MUST include a link that references a publically available specification. It is recommended that such a specification be made available
147 | without cost (other than reasonable shipping and handling if not available by online means).
148 |
Each entry MUST comply with all requirements outlined in the
149 | of the specification.
150 |
Candidate entries MUST be announced on public-html-media@w3.org(subscribe,
151 | archives) so they can be discussed and evaluated for compliance before being added to the
152 | registry.
153 |
154 |
If a registration fails to satisfy a mandatory requirement specified above, then it MAY be removed from the registry (without prejudice).
155 |
This registry is intended to enhance interoperability among implementations and users of
200 | SourceBuffer objects described in the
201 | Media Source Extensions(MSE) specification. In particular, this registry provides the means (1) to identify
202 | and avoid MIME-type collisions among byte stream formats, and (2) to disclose information about byte stream formats accepted by MSE
203 | implementations to promote interoperability.
204 |
205 |
206 |
207 |
2. Organization
208 |
The registry maintains a mapping between MIME-type/subtype pairs and byte stream format specifications. The byte stream format specifications describe the
209 | structure and semantics of byte streams accepted by SourceBuffer objects
210 | created with the associated MIME-type/subtype pair.
211 |
This registry is not intended to include any information on whether a byte stream format is encumbered by intellectual property claims. Implementors and users
212 | are advised to seek appropriate legal counsel in this matter if they intend to implement or use a specific byte stream format.
213 |
214 |
215 |
216 |
3. Registration Entry Requirements
217 |
218 |
Each entry must include a unique MIME-type/subtype pair. If the byte stream format is derived-from an existing file format, then it should use the
219 | MIME-type/subtype pairs typically used for the file format.
Each entry must include a link that references a publically available specification. It is recommended that such a specification be made available
223 | without cost (other than reasonable shipping and handling if not available by online means).
Candidate entries must be announced on public-html-media@w3.org(subscribe,
227 | archives) so they can be discussed and evaluated for compliance before being added to the
228 | registry.
229 |
230 |
If a registration fails to satisfy a mandatory requirement specified above, then it may be removed from the registry (without prejudice).
231 |
255 | This specification defines a Media Source Extensions byte stream format specification based on MPEG-2 Transport Streams.
256 |
Status of This Document
257 |
258 |
259 |
260 |
261 | This document is merely a W3C-internal document. It
262 | has no official standing of any kind and does not represent consensus of the W3C
263 | Membership.
264 |
This specification defines segment formats for implementations that choose to support MPEG-2 Transport Streams
274 | (MPEG-2 TS) specified in ISO/IEC 13818-1.
It defines the MIME-type parameters used to signal codecs, and provides
275 | the necessary format specific definitions for initialization segments, media segments, and random access points required by
276 | the byte stream formats section of the Media Source Extensions spec. This document also defines extra behaviors and state that only apply to this
277 | byte stream format.
278 |
279 |
280 |
281 |
2. MIME-type info
282 |
The MIME-type/subtype pair of "video/MP2T", as specified in RFC 3551, must be used for byte streams that conform
283 | to this specification.
289 | A comma separated list of codec IDs used to specify what codecs will be used in the byte stream. Implementations should accept IDs in the form specified by
290 | RFC 6381.
291 |
Note
Implementations may only implement a subset of the codecs and profiles mentioned in the RFC.
292 |
293 |
294 |
295 |
296 |
297 |
3. General Transport Stream Requirements
298 |
MPEG-2 TS media and initialization segments must conform to the MPEG-2 TS Adaptive Profile (ISO/IEC 13818-1:2012 Amd. 2).
299 |
300 |
The user agent must run the append error algorithm with the decode error parameter set to true if any of the following conditions are met:
301 |
302 |
Segments do not contain complete MPEG-2 TS packets.
303 |
Segments do not contain complete PES packets and sections.
304 |
Segments contain more than one program.
305 |
At least one MPEG-2 TS packet has a non-zero transport_error_indicator.
306 |
307 |
308 |
309 |
4. Initialization Segments
310 |
An MPEG-2 TS initialization segment consists of a single PAT and a single PMT.
311 |
The user agent must run the append error algorithm with the decode error parameter set to true if any of the following conditions are met:
The user agent must accept and ignore other SI, such as CAT, that are invariant for all subsequent media segments.
318 |
The user agent must source attribute values for id, kind, label and language for AudioTrack, VideoTrack and
319 | TextTrack objects as described for MPEG-2 Transport Streams in the in-band tracks spec [INBANDTRACKS].
320 |
321 |
322 |
323 |
5. Media Segments
324 |
325 |
The user agent must run the append error algorithm with the decode error parameter set to true if any of the following conditions are met:
326 |
327 |
A media segment relies on initialization information in another media segment.
328 |
At least one PES packet does not have a PTS timestamp.
329 |
PCR is not present in the Segment prior to the first byte of a TS packet payload containing media data.
330 |
331 |
332 | The user agent will accept and ignore PSI that is identical to the information in the last initialization segment which may appear repeatedly throughout the segment.
333 |
334 |
335 |
336 |
6. Random Access Points
337 |
A random access point as defined in this specification corresponds to Elementary Stream Random Access Point as defined in
338 | ISO/IEC 13818-1.
339 |
340 |
341 |
7. Timestamp Rollover & Discontinuities
342 |
Timestamp rollovers and discontinuities must be handled by the UA. The UA's MPEG-2 TS implementation must maintain an internal offset
343 | variable, MPEG2TS_timestampOffset, to keep track of the offset that needs to be applied to timestamps
344 | that have rolled over or are part of a discontinuity. MPEG2TS_timestampOffset is initially set to 0 when the SourceBuffer is
345 | created. This offset must be applied to the timestamps as part of the conversion process from MPEG-2 TS packets
346 | into coded frames for the coded frame processing algorithm. This results in the coded frame timestamps
347 | for a packet being computed by the following equations:
When a discontinuity is detected, MPEG2TS_timestampOffset must be adjusted to make the timestamps after the discontinuity appear
355 | to come immediately after the timestamps before the discontinuity.
363 | The editors would like to thank Alex Giladi, Bob Lund, David Singer, Duncan Rowden, Glenn Adams, Mark Vickers, and Michael Thornburgh for their contributions to this specification.
364 |
365 |
366 |
367 |
252 | This specification defines a Media Source Extensions byte stream format specification based on MPEG audio streams.
253 |
Status of This Document
254 |
255 |
256 |
257 |
258 | This document is merely a W3C-internal document. It
259 | has no official standing of any kind and does not represent consensus of the W3C
260 | Membership.
261 |
This section specifies the MIME-types that may be passed to isTypeSupported() or addSourceBuffer() for byte streams that conform to this specification.
"audio/mpeg" for MPEG-1/2/2.5 Layer I/II/III streams, as specified in RFC 3003.
280 |
281 |
The "codecs" MIME-type parameter must not be used with these MIME-types.
282 |
283 |
284 |
285 |
3. MPEG Audio Frames
286 |
The format of an MPEG Audio Frame depends on the MIME-type used.
287 |
288 |
If the "audio/aac" MIME-type is used, an MPEG Audio Frame is a sequence of bytes that conform to the adts_frame() syntax specified in Table 1.A.5 of ISO/IEC 14496-3:2009.
289 |
If the "audio/mpeg" MIME-type is used, an MPEG Audio Frame is a sequence of bytes that conform to the frame() syntax element specified in Section 2.4.1.2 of ISO/IEC 11172-3:1993 or the corresponding definition in ISO/IEC 13818-3:1998.
290 |
291 |
292 |
293 |
294 |
4. Metadata Frames
295 |
Since ID3v1, ID3v2 metadata frames, and Icecast headers are common in existing MPEG audio streams, implementations should gracefully handle such frames. Zero or more of these metadata frames are allowed to occur before, after, or between MPEG Audio Frames. Minimal implementations must accept, consume, and ignore these frames. More advanced implementations may choose to expose the metadata information via an inband TextTrack or some other mechanism.
296 |
297 |
298 |
4.1 Icecast headers
299 |
There is no normative spec for Icecast/SHOUTcast headers, just examples. For the purpose of this specification, an Icecast header is defined as beginning with the 4 character sequence "ICY "(U+0049 I, U+0043 C, U+0059 Y, U+0020 SPACE) and ending with a pair of carriage-return line-feed sequences (U+000D CARRIAGE RETURN, U+000A LINE FEED, U+000D CARRIAGE RETURN, U+000A LINE FEED).
300 |
Note
Icecast headers are allowed in the byte streams because some Icecast and SHOUTcast
301 | servers return a status line that looks like "ICY OK 200" instead of a standard HTTP status line.
302 | User-agent network stacks typically interpret this as an HTTP 0.9 response and include the
303 | header in the response body. Allowing these headers to appear provides a simple way to
304 | interoperate with these servers.
248 | This specification defines a Media Source Extensions byte stream format specification based on the WebM container format.
249 |
Status of This Document
250 |
251 |
252 |
253 |
254 | This document is merely a W3C-internal document. It
255 | has no official standing of any kind and does not represent consensus of the W3C
256 | Membership.
257 |
This specification describes a byte stream format based on the WebM container format. It defines the MIME-type parameters used to signal codecs, and provides
267 | the necessary format specific definitions for initialization segments, media segments, and random access points required by
268 | the byte stream formats section of the Media Source Extensions spec.
The size value in the Segment header must signal an "unknown size" or contain a value large enough to include the Segment Information and Tracks elements that follow.
The user agent must accept and ignore any elements other than an EBML Header or a Cluster that occur before, in between, or after the Segment Information and
337 | Tracks elements.
338 |
339 |
The user agent must source attribute values for id, kind, label and language for AudioTrack, VideoTrack and
340 | TextTrack objects as described for WebM in the in-band tracks spec [INBANDTRACKS].
The Cluster header may contain an "unknown" size value. If it does then the end of the cluster is reached when another Cluster header or an element header that indicates the start
352 | of an WebM initialization segment is encountered.
353 |
354 |
355 |
The user agent must run the append error algorithm with the decode error parameter set to true if any of the following conditions are not met:
356 |
357 |
The Timecode element must appear before any Block & SimpleBlock elements in a Cluster.
358 |
Block & SimpleBlock elements are in time increasing order consistent with the WebM spec.
359 |
If the most recent WebM initialization segment describes multiple tracks, then blocks from all the tracks must be interleaved in time increasing order. At least one block from all audio and video
360 | tracks must be present.
361 |
362 |
363 |
The user agent must accept and ignore Cues or Chapters elements that follow a Cluster element.
364 |
365 |
366 |
367 |
5. Random Access Points
368 |
A SimpleBlock element with its Keyframe flag set signals the location of a random access point for that track. Media segments containing multiple tracks are only considered a random access
369 | point if the first SimpleBlock for each track has its Keyframe flag set. The order of the multiplexed blocks must conform to the WebM Muxer Guidelines.
370 |
371 |
372 |
373 |
6. Acknowledgments
374 | The editors would like to thank Frank Galligan, and Philip Jägenstedt for their contributions to this specification.
375 |
376 |
377 |
378 |
239 | This specification defines a Media Source Extensions byte stream format specification based on the ISO Base Media
240 | File Format.
241 |
Status of This Document
242 |
243 |
244 |
245 |
246 | This document is merely a W3C-internal document. It
247 | has no official standing of any kind and does not represent consensus of the W3C
248 | Membership.
249 |
This specification defines segment formats for implementations that choose to support the ISO Base Media File Format
259 | ISO/IEC 14496-12 (ISO BMFF).
MIME-types for this specification must conform to the rules outlined for "audio/mp4" and "video/mp4" in RFC 6381.
268 |
269 |
Note
Implementations may only implement a subset of the codecs and profiles mentioned in the RFC.
270 |
271 |
272 |
273 |
3. Initialization Segments
274 |
An ISO BMFF initialization segment is defined in this specification as a single File Type Box (ftyp) followed by a single Movie Header Box (moov).
275 |
276 |
The user agent must run the end of stream algorithm with the error parameter set to "decode" if any of the following conditions are met:
277 |
278 |
A File Type Box contains a major_brand or compatible_brand that the user agent does not support.
279 |
A box or field in the Movie Header Box is encountered that violates the requirements mandated
280 | by the major_brand or one of the
281 | compatible_brands in the File Type Box.
282 |
The tracks in the Movie Header Box contain samples (i.e. the entry_count in the
283 | stts, stsc or stco boxes are not set to zero).
284 |
A Movie Extends (mvex) box is not contained in the Movie (moov) box to indicate that Movie Fragments are to be expected.
285 |
286 |
The user agent must support setting the offset from media composition time to movie presentation time by handling an Edit Box (edts) containing a single Edit List Box
287 | (elst) that contains a single edit with media rate one. This edit may have a duration of 0 (indicating that it spans all subsequent media) or may have a non-zero duration
288 | (indicating the total duration of the movie including fragments).
289 |
290 |
The user agent must support parameter sets (e.g., PPS/SPS) stored in the sample entry (as defined for avc1/avc2), and should support parameter
291 | sets stored inband in the samples themselves (as defined for avc3/avc4).
292 |
Note
For maximum content interoperability, user agents are strongly advised to support both inband and out-of-band storage of the SPS and PPS.
293 |
294 |
Valid top-level boxes such as pdin, free, and sidx are
295 | allowed to appear before the moov box. These boxes must be accepted and ignored by the user agent and are not
296 | considered part of the initialization segment in this specification.
297 |
298 |
299 |
300 |
4. Media Segments
301 |
An ISO BMFF media segment is defined in this specification as one optional
302 | Segment Type Box (styp) followed by a single Movie Fragment Box
303 | (moof) followed by one or more Media Data Boxes (mdat).
304 | If the Segment Type Box is not present, the segment must conform to the brands listed in the
305 | File Type Box (ftyp) in the initialization segment.
306 |
Valid top-level boxes defined in ISO/IEC 14496-12 other than ftyp,
307 | moov, styp, moof, and mdat are allowed to appear between the end of an
308 | initialization segment or media segment and before the beginning of a new media segment.
309 | These boxes must be accepted and ignored by the user agent and are not considered part of the media segment in this
310 | specification.
311 |
312 |
313 |
The user agent must run the end of stream algorithm with the error parameter set to "decode" if any of the following conditions are met:
314 |
315 |
A box or field in the Movie Fragment Box is encountered that violates the requirements mandated
316 | by the major_brand or one of the
317 | compatible_brands in the Segment Type Box in this
318 | media segment or the File Type Box in the initialization segment if a
319 | Segment Type Box is not present.
At least one Track Fragment Box does not contain a Track Fragment Decode Time Box (tfdt)
326 |
The Media Data Boxes do not contain all the samples referenced by the Track Fragment Run Boxes (trun) of the Movie Fragment Box.
327 |
Inband parameter sets are not present in the appropriate samples and parameter sets are not present in the last initialization segment appended.
328 |
329 |
330 |
A Movie Fragment Box uses movie-fragment relative addressing
331 | when the first Track Fragment Run(trun) box in each Track Fragment Box
332 | has the data-offset-present flag set and either of the following
333 | conditions are met:
334 |
335 |
Every Track Fragment Box in a Movie Fragment Box has the default-base-is-moof flag set.
336 |
Note
This implies that the base-data-offset-present flag
337 | is not set.
338 |
339 |
The Movie Fragment Box contains a single Track Fragment Box and that box does not have the base-data-offset-present flag set.
340 |
341 |
342 |
343 |
344 |
5. Random Access Points
345 |
A random access point as defined in this specification corresponds to a Stream Access Point of type 1 or 2 as defined in Annex I of ISO/IEC 14496-12.
346 |
347 |
348 |
349 |
6. Acknowledgments
350 | The editors would like to thank Chris Poole, Cyril Concolato, David Singer, Jer Noble, Jerry Smith, Joe Steele, John Simmons, Kevin Streeter, Michael Thornburgh, and Steven Robertson for their contributions to this specification.
351 |
352 |
353 |
354 |
Implementors should be aware that this specification is not stable. Implementors who are not taking part in the discussions are likely to find the specification changing out from under them in incompatible ways. Vendors interested in implementing this specification before it eventually reaches the Candidate Recommendation stage should join the mailing list mentioned below and take part in the discussions.
145 |
146 |
147 |
148 |
Introduction
149 |
This specification defines segment formats for implementations that choose to support the ISO Base Media File Format
150 | [[!ISOBMFF]].
It defines the MIME-type parameters used to signal codecs, and provides
151 | the necessary format specific definitions for , , and required by
152 | the of the Media Source Extensions spec.
153 |
154 |
155 |
156 |
MIME-type parameters
157 |
This section specifies the parameters that can be used in the MIME-type passed to or .
158 |
MIME-types for this specification MUST conform to the rules outlined for "audio/mp4" and "video/mp4" in RFC 6381.
159 |
160 |
Implementations MAY only implement a subset of the codecs and profiles mentioned in the RFC.
161 |
162 |
163 |
164 |
Initialization Segments
165 |
An ISO BMFF is defined in this specification as a single File Type Box (ftyp) followed by a single Movie Box (moov).
166 |
167 |
The user agent MUST run the if any of the following conditions are met:
168 |
169 |
A File Type Box contains a major_brand or compatible_brand that the user agent does not support.
170 |
A box or field in the Movie Box is encountered that violates the requirements mandated
171 | by the major_brand or one of the
172 | compatible_brands in the File Type Box.
173 |
The tracks in the Movie Box contain samples (i.e., the entry_count in the
174 | stts, stsc or stco boxes are not set to zero).
175 |
A Movie Extends (mvex) box is not contained in the Movie (moov) box to indicate that Movie Fragments are to be expected.
176 |
177 |
The user agent MUST support setting the offset from media composition time to movie presentation time by handling an Edit Box (edts) containing a single Edit List Box
178 | (elst) that contains a single edit with media rate one. This edit MAY have a duration of 0 (indicating that it spans all subsequent media) or MAY have a non-zero duration
179 | (indicating the total duration of the movie including fragments).
180 |
181 |
The user agent MUST support codec configurations stored out-of-band in the sample entry, and for codecs which allow codec configurations stored inband in the samples themselves, the user agent SHOULD support codec configurations stored inband.
182 |
For example, for codecs which include SPS and PPS parameter sets, for maximum content interoperability, user agents are strongly advised to support both inband (e.g., as defined for avc3/avc4) and out-of-band (e.g., as defined for avc1/2) storage of the SPS and PPS.
183 |
184 |
Valid top-level boxes such as pdin, free, and sidx are
185 | allowed to appear before the moov box. These boxes MUST be accepted and ignored by the user agent and are not
186 | considered part of the in this specification.
187 |
188 |
The user agent MUST source attribute values for id, kind, label and language for , and
189 | objects as described for MPEG-4 ISOBMFF in the in-band tracks spec [[INBANDTRACKS]].
190 |
191 |
192 |
193 |
Media Segments
194 |
An ISO BMFF is defined in this specification as one optional
195 | Segment Type Box (styp) followed by a single Movie Fragment Box
196 | (moof) followed by one or more Media Data Boxes (mdat).
197 | If the Segment Type Box is not present, the segment MUST conform to the brands listed in the
198 | File Type Box (ftyp) in the .
199 |
Valid top-level boxes defined in other than ftyp,
200 | moov, styp, moof, and mdat are allowed to appear between the end of an
201 | or and before the beginning of a new .
202 | These boxes MUST be accepted and ignored by the user agent and are not considered part of the in this
203 | specification.
204 |
205 |
206 |
The user agent MUST run the if any of the following conditions are met:
207 |
208 |
A box or field in the Movie Fragment Box is encountered that violates the requirements mandated
209 | by the major_brand or one of the
210 | compatible_brands in the Segment Type Box in this
211 | or the File Type Box in the if a
212 | Segment Type Box is not present.
213 |
This contains a Segment Type Box that is not compatible with the
214 | File Type Box in the .
215 |
The Movie Fragment Box does not contain at least one Track Fragment Box (traf).
216 |
The Movie Fragment Box does not use .
217 |
External data references are being used.
218 |
At least one Track Fragment Box does not contain a Track Fragment Decode Time Box (tfdt)
219 |
The Media Data Boxes do not contain all the samples referenced by the Track Fragment Run Boxes (trun) of the Movie Fragment Box.
220 |
Inband parameter sets are not present in the appropriate samples and parameter sets are not present in the last initialization segment appended.
221 |
222 |
223 |
A Movie Fragment Box uses movie-fragment relative addressing
224 | when the first Track Fragment Run(trun) box in each Track Fragment Box
225 | has the data-offset-present flag set and either of the following
226 | conditions are met:
227 |
228 |
Every Track Fragment Box in a Movie Fragment Box has the default-base-is-moof flag set.
229 |
This implies that the base-data-offset-present flag
230 | is not set.
231 |
232 |
The Movie Fragment Box contains a single Track Fragment Box and that box does not have the base-data-offset-present flag set.
233 |
234 |
235 |
236 |
237 |
Random Access Points
238 |
A as defined in this specification corresponds to a Stream Access Point of type 1 or 2 as defined in Annex I of .
239 |
240 |
241 |
242 |
243 |
244 |
Acknowledgments
245 | The editors would like to thank for their contributions to this specification.
246 |
247 |
248 |
249 |
250 |
--------------------------------------------------------------------------------
/mp2t-byte-stream-format-respec.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 | MPEG-2 TS Byte Stream Format
6 |
7 |
8 |
9 |
20 |
21 |
119 |
120 |
126 |
127 |
128 |
129 |
130 |
131 | This specification defines a [[!MEDIA-SOURCE]] byte stream format specification based on MPEG-2 Transport Streams.
132 |
133 |
134 |
135 |
Implementors should be aware that this specification is not stable. Implementors who are not taking part in the discussions are likely to find the specification changing out from under them in incompatible ways. Vendors interested in implementing this specification before it eventually reaches the Candidate Recommendation stage should join the mailing list mentioned below and take part in the discussions.
137 |
138 |
139 |
140 |
Introduction
141 |
This specification defines segment formats for implementations that choose to support MPEG-2 Transport Streams
142 | [[!MPEG2TS]].
It defines the MIME-type parameters used to signal codecs, and provides
143 | the necessary format specific definitions for , , and required by
144 | the of the Media Source Extensions spec. This document also defines extra behaviors and state that only apply to this
145 | byte stream format.
146 |
147 |
148 |
149 |
MIME-type info
150 |
The MIME-type/subtype pair of "video/MP2T", as specified in [[!RFC3551]], MUST be used for byte streams that conform
151 | to this specification.
152 |
153 |
This following parameters can be used in the MIME-type passed to or .
154 |
155 |
codecs
156 |
157 | A comma separated list of codec IDs used to specify what codecs will be used in the byte stream. Implementations SHOULD accept IDs in the form specified by [[!RFC6381]].
158 |
Implementations MAY only implement a subset of the codecs and profiles mentioned in the RFC.
159 |
160 |
161 |
162 |
163 |
164 |
General Transport Stream Requirements
165 |
MPEG-2 TS media and initialization segments MUST conform to the MPEG-2 TS Adaptive Profile (ISO/IEC 13818-1:2012 Amd. 2).
166 |
167 |
The user agent MUST run the if any of the following conditions are met:
168 |
169 |
Segments do not contain complete MPEG-2 TS packets.
170 |
Segments do not contain complete PES packets and sections.
171 |
Segments contain more than one program.
172 |
At least one MPEG-2 TS packet has a non-zero transport_error_indicator.
173 |
174 |
175 |
176 |
Initialization Segments
177 |
An MPEG-2 TS initialization segment consists of a single PAT and a single PMT.
178 |
The user agent MUST run the if any of the following conditions are met:
179 |
180 |
A PAT is not present in the
181 |
A PMT is not present in the
182 |
183 |
184 |
The user agent MUST accept and ignore other SI, such as CAT, that are invariant for all subsequent media segments.
185 |
The user agent MUST source attribute values for id, kind, label and language for , and
186 | objects as described for MPEG-2 Transport Streams in the in-band tracks spec [[INBANDTRACKS]].
187 |
188 |
189 |
190 |
Media Segments
191 |
192 |
The user agent MUST run the if any of the following conditions are met:
193 |
194 |
A media segment relies on initialization information in another media segment.
195 |
At least one PES packet does not have a PTS timestamp.
196 |
PCR is not present in the Segment prior to the first byte of a TS packet payload containing media data.
197 |
198 |
199 | The user agent will accept and ignore PSI that is identical to the information in the last initialization segment which may appear repeatedly throughout the segment.
200 |
201 |
202 |
203 |
Random Access Points
204 |
A random access point as defined in this specification corresponds to Elementary Stream Random Access Point as defined in
205 | [[!MPEG2TS]].
206 |
207 |
208 |
Timestamp Rollover & Discontinuities
209 |
Timestamp rollovers and discontinuities MUST be handled by the UA. The UA's MPEG-2 TS implementation MUST maintain an internal offset
210 | variable, MPEG2TS_timestampOffset, to keep track of the offset that needs to be applied to timestamps
211 | that have rolled over or are part of a discontinuity. MPEG2TS_timestampOffset is initially set to 0 when the SourceBuffer is
212 | created. This offset MUST be applied to the timestamps as part of the conversion process from MPEG-2 TS packets
213 | into for the . This results in the coded frame timestamps
214 | for a packet being computed by the following equations:
Each time a timestamp rollover is detected, 2^33 MUST be added to .
222 |
When a discontinuity is detected, MUST be adjusted to make the timestamps after the discontinuity appear
223 | to come immediately after the timestamps before the discontinuity.
224 |
When is called, MUST be set to 0.
225 |
When is successfully set, MUST be set to 0.
226 |
227 |
228 |
229 |
230 |
231 |
232 |
Acknowledgments
233 | The editors would like to thank for their contributions to this specification.
234 |
235 |
236 |
237 |
--------------------------------------------------------------------------------
/mpeg-audio-byte-stream-format-cr.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 | MPEG Audio Byte Stream Format
6 |
7 |
8 |
9 |
10 |
11 |
12 |
13 |
14 |
15 |
16 |
17 |
57 |
58 |
174 |
233 | This specification defines a Media Source Extensions byte stream format specification based on MPEG audio streams.
234 |
Status of This Document
235 |
236 |
237 |
238 |
239 | This document is merely a W3C-internal document. It
240 | has no official standing of any kind and does not represent consensus of the W3C
241 | Membership.
242 |
This section specifies the MIME-types that may be passed to isTypeSupported() or addSourceBuffer() for byte streams that conform to this specification.
"audio/mpeg" for MPEG-1/2/2.5 Layer I/II/III streams, as specified in RFC 3003.
261 |
262 |
The "codecs" MIME-type parameter must not be used with these MIME-types.
263 |
264 |
265 |
266 |
3. MPEG Audio Frames
267 |
The format of an MPEG Audio Frame depends on the MIME-type used.
268 |
269 |
If the "audio/aac" MIME-type is used, an MPEG Audio Frame is a sequence of bytes that conform to the adts_frame() syntax specified in Table 1.A.5 of ISO/IEC 14496-3:2009.
270 |
If the "audio/mpeg" MIME-type is used, an MPEG Audio Frame is a sequence of bytes that conform to the frame() syntax element specified in Section 2.4.1.2 of ISO/IEC 11172-3:1993 or the corresponding definition in ISO/IEC 13818-3:1998.
271 |
272 |
273 |
274 |
275 |
4. Metadata Frames
276 |
Since ID3v1, ID3v2 metadata frames, and Icecast headers are common in existing MPEG audio streams, implementations should gracefully handle such frames. Zero or more of these metadata frames are allowed to occur before, after, or between MPEG Audio Frames. Minimal implementations must accept, consume, and ignore these frames. More advanced implementations may choose to expose the metadata information via an inband TextTrack or some other mechanism.
277 |
278 |
279 |
4.1 Icecast headers
280 |
There is no normative spec for Icecast/SHOUTcast headers, just examples. For the purpose of this specification, an Icecast header is defined as beginning with the 4 character sequence "ICY "(U+0049 I, U+0043 C, U+0059 Y, U+0020 SPACE) and ending with a pair of carriage-return line-feed sequences (U+000D CARRIAGE RETURN, U+000A LINE FEED, U+000D CARRIAGE RETURN, U+000A LINE FEED).
281 |
Note
Icecast headers are allowed in the byte streams because some Icecast and SHOUTcast
282 | servers return a status line that looks like "ICY OK 200" instead of a standard HTTP status line.
283 | User-agent network stacks typically interpret this as an HTTP 0.9 response and include the
284 | header in the response body. Allowing these headers to appear provides a simple way to
285 | interoperate with these servers.
Implementors should be aware that this specification is not stable. Implementors who are not taking part in the discussions are likely to find the specification changing out from under them in incompatible ways. Vendors interested in implementing this specification before it eventually reaches the Candidate Recommendation stage should join the mailing list mentioned below and take part in the discussions.
127 |
128 |
129 |
130 |
Introduction
131 |
This specification defines segment formats for implementations that choose to support MPEG audio streams specified in , , and .
132 |
It defines the used to signal codecs, and provides the necessary format specific definitions for , , and required by the of the Media Source Extensions spec. It also defines extra behaviors and state that only apply to this byte stream format.
133 |
134 |
135 |
136 |
MIME-types
137 |
This section specifies the MIME-types that may be passed to or for byte streams that conform to this specification.
138 |
139 |
"audio/aac" for sequences of ADTS frames, as specified in .
140 |
"audio/mpeg" for MPEG-1/2/2.5 Layer I/II/III streams, as specified in RFC 3003 [[!RFC3003]].
141 |
142 |
The "codecs" MIME-type parameter MUST NOT be used with these MIME-types.
143 |
144 |
145 |
146 |
MPEG Audio Frames
147 |
The format of an MPEG Audio Frame depends on the used.
148 |
149 |
If the "audio/aac" MIME-type is used, an MPEG Audio Frame is a sequence of bytes that conform to the adts_frame() syntax specified in Table 1.A.5 of .
150 |
If the "audio/mpeg" MIME-type is used, an MPEG Audio Frame is a sequence of bytes that conform to the frame() syntax element specified in Section 2.4.1.2 of or the corresponding definition in .
151 |
152 |
153 |
154 |
155 |
Metadata Frames
156 |
Since , metadata frames, and are common in existing MPEG audio streams, implementations SHOULD gracefully handle such frames. Zero or more of these metadata frames are allowed to occur before, after, or between . Minimal implementations MUST accept, consume, and ignore these frames. More advanced implementations MAY choose to expose the metadata information via an inband or some other mechanism.
157 |
158 |
159 |
Icecast headers
160 |
There is no normative spec for Icecast/SHOUTcast headers, just examples. For the purpose of this specification, an Icecast header is defined as beginning with the 4 character sequence "ICY "(U+0049 I, U+0043 C, U+0059 Y, U+0020 SPACE) and ending with a pair of carriage-return line-feed sequences (U+000D CARRIAGE RETURN, U+000A LINE FEED, U+000D CARRIAGE RETURN, U+000A LINE FEED).
161 |
Icecast headers are allowed in the byte streams because some Icecast and SHOUTcast
162 | servers return a status line that looks like "ICY OK 200" instead of a standard HTTP status line.
163 | User-agent network stacks typically interpret this as an HTTP 0.9 response and include the
164 | header in the response body. Allowing these headers to appear provides a simple way to
165 | interoperate with these servers.
166 |
167 |
168 |
169 |
170 |
Segment Definitions
171 |
The MPEG audio byte stream is a combination of one or more and zero or more .
Non-technical description of the Process Flowchart image
18 |
19 |
The flowchart reads from a top down perspective. At the top is a container
20 | marked MediaSource, with 3 arrows flowing down to containers each marked
21 | SourceBuffer. (Not indicated, but for sake of clarity in this description
22 | referred to here-after as SourceBuffer 1, shown with a blue background;
23 | SourceBuffer 2, shown with a mauve background; and SourceBuffer 3, shown
24 | with a rose background; as described from Left to Right).
25 |
26 |
27 |
Below these 3 SourceBuffer containers is a dashed line, with an indication
28 | that the top half (MediaSource plus the 3 SourceBuffer containers) are taken
29 | from the Media Source API, while the details to follow are taken from the
30 | HTMLMediaElement.
31 |
32 |
33 |
34 |
35 |
Flowing down from SourceBuffer 1
36 |
37 |
38 |
Flowing down from SourceBuffer 1 is a blue triangle with 3 arrows pointing
39 | to 3 process indications, each labeled Track Buffer.
40 |
41 |
The first Track Buffer from the left then flows to a red box labeled Video
42 | Decoder, which then flows to an indicator (three dots stacked vertically)
43 | labeled Video Tag Display Region.
44 |
45 |
The second Track Buffer flows to a green box labeled Audio Decoder, which
46 | then follows to an indicator of an open switch, which then continues to an
47 | indicator (green circle with an X) labeled Audio Device.
48 |
49 |
The third Track Buffer also flows to a green box labeled Video Decoder,
50 | which also flows to a switch, this time however indicated closed, which then
51 | also continues to the same indicator (green circle with an X) labeled Audio
52 | Device as the second Track Buffer.
53 |
54 |
55 |
56 |
57 |
Flowing down from SourceBuffer 2
58 |
59 |
60 |
Flowing down from SourceBuffer 2 is a mauve triangle with an arrow flowing
61 | to a red box labeled Video Decoder, which then flows to the same Video Tag
62 | Display Region previously mentioned in SourceBuffer 1.
63 |
64 |
65 |
Flowing down from SourceBuffer 3
66 |
67 |
68 |
Flowing down from SourceBuffer 3 is a rose triangle with an arrow flowing to
69 | a green box labeled Audio Decoder, which then flows through a closed switch,
70 | which then also continues to the same indicator (green circle with an X)
71 | labeled Audio Device previously mentioned in SourceBuffer 1
Implementors should be aware that this specification is not stable. Implementors who are not taking part in the discussions are likely to find the specification changing out from under them in incompatible ways. Vendors interested in implementing this specification before it eventually reaches the Candidate Recommendation stage should join the mailing list mentioned below and take part in the discussions.
151 |
152 |
153 |
154 |
Introduction
155 |
This specification describes a byte stream format based on the WebM container format [[!WEBM]]. It defines the MIME-type parameters used to signal codecs, and provides
156 | the necessary format specific definitions for , , and required by
157 | the of the Media Source Extensions spec.
158 |
159 |
160 |
161 |
MIME-type parameters
162 |
This section specifies the parameters that can be used in the MIME-type passed to or .
163 |
164 |
codecs
165 |
166 | A comma separated list of codec IDs used to specify what codecs will be used in the byte stream.
167 |
168 |
169 |
170 |
Codec ID
171 |
Valid with "audio/webm"
172 |
Valid with "video/webm"
173 |
174 |
175 |
176 |
177 |
vorbis
178 |
true
179 |
true
180 |
181 |
182 |
opus
183 |
true
184 |
true
185 |
186 |
187 |
vp8
188 |
false
189 |
true
190 |
191 |
192 |
vp9
193 |
false
194 |
true
195 |
196 |
197 |
vp09... as described in the VP Codec ISO Media File Format Binding document[[VP09CODECSPARAMETERSTRING]]
198 |
false
199 |
true
200 |
201 |
202 |
203 |
204 | Implementations SHOULD support all of the codec IDs mentioned in the table above.
205 |
206 |
207 | Implementations SHOULD encourage applications to prefer the "vp09..." codec ID over "vp9". The "vp09..." format provides detailed profile and color information, enabling implementations to give more accurate answers for codec support.
208 |
209 |
210 |
Examples of valid MIME-types with a codecs parameter.
A WebM MUST contain a subset of the elements at the start of a typical WebM file.
227 |
228 |
The user agent MUST run the if any of the following conditions are not met:
229 |
230 |
The MUST start with an element, followed by a header.
231 |
The size value in the header MUST signal an "unknown size" or contain a value large enough to include the and elements that follow.
232 |
A element and a element MUST appear, in that order, after the header and before any further or elements.
233 |
234 |
235 |
The user agent MUST accept and ignore any elements other than an or a that occur before, in between, or after the and
236 | elements.
237 |
238 |
The user agent MUST source attribute values for id, kind, label and language for , and
239 | objects as described for WebM in the in-band tracks spec [[INBANDTRACKS]].
240 |
241 |
242 |
243 |
Media Segments
244 |
A WebM is a single element.
245 |
246 |
The user agent uses the following rules when interpreting content in a :
247 |
248 |
The TimecodeScale in the most recently appended applies to all timestamps in the
249 |
The Timecode element in the contains a in TimecodeScale units.
250 |
The Cluster header MAY contain an "unknown" size value. If it does then the end of the cluster is reached when another header or an element header that indicates the start
251 | of an is encountered.
252 |
253 |
254 |
The user agent MUST run the if any of the following conditions are not met:
255 |
256 |
The Timecode element MUST appear before any Block & SimpleBlock elements in a .
257 |
Block & SimpleBlock elements are in time increasing order consistent with the .
258 |
If the most recent describes multiple tracks, then blocks from all the tracks MUST be interleaved in time increasing order. At least one block from all audio and video
259 | tracks MUST be present.
260 |
261 |
262 |
The user agent MUST accept and ignore or elements that follow a element.
263 |
264 |
265 |
266 |
Random Access Points
267 |
Either a SimpleBlock element with its Keyframe flag set, or a BlockGroup element having no ReferenceBlock elements, signals the location of a for that track. The order of multiplexed blocks within a MUST conform to the .
268 |
269 |
270 |
271 |
272 |
273 |
Acknowledgments
274 | The editors would like to thank for their contributions to this specification.
275 |
276 |
277 |
278 |
--------------------------------------------------------------------------------
79 |