├── LICENSE.md ├── README.md ├── TTMLWorkflow.png └── TTMLWorkflow.pptx /LICENSE.md: -------------------------------------------------------------------------------- 1 | # W3C SOFTWARE AND DOCUMENT NOTICE AND LICENSE 2 | 3 | http://www.w3.org/Consortium/Legal/2015/copyright-software-and-document 4 | 5 | This work is being provided by the copyright holders under the following license. 6 | 7 | License 8 | ------- 9 | 10 | By obtaining and/or copying this work, you (the licensee) agree that you have read, understood, and will comply with the following terms and conditions. 11 | 12 | Permission to copy, modify, and distribute this work, with or without modification, for any purpose and without fee or royalty is hereby granted, provided that you include the following on ALL copies of the work or portions thereof, including modifications: 13 | 14 | The full text of this NOTICE in a location viewable to users of the redistributed or derivative work. 15 | 16 | Any pre-existing intellectual property disclaimers, notices, or terms and conditions. If none exist, the W3C Software and Document Short Notice should be included. 17 | 18 | Notice of any changes or modifications, through a copyright statement on the new code or document such as "This software or document includes material copied from or derived from [title and URI of the W3C document]. Copyright © [YEAR] W3C® (MIT, ERCIM, Keio, Beihang)." 19 | 20 | Disclaimers 21 | ----------- 22 | 23 | THIS WORK IS PROVIDED "AS IS," AND COPYRIGHT HOLDERS MAKE NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO, WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE OR DOCUMENT WILL NOT INFRINGE ANY THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS. 24 | 25 | COPYRIGHT HOLDERS WILL NOT BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF ANY USE OF THE SOFTWARE OR DOCUMENT. 26 | 27 | The name and trademarks of copyright holders may NOT be used in advertising or publicity pertaining to the work without specific, written prior permission. Title to copyright in this work will at all times remain with copyright holders. 28 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # TTML in MP4 and MPEG-DASH guidelines 2 | 3 | ### About 4 | 5 | Authors: 6 | - Romain Bouqueau (romain.bouqueau@gpac-licensing.com) 7 | - Cyril Concolato (cyril.concolato@telecom-paristech.fr) 8 | 9 | Contributors: 10 | - Nigel Megitt (nigel.megitt@bbc.co.uk) 11 | - Andreas Tai (tai@irt.de) 12 | - Frans de Jong (dejong@ebu.ch) 13 | 14 | ### Introduction 15 | 16 | This document describes different workflows for the delivery of TTML content in MP4 and MPEG-DASH. It tries to provide hints on how to build such workflows based on existing tools. Its goal is to drive the development of TTML tools so that maximum interoperability is achieved. 17 | 18 | ### History and profiles for TTML: EBU work, HbbTV 2.0, IMSC 19 | 20 | About the technologies used in this article: 21 | - [MP4](https://en.wikipedia.org/wiki/MPEG-4_Part_14) is a container format standardized by MPEG. 22 | - [MPEG-DASH](http://standards.iso.org/ittf/PubliclyAvailableStandards/c065274_ISO_IEC_23009-1_2014.zip) is an adaptive bitrate (ABR) streaming standard allowing delivery of content using conventional HTTP web servers also standardized by MPEG. 23 | - [TTML](http://www.w3.org/TR/ttml1/) (Timed Text Markup Language) is a subtitling format designed by W3C. 24 | - [EBU-TT-D](https://tech.ebu.ch/publications/tech3380) is a profile of TTML designed for live and on demand distribution of subtitles over IP based networks. EBU-TT-D restricts TTML. EBU-TT-D is designed by the EBU. 25 | - [HbbTV 2.0](https://www.hbbtv.org/pages/about_hbbtv/HbbTV_specification_2_0.pdf) is a standard for hybrid digital TV delivery. HbbTV 2.0 mandates the implementation of EBU-TT-D, MP4 and MPEG-DASH. 26 | - [IMSC](http://www.w3.org/TR/ttml-imsc1/) is a pair of TTML profiles, one for text and one for images designed for subtitles and captions. The IMSC Text profile is a superset of EBU-TT-D. 27 | 28 | ### Overview of the workflow 29 | Generally speaking, we can assume that TTML workflows follow the architecture provided by the following image: 30 | ![Image of Workflow](/TTMLWorkflow.png) 31 | 32 | In this workflow, the MP4 packager and DASH packager could be the same tool, as it is the case with [MP4Box](http://gpac.io/mp4box). Similarly, the DASH Access Engine and the MP4 Parser and the TTML Renderer could be the same tool, or separate tools such as respectively [DASH.js](https://github.com/Dash-Industry-Forum/dash.js/wiki), [MP4Box.js](https://github.com/gpac/mp4box.js/) and a TTML to HTML rendering tool. 33 | 34 | #### Producing TTML content over MP4 and DASH 35 | 36 | Given this workflow, there are several options to produce, package and deliver TTML content over MP4 and DASH. All options have in common that they try to minimize the quantity of downloaded data during the streaming session: this means avoiding downloading the same TTML content multiple times; and at the same time not requiring the download of the whole TTML content to start the session (especially relevant for live applications). Packaging the TTML content of the entire session as a single DASH segment is indeed not optimal. Packaging of the TTML requires the content to be spread over multiple DASH segments. This can be useful for seeking or for inserting ads between segments. 37 | 38 | DASH segments are typically of constant duration and aligned across audio and video representations. This is not a strict requirement though. Since TTML content does not have a constant rate of change, segmentation of TTML content may lead to either variable duration segments or to data duplication across segments. Such duplication should be avoided and limited, possibly to the last sample of a segment containing some data that is present in the first sample of the next segment. 39 | 40 | Note: [preliminary figures](https://github.com/rbouqueau/TTML_in_MP4_DASH_statement/issues/6#issuecomment-172486556) show that subtitles account for 0.0004% to 0.07% of the whole "programme" bandwith. As a consequence we don't think debate about storage, network and distribution costs are sensible. 41 | 42 | #### Need for a TTML Segmenter 43 | 44 | In above workflow it may be difficult for tools that have only simple TTML capabilities, to process a TTML document for the purpose of creating small, self contained, non-timewise-overlapping TTML documents. The TTML Segmenter segments one (or more) TTML input document(s) into output TTML documents, each containing only the timed data needed for presentation within its segment time to avoid unnecessary data duplication 45 | 46 | The example below shows a TTML document with successive `p` elements overlapping in time: 47 | 48 | ```xml 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | Title at the top. 65 | 66 | 67 | Text at the bottom. 68 | 69 | 70 | 71 | 72 | ``` 73 | 74 | Some workflows may decide that the TTML Authoring tool will post-process the TTML content to produce those non-timewise-overlapping TTML documents with a fine granularity to support the smallest segment duration and take care of timebase conversions. Such a post-processing TTML Segmenting tool would make the task of tools down the chain easier. Other workflows may decide to leave the segmentation to tools down the chain like the DASH packager because the segment duration is only known at that level in the workflow. Yet other workflows may use tools in-between to make the TTML authoring DASH-unaware and the DASH processing TTML-unaware. Depending on the design choice, the interface between the tools in the workflow will not be the same. 75 | 76 | The TTML content may be segmented in the following ways: 77 | - A single TTML document is created for the entire streaming session. The content is marked as redundant in all but the first sample or segment (technical details below in this document). 78 | - The TTML Segmenter attempts to split the input documents using a strategy to choose the best times at which to begin and end each segment based on the content and other heuristics such as maximum segment size in data or in time. This strategy would typically try to avoid any content overlapping with other segments but this may not always be possible. In this case the output of the TTML segmenter is both a set of TTML documents and some kind of manifest indicating the times of each segment in turn, suitable for use within the packager. 79 | - The TTML Segmenter splits the input documents into predefined segment durations. For each segment it selects all of the content that overlaps in time with the period of interest, and then selects all referenced styles, regions etc in the head. Some segments may contain no content. Some adjacent segments may duplicate some or all of their content. 80 | 81 | #### Interface between MP4 Parser and TTML Renderer 82 | The MP4 standard assumes that only one sample at a time is active. This means that the MP4 parser will deliver one TTML document at a time to the TTML renderer and will assume that the previous TTML document will be replaced by the new one, and that it will be used for a given duration. This standard behavior thus constrains the upper part of the workflow, in the sense that samples cannot overlap in time and therefore the contained TTML document should not overlap in time. This improves interoperability by reducing the number of choices left. 83 | 84 | Some optimizations at the MP4 level allow for the MP4 Parser to indicate that a new TTML document is the same as the previous one i.e. using the ```sample_has_redundancy``` field of the [```sdtp``` box](https://github.com/gpac/mp4box.js/blob/9f0bf463a979aa795e83a488360ed9db0fbf1329/src/parsing/sdtp.js). In this case the new document only extends the duration of the previous one. This can be useful when a TTML document has been duplicated between the last sample of a segment and the first sample of the next segment. We don't expect TTML samples to be big in size so repeating them seems acceptable. 85 | 86 | #### Interface between TTML Authoring Tool and MP4 Packager 87 | There are several possibilities here. To achieve interoperability, workflow designers have to choose a strategy and make sure the tools are the right ones. This depends on the TTML Authoring tool. This tool may produce: 88 | - A single TTML document valid for the entire streaming session. If so, either the MP4 packager will have to split the TTML document into multiple samples, or the DASH packager will have to split the sample into multiple samples and segments to avoid unnecessary downloads. This task can be complex for general TTML documents, but it can be simpler for some profiles, such as EBU-TT-D. Hence, the workflow architecture may differ depending on the type of TTML documents. 89 | - Multiple non-timewise-overlapping TTML documents. If the TTML authoring tool is aware of the target DASH segment duration, it should ideally provide one TTML document per segment. If the TTML authoring tool is not aware of the DASH delivery parameters, it should try to produce the TTML documents with the smallest duration that cannot be further split. 90 | 91 | ### Examples 92 | 93 | Examples can be found [here](https://github.com/rbouqueau/TTML_in_MP4_DASH_statement/wiki/ebuttd_in_isobmff_samples). 94 | 95 | ### Conclusion 96 | 97 | If you have any feedback, remarks and questions. Please free to contact us directly or via [our github project page](https://github.com/rbouqueau/TTML_in_MP4_DASH_statement). Thank you. 98 | -------------------------------------------------------------------------------- /TTMLWorkflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rbouqueau/TTML_in_MP4_DASH_statement/c0787604c018cd331554aba7a640e36d84805e70/TTMLWorkflow.png -------------------------------------------------------------------------------- /TTMLWorkflow.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rbouqueau/TTML_in_MP4_DASH_statement/c0787604c018cd331554aba7a640e36d84805e70/TTMLWorkflow.pptx --------------------------------------------------------------------------------