├── CONTRIBUTING.md ├── LICENSE.md ├── README.md ├── explainer-datamodel.md ├── explainer-metrics.md ├── explainer-rendering.md ├── explainerresources ├── Available-Width.png ├── Example1.html ├── Example1.png ├── Example2.html ├── Example2.png ├── coordinates.png ├── metrics-structure.png ├── text-position-relationship.png ├── vertical-text-cn.png └── vertical-text-en.png ├── impl-chromium-notes.md ├── meetings ├── 2021-06-17.md └── 2021-08-18.md └── w3c.json /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Web Platform Incubator Community Group 2 | 3 | This repository is being used for work in the W3C Web Platform Incubator Community Group, governed by the [W3C Community License 4 | Agreement (CLA)](http://www.w3.org/community/about/agreements/cla/). To make substantive contributions, 5 | you must join the CG. 6 | 7 | If you are not the sole contributor to a contribution (pull request), please identify all 8 | contributors in the pull request comment. 9 | 10 | To add a contributor (other than yourself, that's automatic), mark them one per line as follows: 11 | 12 | ``` 13 | +@github_username 14 | ``` 15 | 16 | If you added a contributor by mistake, you can remove them in a comment with: 17 | 18 | ``` 19 | -@github_username 20 | ``` 21 | 22 | If you are making a pull request on behalf of someone else but you had no part in designing the 23 | feature, you can remove yourself with the above syntax. 24 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | All Reports in this Repository are licensed by Contributors 2 | under the 3 | [W3C Software and Document License](http://www.w3.org/Consortium/Legal/2015/copyright-software-and-document). 4 | 5 | Contributions to Specifications are made under the 6 | [W3C CLA](https://www.w3.org/community/about/agreements/cla/). 7 | 8 | Contributions to Test Suites are made under the 9 | [W3C 3-clause BSD License](https://www.w3.org/Consortium/Legal/2008/03-bsd-license.html) 10 | 11 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | **Note:** This effort has been superceded by the [**Canvas placeElement()**](https://github.com/WICG/canvas-place-element) proposal. 2 | 3 | Welcome! 4 | ============= 5 | This is the home for the **Formatted Text** incubation effort. There are several explainers 6 | for this feature: 7 | 8 | 1. This readme is a general introduction to the problem space and the motivation for this 9 | effort. 10 | 2. Input data model for the API is described in [data model](explainer-datamodel.md). 11 | 3. Output data model or [text metrics](explainer-metrics.md). 12 | 4. [Rendering](explainer-rendering.md) of the output data model. 13 | 14 | ## Introduction & Challenges 15 | 16 | ### Imperative multi-line text lacking in canvas 17 | 18 | Applications like word processors, spreadsheets, PDF viewers, etc., face a 19 | choice when moving their legacy presentation algorithms to the web. The view layer of 20 | these applications can be designed to output HTML, SVG or use Canvas. Canvas 21 | is an expedient choice for some view models since it can easily present data models 22 | where the view model and the data model are not tightly coupled. Additionally, the 23 | Canvas APIs map well to the imperative APIs the legacy algorithms used for rendering 24 | content. 25 | 26 | In gaming and mixed reality (VR and AR) scenarios today, Canvas is the only logical 27 | choice for presenting a rapidly-changing view. 28 | 29 | In these scenarios, it is often necessary to present text to the user. The 2D Canvas 30 | API currently provides a relatively simplistic text rendering capability: A single run 31 | of text can be 32 | [measured](https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/measureText) 33 | and [rendered](https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/fillText) 34 | **as a single line**, optionally compressed to fit 35 | within a certain width. If the developer needs to present more than just a few words 36 | (e.g., a paragraph of text) then things get very complicated very quickly. Some options 37 | might include trying a hybrid approach of overlaying HTML elements with the Canvas to 38 | leverage HTML's native text rendering and wrapping capabilities, or attempting to 39 | write your own line breaking logic in JavaScript using the primitive Canvas text measuring 40 | and rendering APIs as a starting point. Neither of these options are very desirable, 41 | and the HTML hybrid approach may not be available depending on the scenario (e.g., in 42 | VR headsets). 43 | 44 | ### Lack of unified metrics for multi-line text in HTML 45 | 46 | In other scenarios, separately from Canvas, applications are interested in getting more 47 | detailed text metrics for multi-line text (e.g., from previously rendered text in the DOM). 48 | Precise text metrics from the source text can help inform placement of overlays, annotations, 49 | translations, etc. Unfortunately, text metrics that help map source text placement in a 50 | given layout are not currently provided by the web platform. As a result a variety of 51 | workarounds are employed to try to approximate this data today often at the cost of 52 | slower performance for users. 53 | 54 | It is our aspiration to bring a unified handling of multi-line text metrics to the web 55 | platform that will service the imperative data model for canvas scenarios as well as 56 | address many other use cases for text metrics in HTML. 57 | 58 | ### Challenges for JavaScript implementations 59 | 60 | Why would a JavaScript implementation of line-breaking and wrapping be hard? While this 61 | may seem trivial at first glance, there are a number of challenges especially for 62 | applications that must be designed to handle rendering of text in multiple languages. 63 | Consider the complexity involved in the following requirements for a robust line 64 | breaking algorithm: 65 | 66 | * Identify break opportunities between words or 67 | [Graphemes](https://en.wikipedia.org/wiki/Grapheme). Break opportunities are 68 | based primarily on the Unicode Spec but also use dictionaries for languages 69 | like Thai and French that dictate additional line breaking rules. 70 | * Identify grapheme clusters. Grapheme clusters are character combinations (such as 71 | [Diacritics](https://en.wikipedia.org/wiki/Diacritic) and 72 | [Ligatures](https://en.wikipedia.org/wiki/Orthographic_ligature)) that result 73 | in a single glyph and hence should not be broken up. 74 | E.g.: `g` (Latin small letter G 0067) + `◌̈ ` (Combining dieresis 0308) = `g̈` 75 | * Handle Bidi text. For proper Bidi rendering the bidi level context needs to be 76 | considered across lines. 77 | * Text Shaping and Kerning. These features can affect the measured pixel length 78 | of a line. 79 | 80 | Javascript libraries could perform line breaking, but as noted above, this is 81 | an arduous task. This gets more complicated if text with different formatting 82 | characteristics (e.g., size, bold, italic) are needed. 83 | 84 | ## Goals 85 | 86 | The browser already has a powerful line breaking, text shaping component used 87 | for regular HTML and SVG layout. Unfortunately, this component has been tightly coupled 88 | into the browser's rendering pipeline, and not exposed in an imperative way. 89 | Furthermore HTML's DOM provides only a very limited set of metrics for text elements 90 | and almost no insight into the post-layout details of how the source text was 91 | formatted. SVG's DOM is slightly more helpful by providing some additional text metrics. 92 | 93 | Our goal is to create an abstraction that allows authors to collect multi-line 94 | formatted text into a data model that is independent of the HTML/SVG data model 95 | but can still leverage the power of the browser's native line breaking and text 96 | shaping component. We think it is valuable to re-use formatting principles from CSS 97 | as much as possible. Given the data model, we also want to provide a way to 98 | initiate layout and rendering of that data model to a canvas, and likewise provide 99 | a way to initiate layout and then read-back the text metrics that describe how the text 100 | was formatted and laid out given prescribed layout constraints (without requiring 101 | the text to be rendered at all). 102 | 103 | ## Related Work 104 | 105 | This proposal builds on a variety of existing and proposed specifications in the 106 | web platform, for whose efforts we are very grateful, and from whom we expect to 107 | get lots of feedback. 108 | * [CSS Text 3](https://drafts.csswg.org/css-text-3/) 109 | * [CSS Inline Layout 3](https://drafts.csswg.org/css-inline-3/) 110 | * [CSS Houdini Layout API](https://drafts.css-houdini.org/css-layout-api-1/) 111 | * [HTML Canvas API and TextMetrics](https://html.spec.whatwg.org/multipage/canvas.html#drawing-text-to-the-bitmap) 112 | * Open issues: [3994](https://github.com/whatwg/html/issues/3994), [4026](https://github.com/whatwg/html/issues/4026), [4030](https://github.com/whatwg/html/issues/4030), [4033](https://github.com/whatwg/html/issues/4033), [4034](https://github.com/whatwg/html/issues/4034), 113 | * [SVG Text](https://svgwg.org/svg2-draft/text.html) 114 | * [Proposed Canvas Text Modifiers](https://github.com/fserb/canvas2D/blob/master/spec/text-modifiers.md) 115 | * [CSS Houdini Font Metrics API](https://drafts.css-houdini.org/font-metrics-api/) 116 | * [CSS Houdini Box Tree API](https://drafts.css-houdini.org/box-tree-api-1/) 117 | * [Other platforms for drawing formatted text](https://docs.microsoft.com/en-us/dotnet/framework/wpf/advanced/drawing-formatted-text) 118 | * Google SKIA's Text API Proposals 119 | * [Text API Overview](https://github.com/google/skia/blob/main/site/docs/dev/design/text_overview.md) 120 | * [Canvas2D extensions for Shaped Text](https://github.com/google/skia/blob/main/site/docs/dev/design/text_c2d.md) 121 | * [Shaped Text](https://github.com/google/skia/blob/main/site/docs/dev/design/text_shaper.md) 122 | * [Experimental SKText's SelectableText interface](https://skia.googlesource.com/skia/+/refs/heads/main/experimental/sktext/include/Interface.h) 123 | * ECMA 402 [proposal to expose "line break opportunities"](https://github.com/tc39-transfer/proposal-intl-segmenter-v2) of text which implement the 124 | [Unicode® Standard Annex #14 UNICODE LINE BREAKING ALGORITHM](https://www.unicode.org/reports/tr14/) 125 | 126 | ## Implementations 127 | 128 | * Chromium has an initial implementation behind the _Experimental Web Platform Features_ flag. See [variations from the explainer in the chromium experimental implementation notes](impl-chromium-notes.md) for additional important details. 129 | 130 | ## Open issues and questions 131 | 132 | Please review and comment on our [existing open issues](https://github.com/WICG/canvas-formatted-text/issues). 133 | 134 | ## Alternatives Considered 135 | 136 | ### Fixed Data Model Objects 137 | A previous iteration of this proposal used WebIDL interfaces as containers for a 138 | group of text (`FormattedText` objects), which contained an array of text run objects 139 | (`FormattedTextRun`) with properties for style, etc. 140 | 141 | The prior reason for having an express data model was to enable persistent text 142 | (e.g., similar to DOM's `Text` nodes) to be used for memory-resident updates. However, 143 | experience with DOM `Text` nodes helps us understand that in most cases, the `Text` 144 | node object itself is irrelevant. Instead, what is needed is the JavaScript string 145 | from the `Text` node in order to mutate, aggregate, and change the text. In the DOM, 146 | such string changes need to be presented via document layout (e.g., "update the 147 | rendering"), and the only way to do that is to re-add changed strings into DOM `Text` 148 | nodes in an attached document. For Canvas-based scenarios, rendering is not done 149 | through document layout, but with explicit drawing commands (which then paint when 150 | rendering is udpated). Therefore having a DOM `Text` node-like data model really did 151 | not add much value. 152 | 153 | Furthermore, simplifying the data model was advantageous for performance-critical 154 | scenarios that inlcude the creation time of the data model, in addition to layout and 155 | rendering optimizations. 156 | 157 | ### Measure/render pattern for intermediate line objects 158 | In a previous iteration of this proposal, we called for a "simple" and "advanced" 159 | model for rendering formatted text to the canvas. The advanced model allowed authors 160 | to place arbitrary lines by alternately measuring a part of the data model and then 161 | rendering the resulting "line metrics" object. 162 | 163 | **Note: a newer version of this _measure/render_ design has been integrated into the 164 | latest iteration of this proposal, although it uses an iterable design pattern. Scenarios 165 | that require potentially flowing text through _multiple_ containers (such as implementing 166 | custom line positioning for CSS regions/ pagination / multicolumn were the motivation 167 | for bringing this capability back despite some of the downsides noted below.** 168 | 169 | Under that design, we were assuming that authors would want to cache and re-use the 170 | line objects that were produced. If authors would not re-use these objects, then no 171 | optimization could be made for performance (since we were letting authors do the 172 | fragmentation themselves). For simple use cases where the canvas will be re-painted 173 | without changes, having authors cache and reuse line objects seemed a reasonable 174 | request. However, if authors decide to only re-paint the canvas when things change, 175 | then to recapture performance, authors are left to implement their own line invalidation 176 | logic or to just throw away all the lines and start from scratch--the worst-case scenario 177 | for performance. 178 | 179 | We thought about addressing this concern by adding a "dirty" flag to lines that have been 180 | invalidated to help the author create an efficient invalidation scheme. But line analysis 181 | and caching logic has never been exposed to authors before in the web platform, and we 182 | didn't want to create a feature with this foot-gun. The advanced use case was primarily 183 | about enabling occusions and supporting things like floaters obstructing lines, and CSS 184 | already has standards for those scenarios--when we decided to embrace more CSS constructs 185 | for this feature, it was decided that the advanced use case could be dropped entirely. 186 | 187 | **Note:** many of the advanced features possible with this iteration of the proposal 188 | require new CSS features that are only recently starting to be interoperably implemented 189 | (E.g., CSS Shapes). 190 | 191 | ### Imperative model 192 | The proposal here addresses two separate problems. One of styling ranges of text 193 | and having an object model and two of auto wrapping text. 194 | 195 | An alternative design considered was to support auto wrapping without requiring 196 | that the developer provides all the text upfront. Similar to canvas path, the 197 | developer would call `setTextWrapWidth( availableWidth )` and follow through with 198 | multiple calls to `fillText` on the canvas context that renders text and advances 199 | a cursor forward. 200 | 201 | Such an imperative style API was not pursued for two reasons. With bidi text, the 202 | entire width of a right-to-left segment needs to be determined before any of the 203 | `fillText` calls can be rendered. This issue can be addressed by adding a finalization 204 | step say `finalizeFillText()`. However still, such an imperative API adds a performance 205 | cost in terms of recreating the formatted text when the developer is simply trying to 206 | redraw the same text content for a changing available width. 207 | 208 | ## Privacy Considerations 209 | 210 | HTML5 canvas is a browser fingerprinting vector 211 | (see [canvas fingerprinting](https://en.wikipedia.org/wiki/Canvas_fingerprinting)). 212 | Fingerprinting happens through APIs `getImageData`, `toDataURL`, etc. that allow 213 | readback of renderer content exposing machine specific rendering artifacts. 214 | This proposal adds the ability to render multiple lines of text, potential 215 | differences in text wrapping across browsers could contribute to additional 216 | fingerprinting. The existing implementer mitigations in some user agents that 217 | prompt users on canvas readback continues to work here. 218 | 219 | We are currently evaluating whether this API would increase fingerprinting surface 220 | area and will update this section with our findings. We welcome any community feedback. 221 | 222 | ## Contributors: 223 | (In alphabetical order) 224 | 225 | [dlibby-](https://github.com/dlibby-), 226 | [ijprest](https://github.com/ijprest), 227 | [sushraja-msft](https://github.com/sushraja-msft), 228 | [travisleithead](https://github.com/travisleithead), 229 | -------------------------------------------------------------------------------- /explainer-datamodel.md: -------------------------------------------------------------------------------- 1 | # Formatted Text - Input Data Model 2 | 3 | This document describes how to provide input text (and associated formats) that describe multi-line 4 | formatted text in order to have it shaped and formatted into metrics. The input consists of text 5 | (JavaScript strings), along with metadata (e.g., for internationalization) and CSS formats/style. 6 | Formatted text can be created in both document and non-document (Worker) scenarios and has no 7 | dependency on DOM Text nodes or Elements. 8 | 9 | This explainer focuses on the input data model for formatted text. The output is described in 10 | a separate [formatted text metrics](explainer-metrics.md) explainer. The obtained metrics objects 11 | can then be [rendered](explainer-rendering.md) if desired. For the motivating problems and use 12 | cases, see the [readme](README.md). 13 | 14 | ## Providing an input model for formatted text 15 | 16 | On the web today, to render text there are two general options: use the DOM (HTML's 17 | `CharacterData`/`Text` nodes or SVG Text-related Elements), where text is collected in the 18 | document's "retained" mode infrastructure; the web platform decides when and how to compose and 19 | render the text with declarative input from you (in the form of CSS); alternatively, Canvas can 20 | be used to measure (and render) the text directly with JavaScript (an "immediate" mode approach). 21 | Canvas provides very limited text support today and (by design) leaves any special formatting, text 22 | wrapping, international support, etc., up to the JavaScript author. 23 | 24 | Input text and formats are fed into one of two "formatter functions" that produce either a container 25 | of metrics (a `FormattedText` instance) or an iterator that produces line metrics (`FormattedTextLine` 26 | instances). The capabilities of these (and other) metrics objects are described separately in 27 | [the output text metrics](explainer-metrics.md). 28 | 29 | This explainer focuses on the **input** parameters for formatting text. 30 | 31 | ### Principles 32 | 33 | * Don't create yet-another-data-model for holding text (e.g., `Text` nodes)--just use strings! 34 | * Scope formatting to the needs of inline text layout. 35 | * Leverage CSS as the universal text layout system for the web (even in an imperative model) which 36 | also provides for future extensibility. 37 | * Avoid multi-level hierarchal text structures (which ultimately must be linearly iterated for 38 | rendering anyway) 39 | * Object-centric model (versus range+indexes) avoids common problems with overlapping indexes and 40 | is a familiar design pattern for authors. 41 | * Provide flexibility for simple use cases (flowing a single paragraph in a container) _and_ 42 | advanced use cases (fragmentation, pagination, custom flowing and wrapping such as CSS regions). 43 | * Provide easy-to-use ergonomics for JavaScript authors (iterator protocol) 44 | * Extensibility for internationalization hints and reserved future scenarios. 45 | 46 | ## Formatting text all at once 47 | 48 | Formatted text can be obtained using two formatting functions: `format` and `lines`, both available as 49 | static functions of the `FormattedText` constructor object. The parameters to both functions are 50 | similar. This section will describe the input data model in terms of the `format` formatting function. 51 | `lines` will be described later on. 52 | 53 | `format` produces output [text metrics](explainer-metrics.md) directly in one call. The result of all 54 | the text formatting (and potential line wrapping) is available on a synchronously returned 55 | `FormattedText` instance, which represents the text metrics for a container (or paragraph) of formatted 56 | text. The input parameters are: 57 | 58 | 1. text or an array of text 59 | 2. metadata/ style for all the text 60 | 3. constraints for the formatting process 61 | 62 | Each of these is explained in more detail below. 63 | 64 | ### Expressing text 65 | 66 | Text can be expressed as JavaScript strings, or as a JavaScript object with property named 67 | `text` that has a JavaScript string as its value. 68 | 69 | ```js 70 | // The following representations of text are equivalent: 71 | FormattedText.format( "hello" ); 72 | FormattedText.format( { text: "hello" } ); 73 | ``` 74 | 75 | Text can also be expressed in separate runs (which are concatenated together when formatted) 76 | when wrapped in an Array. The following are equivalent: 77 | 78 | ```js 79 | // The following text runs are equivalent: 80 | FormattedText.format( [ "hello", " world!" ] ); 81 | FormattedText.format( [ { text: "hello" }, { text: " world!" } ] ); 82 | ``` 83 | 84 | ### Adding some style 85 | 86 | Without additional input, the formatter functions will apply default styles in their operation, based 87 | on the default styles of CSS (such as `font-family`, `font-size`, `color`, `writing-mode` etc.). But 88 | default styles are boring. Here's how to add your own style and formatting to the text. 89 | 90 | Style for *all the text* (global styles for the purposes of this function call) are provided via 91 | the second parameter to `format`. Style input uses the same syntax as Element inline styles 92 | (i.e., `` semi-colon separated property/value pairs) 93 | and is ultimately parsed with the same CSS parser. The following formatted text outputs will all 94 | be blue: 95 | 96 | ```js 97 | FormattedText.format( "hello world!", "color: blue" ); 98 | FormattedText.format( [ "hello", " world!" ], "color: blue" ); 99 | FormattedText.format( [ { text: "hello" }, { text: " world!" } ], "color: blue" ); 100 | ``` 101 | 102 | Like text, style strings can be wrapped in a JavaScript object with property named `style` that 103 | has the style text string as its value. The following are equivalent: 104 | 105 | ```js 106 | // The following are equivalent expressions of style: 107 | FormattedText.format( "hello world!", "color: blue" ); 108 | FormattedText.format( "hello world!", { style: "color: blue" } ); 109 | ``` 110 | 111 | ### Styling specific text runs 112 | 113 | If specific text runs need to override or have specific style values, then style can also be 114 | added to the object-form of a text run. In this example, the word "brown" will be colored 115 | brown and bold and all the text will be italic: 116 | 117 | ```js 118 | FormattedText.format( [ "The quick ", 119 | { text: "brown", style: "color: brown; font-weight: bold" }, 120 | " fox jumps over the lazy dog." 121 | ], "font-style: italic" ); 122 | ``` 123 | 124 | And in the following example, all the text will be blue _except_ for the word brown, which 125 | will be colored brown: 126 | 127 | ```js 128 | FormattedText.format( [ "The quick ", 129 | { text: "brown", style: "color: brown" }, 130 | " fox jumps over the lazy dog." 131 | ], { style: "color: blue" } ); 132 | ``` 133 | 134 | A wide range of inline layout-related CSS is supported as style input to the formatting functions. 135 | 136 | Styles specified in the second parameter apply to all text runs except where the specific style 137 | property is also specified (overridden) on an individual text object (in the first parameter). 138 | 139 | ### Specifying constraints 140 | 141 | The inline-size constraint is the 3rd parameter for both formatting functions. `format` has 142 | an additional 4th parameter for the block-size constraint. Because the `format` function 143 | produces all the lines of text at once, the block-size constraint (param 4) allows the author to 144 | specify a maximum bound for the container in both directions. 145 | 146 | To specify an inline-direction constraint for line wrapping with the `lines` iterator: 147 | 148 | ```js 149 | // Specify the desired default line wrapping distance (in the inline direction) of 150 pixels for each 150 | // line that will be produced by the lines() iterator. 151 | FormattedText.lines( "The quick brown fox jumps over the lazy dog.", null, 150 ); 152 | ``` 153 | 154 | For `format`, an additional block-size constraint is provided via a 4th parameter: 155 | 156 | ```js 157 | // Wrap any text that exceeds 150 pixels, and constrain to 300 pixels in the block direction 158 | FormattedText.format( "The quick brown fox jumps over the lazy dog.", null, 150, 300 ); 159 | ``` 160 | 161 | The 3rd (and for `format`) 4th parameters are `unsigned long` values. Omitted values are assumed 162 | to be infinite. 163 | 164 | The layout flow via `writing-mode` and `direction` style properties can only be set in the 2nd 165 | parameter (global styles for all the text) and this value (or the default value) is used to 166 | determine the orientation of the inline and block constraint values*. For example, 167 | when the value is `horizontal-tb` (e.g., Latin-based languages) the 3rd parameter (inline-size 168 | width constraint) is a horizontal constraint for line wrapping; overflow of the block-size 169 | constraint (4th parameter) will occur in the vertical direction. For specified writing-modes such as 170 | `vertical-rl` (e.g., Chinese) the inline-size constraint affects the vertical direction, with overflow 171 | occurring horizontally. 172 | 173 | *HTML will handle `writing-mode` set on _inline_ elements (when the result is an orthogonal 174 | writing mode direction) by "blockifying" the inline container into an inline-block in order to support the 175 | writing mode. This conversion from inline content to inline block is not supported in the Formatted 176 | Text input model, and thus paragraphs of text with nested orthogonal writing mode directions are not 177 | supported. 178 | 179 | **Note**: [Issue 43: What should constraining the block-progression direction do?](https://github.com/WICG/canvas-formatted-text/issues/43) tracks an unresolved issue about allowing constraints in both directions. 180 | 181 | ## Formatting Text line by line 182 | 183 | As an alternative to `format`, the **`lines`** formatting function will prepare an iterator that progressively 184 | formats the provided input line by line. The lines are returned by the iterator directly, as `FormattedTextLine` 185 | instances, without any container structure (e.g., not contained within a `FormattedText` instance as `format` 186 | produces). Because there is no container structure, web authors are responsible for using the metrics exposed 187 | on each line to manually position them as desired. 188 | 189 | ```js 190 | // Format the input text as specified, with line-breaks at 350 pixels. 191 | let iter = FormattedText.lines( [ "The quick ", 192 | { text: "brown", style: "color: brown; font-weight: bold" }, 193 | " fox jumps over the lazy dog." 194 | ], "font-style: italic", 350 ); 195 | // Use for..of to advance the iterator and get each line. 196 | for ( let line of iter ) { 197 | // TODO: lookup metrics info on the line instance and/or prepare to render it in a custom location 198 | } 199 | ``` 200 | 201 | ### Custom line breaking 202 | 203 | In many scenarios, authors may want to customize the inlineSize constraint applied to each line as the iterator 204 | runs. The iterator returned by `lines` has a read/write property called `inlineSize` which specifies where the 205 | next line's inlineSize break opportunity should be. It's default value is the value provided in the `lines` 206 | function (parameter 3). The `inlineSize` property can be set to change the value used by the iterator. If the 207 | `inlineSize` value is changed, the change is "sticky" and will persist for subsequent lines until changed again. 208 | 209 | For example, if there is some external shape that text needs to wrap around, the lines iterator can be adjusted 210 | at the appropriate times to shorten or lengthen the `inlineSize` constraint so that the lines can be positioned 211 | properly (line positioning must be done manually). 212 | 213 | ![Image of a cat in the upper-left corner of a text box, with text flowing to the image's right side and continuing below it.](explainerresources/Available-Width.png) 214 | 215 | ```js 216 | // Caller provides text to format, a CSS font string, width/height constraints, 217 | // a box location/dimensions, and a rendering function callback 218 | // that accepts a `FormattedTextLine` (result from the lines iterator). 219 | function wrapAroundFloatLeftBox( text, cssFont, constraints, box = { width: 200, height: 200, marginRightBottom: 10 }, renderFunc ) { 220 | 221 | // Prepare the input text and get a line iterator 222 | // (Set the default inlineSize for line-wrapping to the known constrained space.) 223 | let formattedTextIterator = FormattedText.lines( text, cssFont, constraints.width - box.marginRightBottom - box.width ); 224 | 225 | // sum of line metrics (collected from line instances) 226 | let lineHeightTotal = 0; 227 | // horizontal position offset for line placement (starts offset) 228 | let offsetX = box.marginRightBottom + box.width; 229 | 230 | // Start iterating lines... 231 | for ( let line of formattedTextIterator ) { 232 | 233 | // Render the line at the current offsets 234 | renderFunc( line, offsetX, lineHeightTotal ); 235 | 236 | // Prepare for the next line (using line metrics) 237 | lineHeightTotal += line.height; 238 | 239 | // Determine if the offsetX position needs to change 240 | if ( lineHeightTotal > ( box.marginRightBottom + box.height ) ) { 241 | offsetX = 0; 242 | formattedTextIterator.inlineSize = constraints.width; // Change the iterator's line-break position for subsequent lines... 243 | } 244 | } 245 | } 246 | ``` 247 | 248 | *Note*, the iterator's [Symbol.iterator] function (used by the iterator protocol in for..of loops) 249 | is a self-reference to the same iterator object, allowing convenient use inside of for..of loops. 250 | 251 | ### Rewinding line iteration 252 | 253 | In some scenarios (for example, an algorithm for a balanced multi-column line layout, shown later), 254 | it is sometimes necessary to "roll-back" and reprocess a line that was previously produced by the 255 | iterator to adjust its line-break constraints. 256 | 257 | ⚠️**Issue**: traditional JS iterator protocol does not expect rewinding behavior. This needs careful 258 | review and consideration for any unexpected language side-effects. 259 | 260 | The `lines` iterator keeps track of the number of lines that it has produced so far via the `lineCount` 261 | readonly property. In the event that the web author needs to revisit a prior line, the iterator can be 262 | `reset` back to a line `<=` the current iterator's `lineCount` value. Any line instances that were 263 | previously provided via the iterator following the line that the iterator is reset to are "discarded" 264 | (no longer usable). The `reset` function takes a singe integer parameter. If the number is zero or 265 | positive, the iterator is reset to the specified line index. 266 | 267 | ```js 268 | formattedTextIterator.reset(0); // Rewinds the iterator to before the first line (the beginning) 269 | 270 | formattedTextIterator.reset(1); // Rewinds the iterator to just after the first line (start of the second line) 271 | ``` 272 | A negative value resets the iterator to a relative number of lines from the current `lineCount` value. 273 | 274 | ```js 275 | // Assuming formattedTextIterator.lineCount == 1 276 | formattedTextIterator.reset(-1); // Rewinds the iterator by 1 line to the beginning (line 0). lineCount will be 0. 277 | 278 | // Assuming formattedTextIterator.lineCount == 5 279 | formattedTextIterator.reset(-2); // Rewinds the iterator by 2 lines to the end of line 3 (start of line 4). lineCount will be 3 280 | ``` 281 | 282 | The following example implements a naïve multi-column line balancer, where the text lines should be 283 | (approximately) balanced between columns of differing widths. 284 | 285 | ```js 286 | // Text line positioner for two non-uniform columns 287 | // (assumes infinite block-size for each column) 288 | // (assumes uniform formatting for all the text) 289 | function multiColumnFiller( text, col1Box, col2Box, renderFunc ) { 290 | let col1_inlineSize = col1Box.width; 291 | let col2_inlineSize = col2Box.width; 292 | 293 | // What is the ratio of the first column's width compared to the total width? 294 | let ratio = col1_inlineSize / ( col1_inlineSize + col2_inlineSize ); 295 | // Naïvely then, 'ratio' of the total number of lines that will fit in col1 296 | // will stay in col1, the rest will be balanced to col2. 297 | 298 | // How many lines fit in col1 given its size? 299 | let lineIter = FormattedText.lines( text, null, col1_inlineSize ); 300 | let col1Lines = []; 301 | for ( let line of lineIter ) { // process all the lines and return them in an array 302 | col1Lines.push( line ); 303 | } 304 | 305 | // Rough-out the line at which to move the rest to column 2 306 | let cutLineIndex = Math.floor( ratio * lineIter.lineCount ); 307 | // Discard the lines (inclusive) after this cut line from column 1's lines. 308 | col1Lines.splice( cutLineIndex ); 309 | 310 | // Reset the iterator to the cut-line index, adjust the iterator's inlineSize 311 | // constraint and get the rest of the lines 312 | lineIter.reset( cutLineIndex ); 313 | lineIter.inlineSize = col2_inlineSize; 314 | let col2Lines = []; 315 | for ( let line of lineIter ) { 316 | col2Lines.push( line ); 317 | } 318 | 319 | // Assign positions for each of the lines in both columns and render (left-justified) 320 | // First column location 321 | let xPos = col1Box.x; 322 | let yPos = col1Box.y; 323 | for ( let line of col1Lines ) { 324 | renderFunc( line, xPos, yPos ); 325 | yPos += line.height; 326 | } 327 | // Second column location 328 | xPos = col2Box.x; 329 | yPos = col2Box.y; 330 | for ( let line of col2Lines ) { 331 | renderFunc( line, xPos, yPos ); 332 | yPos += line.height; 333 | } 334 | } 335 | ``` 336 | 337 | ## Comparison to HTML 338 | 339 | The formatting functions are used to drive the web platform's layout engine, therefore, the output of 340 | `format` and `lines` should be the equivalent to what can be already performed in HTML using simple 341 | elements like `div` and `span`. The following two expressions are functionally equivalent, with the 342 | exception that the result of `format` has not been rendered (how to 343 | [render it is described separately](explainer-rendering.md)): 344 | 345 | ```js 346 | FormattedText.format( [ "The quick ", 347 | { text: "brown", style: "color: brown; font-weight: bold" }, 348 | " fox jumps over the lazy dog" 349 | ], null, 150, 100 ); 350 | ``` 351 | 352 | ```html 353 |
354 | 355 | The quick 356 | brown 357 | fox jumps over the lazy dog 358 | 359 |
360 | ``` 361 | 362 | Above, the `
` element is the container for an inline formatting context which sets the 363 | available width and height in which to layout the content, and the `` gets 364 | any meta object styling (in this case, nothing), and contains the formatting for the text runs. 365 | The text run styles on the word "brown" are applied to its immediate containing span. 366 | 367 | Another illustrative comparison shows how style on the meta object provided to `format` could be 368 | visualized: 369 | 370 | ```js 371 | FormattedText.format( [ "The quick ", 372 | { text: "brown", style: "color: brown" }, 373 | " fox jumps over the lazy dog." 374 | ], { style: "color: blue" } ); 375 | ``` 376 | 377 | ```html 378 |
379 | 380 | The quick 381 | brown 382 | fox jumps over the lazy dog 383 | 384 |
385 | ``` 386 | 387 | ## CSS to achieve advanced scenarios 388 | 389 | ### Vertical Text 390 | 391 | By leveraging existing CSS styles for writing modes and other related properties, 392 | `format` can support a variety of vertical text scenarios. For example, by simply 393 | leveraging the `writing-mode` property we can orient text in its traditional vertical 394 | direction: 395 | 396 | ```js 397 | let bold = "font-weight: bold"; 398 | let meta = "writing-mode: vertical-rl; font-size: 36pt"; 399 | FormattedText.format( [ "不怕慢，", { text: "就怕站", style: bold } ], meta, 200 ); 400 | ``` 401 | 402 | [When rendered](explainer-rendering.md), and constrained as indicated, this will render as: 403 | 404 | Characters of an ancient Chinese proverb, vertically oriented in two columns, the second column bold 405 | 406 | In combination with other related CSS properties, many other vertical text layouts are 407 | possible: 408 | 409 | ```js 410 | let bold = "font-weight:bold"; 411 | let styles = `writing-mode: vertical-lr; 412 | text-orientation: upright; 413 | line-height: 2; 414 | text-align: center; 415 | font-size: 12pt`; 416 | FormattedText.format( [ "It's better to make slow progress", 417 | { text: " than no progress at all", style: bold } 418 | ], styles, 250 ); 419 | ``` 420 | 421 | This might render as: 422 | 423 | The text 'It's better to make slow progress than no progress at all' rendered vertically from left-to-right in five columns 424 | 425 | ### Reusing styles (`FormattedTextStyle`) 426 | 427 | When style text strings are passed to the `format` function, they must be parsed into 428 | CSS properties (including verifying valid syntax). This process is usually fast but not 429 | free. It is likely that when preparing to format many text strings, or when repeatedly 430 | calling `format` in performance critical code paths, there are opportunities to re-use 431 | sets of CSS styles as a unit. 432 | 433 | A new write-once object is introduced to collect and cache these styles. It makes use 434 | of a [StylePropertyMapReadOnly](https://drafts.css-houdini.org/css-typed-om/#stylepropertymapreadonly) 435 | to reflect the parsed values for read-only inspection following construction: 436 | 437 | ```js 438 | // Save the upright text styles from the previous example for later re-use 439 | let styles = new FormattedTextStyle( `writing-mode: vertical-lr; 440 | text-orientation: upright; 441 | line-height: 2; 442 | text-align: center` ); 443 | styles.styleMap.has( "text-orientation" ); // returns true 444 | styles.styleMap.size; // returns 4, the number of declarations in the map 445 | for ( let [prop, val] of styles ) { // Enumerate everything in the map 446 | console.log( `${prop}: ${val}` ); 447 | } 448 | ``` 449 | 450 | The `FormattedTextStyle` object can be used in all the places in the data model that 451 | accept a style text string. For example the following are equivalent: 452 | 453 | ```js 454 | let bold = "font-weight: bold"; 455 | let vertical = "writing-mode: vertical-rl"; 456 | FormattedText.format( [ "不怕慢，", { text: "就怕站", style: bold } ], vertical ); 457 | 458 | let reusableBold = new FormattedTextStyle( "font-weight: bold" ); 459 | let reusableVertical = new FormattedTextStyle( "writing-mode: vertical-rl" ); 460 | FormattedText.format( [ "不怕慢，", { text: "就怕站", style: reusableBold } ], reusableVertical ); 461 | ``` 462 | 463 | ### How much CSS should be supported? 464 | 465 | The `format` and `lines` functions supports various CSS properties that influence how the 466 | text's lines will ultimately be positioned. There are also many CSS properties that do not 467 | apply to text, that convert between typical text layout and other layouts, or that take 468 | normal flow content out of flow (e.g., `float`, `position`, `display`, etc.). For the purposes 469 | of a formatted text object model, not all CSS properties can or should be supported. The 470 | guidelines for what CSS to support and what not to support follow. 471 | 472 | ### Focus on text-related CSS properties 473 | 474 | We believe it makes sense to only support CSS properties that provide specific features 475 | for inline-level content (text) and the management of the text's container (the metadata 476 | parameter). Other properties, especially those that would change the layout characteristics 477 | of text objects from their assumed inline-level nature, will not be supported. In the data 478 | model we keep the semantics of text runs consistent with the CSS that can be applied. 479 | 480 | Applying the above principle means that `float` would not 481 | be supported because it has the effect of changing the object's computed display value 482 | from inline to block (taking it out-of-flow). `position: absolute` as a metadata property also takes 483 | the object out of the normal flow. Likewise, the `width` and `height` properties on specific text runs would 484 | be ignored because they do not apply to inline-level elements. Attempting to set these 485 | properties would have no effect. On the other hand, `padding` **would be supported** and 486 | applied to text runs in accordance with the rules of CSS inline layout, 487 | e.g., the inline-direction `padding` values are taken into account while the 488 | block-direction values are not. 489 | 490 | The container for text runs is an independent containing block. It is essentially a 491 | `display: flow-root` object that establishes an inline formatting context for its children. 492 | When used in the metadata parameter, properties like `top`, `right`, `bottom`, and `left` do 493 | not apply (i.e., the text run container acts as its own initial containing block). Positioning 494 | the resulting formatted text for the purpose of rendering in some context must be done separately. 495 | 496 | In some cases, we imagine it will be useful to allow `format` metadata properties to support 497 | some limited alternate layout container types where those alternate types provide unique text layout 498 | capabilities. For example, we expect to support an inner display type of `ruby` in order to 499 | become a Ruby container and enable the use of Ruby annotated layout. Other container types are 500 | not currently planned to initially support 501 | but are interesting long-term candidates to consider (e.g., multi-column containers created via 502 | the `columns` shorthand property), while still others are less-likely to be supported (e.g., `flex` 503 | and `grid` container types, which are less useful for formatted text scenarios). 504 | 505 | There are various CSS properties that provide helpful graphical emphasis to text that are 506 | also supported. These are for convenience in supporting common text formatting scenarios 507 | that would otherwise require detailed introspection of the object model's related metrics 508 | in order to correctly layout and render as desired with respect to the text. Because these 509 | features are already available in CSS layout engines and significantly ease author burden, 510 | many of these CSS properties will be supported. Some supported examples include: 511 | `text-decoration`, `text-shadow`, `box-shadow`, even `border`, `outline`, and limited 512 | `background` support (where the metrics and composition processing do not require external 513 | dependencies, such as image resources typically loaded by `url()` functions). 514 | 515 | ### Future extensions 516 | 517 | By leveraging CSS, we get the added benefit of a well-known constraint language for 518 | expressing box bounds and the expected behavior for content (in our case formatted text 519 | content) that overflows those bounds. `width` and `height` and corresponding `min-width` 520 | or `max-height` properties express the desired constraints for eventual line-box 521 | computation when it comes to rendering the object model or returning metrics. Similarly, 522 | `overflow` and `clip-path` can further ensure the text content expands or is clipped to 523 | fit the desired constraints. These properties would only apply in the metadata parameter 524 | of `format`. 525 | 526 | CSS continues to evolve, and it makes sense to extend relevant new CSS properties to 527 | this object model as they become a part of the web platform. For example, while not 528 | widely supported at the time of writing, support for `shape-inside` (CSS Shapes L2) 529 | and CSS Exclusions provide exciting growth opportunities for text using this model. 530 | 531 | ## Internationalization 532 | 533 | CSS provides various existing properties for handling internationalization of text, such 534 | as `writing-mode`, `direction`, `unicode-bidi`, and others. However, it does not have a 535 | property for expressing language (CSS Selectors provide `:lang(xx)` but this is not a 536 | property). 537 | 538 | Therefore, all text objects (and metadata objects) will support an optional `lang` property 539 | whose value will accept the set of supported values of the equivalent HTML `lang` attribute 540 | (or XML/XHTML `xml:lang` attribute). 541 | 542 | An example where the `lang` property is used to provide clarify on the text object directly: 543 | 544 | ```js 545 | FormattedText.format( { text: "不怕慢就怕站", lang: "zh-Hans" } ); 546 | ``` 547 | 548 | Or it can be applied to the metadata object (generally the preferred option unless there 549 | are multiple text runs of differing language): 550 | 551 | ```js 552 | FormattedText.format( [ "不怕慢", "就怕站" ], { lang: "zh-Hans", style: "color: red" } ); 553 | ``` 554 | 555 | ### Bidi Text 556 | 557 | No additional work is needed from web developers to support bidi text. At `format` time, bidi 558 | analysis is done on the input text which creates internal bidi runs if necessary. For example: 559 | 560 | ```js 561 | FormattedText.format( "Sample arabic بمدينة مَايِنْتْس، ألمانيا text.", "font: 30px Arial", { width: 350 } ); 562 | ``` 563 | 564 | Might produce the following rendered output: 565 | 566 | !["Wrapped text rendered in a canvas."](explainerresources/Example2.png) 567 | 568 | ## WebIDL 569 | 570 | ```webidl 571 | [Exposed=Window,Worker] 572 | interface FormattedText { 573 | static FormattedText format( ( DOMString or FormattedTextRun or sequence<( DOMString or FormattedTextRun )> ) text, 574 | optional ( DOMString or FormattedTextStyle or FormattedTextMetadata ) metadata, 575 | optional double inlineSize, optional double blockSize ); 576 | static FormattedTextIterator lines( ( DOMString or FormattedTextRun or sequence<( DOMString or FormattedTextRun )> ) text, 577 | optional ( DOMString or FormattedTextStyle or FormattedTextMetadata ) metadata, 578 | optional double inlineSize ); 579 | }; 580 | 581 | [Exposed=Window,Worker] 582 | interface FormattedTextStyle { 583 | constructor( ( DOMString or StylePropertyMapReadOnly or CSSStyleDeclaration ) styleText ); 584 | [SameObject] readonly attribute StylePropertyMapReadOnly styleMap; 585 | }; 586 | 587 | dictionary FormattedTextMetadata { 588 | ( DOMString or FormattedTextStyle) style; 589 | DOMString lang; 590 | }; 591 | 592 | dictionary FormattedTextRun : FormattedTextMetadata { 593 | DOMString text = ""; 594 | }; 595 | 596 | interface FormattedTextIterator { 597 | readonly attribute unsigned long lineCount; 598 | FormattedTextIterator [Symbol.iterator](); // returns self 599 | FormattedTextIteratorProtocolResult next(); 600 | attribute double inlineSize; 601 | void reset( long lineIndex ); 602 | }; 603 | 604 | dictionary FormattedTextIteratorProtocolResult { 605 | FormattedTextLine? value; 606 | boolean done; 607 | }; 608 | ``` 609 | 610 | ## Supported CSS Table 611 | 612 | We've compiled a list of text-related (or generally applicable) CSS properties that we believe make 613 | sense to support on metadata and text objects. This list is not exhaustive. 614 | For example, it does not include many of the new logical properites such as `inline-size` for 615 | `width`. This list is provided for potential testing purposes and to facilitate discussion. 616 | 617 | | CSS Property | metadata object | text object | inherits | Notes | 618 | |--------------|---------------|------------------|----------|-------| 619 | | background | ✔ | ✔ | no | background-attachment, background-origin, background-position, background-repeat, background-size operate on an external resource image and will be ignored. Background-image will only support `` functions | 620 | | border | ✔ | ✔ | no | | 621 | | border-image | ✔ | ✔ | no | border-image-source will only supports `` functions | 622 | | border-radius | ✔ | ✔ | no | | 623 | | box-decoration-break | ✔ | ✔ | no | | 624 | | box-shadow | ✔ | ✔ | no | | 625 | | box-sizing | ✔ | | no | | 626 | | clip-path | ✔ | ✔ | no | | 627 | | color | ✔ | ✔ | yes | | 628 | | direction | ✔ | ✔ | yes | | 629 | | display | ✔ | ✔ | no | Generally not supported, but may make exceptions, e.g., ruby | 630 | | font | ✔ | ✔ | yes | | 631 | | font-feature-settings | ✔ | ✔ | yes | | 632 | | font-kerning | ✔ | ✔ | yes | | 633 | | font-size-adjust | ✔ | ✔ | yes | | 634 | | height | ❌ | ❌ | no | | 635 | | hyphens | ✔ | ✔ | yes | | 636 | | letter-spacing | ✔ | ✔ | yes | | 637 | | line-break | ✔ | ✔ | yes | | 638 | | line-height | ✔ | ✔ | yes | | 639 | | margin | ✔ | ✔ | no | | 640 | | mask | ✔ | ✔ | no | mask-border-source will only supports `` functions | 641 | | mask-border | ✔ | ✔ | no | | 642 | | max-height | ❌ | ❌ | no | | 643 | | max-width | ❌ | ❌ | no | | 644 | | min-height | ❌ | ❌ | no | | 645 | | min-width | ❌ | ❌ | no | | 646 | | opacity | ✔ | ✔ | no | | 647 | | outline | ✔ | ✔ | no | | 648 | | overflow-wrap | ✔ | ✔ | yes | | 649 | | padding | ✔ | ✔ | no | | 650 | | tab-size | ✔ | ✔ | yes | | 651 | | text-align | ✔ | | yes | | 652 | | text-align-all | ✔ | | yes | | 653 | | text-align-last | ✔ | | yes | | 654 | | text-combine-upright | ✔ | ✔ | yes | | 655 | | text-decoration | ✔ | ✔ | no | | 656 | | text-emphasis | ✔ | ✔ | yes | | 657 | | text-indent | ✔ | | yes | | 658 | | text-justify | ✔ | ✔ | yes | | 659 | | text-orientation | ✔ | ✔ | yes | | 660 | | text-overflow | ✔ | | no | | 661 | | text-shadow | ✔ | ✔ | yes | | 662 | | text-transform | ✔ | ✔ | yes | | 663 | | text-underline-offset | ✔ | ✔ | yes | | 664 | | text-underline-position | ✔ | ✔ | yes | | 665 | | transform | ✔ | | no | | 666 | | transform-box | ✔ | | no | | 667 | | transform-origin | ✔ | | no | | 668 | | unicode-bidi | ✔ | ✔ | no | | 669 | | white-space | ✔ | ✔ | yes | | 670 | | width | ❌ | ❌ | no | | 671 | | word-break | ✔ | ✔ | yes | | 672 | | word-spacing | ✔ | ✔ | yes | | 673 | | word-wrap | ✔ | ✔ | yes | | 674 | | writing-mode | ✔ | ❌ | yes | | 675 | 676 | ### Limitations 677 | 678 | * **CSS pseudo-elements** (`::first-letter`, `::first-line`, `::before`, `::after`). Pseudo-elements 679 | would require some unique way to specify the style map for these. A workaround for `::first-letter` 680 | is relatively easy (separating it into a separate text runs, or implementing support for 681 | the related property `initial-letter`), but `::first-line` is harder to target given it depends on 682 | where the line breaks. 683 | * explicit values of `inherit` on CSS properties that aren't specified to inherit by default will not 684 | likely be supported. 685 | 686 | ## Accessibility Considerations 687 | 688 | While the input to the `format` or `lines` functions is not expected to be accessible (it's the web 689 | author's internal data model), the resulting text metrics output will be useful in providing the means 690 | to enable fully accessible scenarios. This will be described in greater detail in the text metrics 691 | explainer. 692 | 693 | ## The output of `format` 694 | The [next explainer](explainer-metrics.md) describes the output from the `format` and `lines` functions, 695 | e.g., the `FormattedText` and `FormattedTextLine` instance objects and related info. 696 | 697 | ## Contributors: 698 | [dlibby-](https://github.com/dlibby-), 699 | [ijprest](https://github.com/ijprest), 700 | [sushraja-msft](https://github.com/sushraja-msft), 701 | [travisleithead](https://github.com/travisleithead) 702 | -------------------------------------------------------------------------------- /explainer-metrics.md: -------------------------------------------------------------------------------- 1 | # Formatted Text - Metrics 2 | 3 | A representation of formatted text metrics for inline layout content: the result of the `format` 4 | or `lines` functions or [potentially] other APIs that extract formatted text metrics from other 5 | sources (e.g., DOM nodes, layout children). 6 | 7 | For a general overview of the feature, see the repo's [readme](README.md). 8 | You can also learn more about the [formatted text data model](explainer-datamodel.md) and 9 | [how to render it](explainer-rendering.md). 10 | 11 | # Use cases 12 | 13 | ## 1. Use case: text placement 14 | 15 | This use case is the most basic use case we can imagine--identifying the placement of some 16 | Formatted Text into a view layer (like Canvas). Placement needs two things, a reference 17 | coordinate (x/y) and size metrics (bounding box of width/height). 18 | 19 | * Metrics provide the final shaped and formatted text width and height. 20 | 21 | Authors ensure rendered text is properly constrained to fit in the space provided by adjusting 22 | `font-size`, line width, line-spacing, etc., on the input text objects. 23 | 24 | ## 2. Use Case: line placement and custom per-line lengths 25 | 26 | In this case, the author would like to specify per-line constraints and intends to render 27 | each line iteratively (such as for captions), or with custom spacing such as to fit into a 28 | unique layout or flow (or handle inline gaps such as for figures that flow with the text). 29 | The `lines` iterator provides access to one or all lines given a line length constraint. 30 | 31 | Metrics provide: 32 | * Formatted line objects with width and height, including... 33 | * Pointers back to the input characters for the bounds positions of each line. 34 | 35 | ## 3. Use Case: specific glyph placement 36 | 37 | **Note**: this use case may be dropped. 38 | 39 | ⚠🚧 We would like to validate this use case for Canvas 2D scenarios. For WebGL scenarios, we 40 | understand the key information needed for rendering is the given shaped font's glyph id and 41 | glyph advance information. Is a Canvas 2d rendering API needed? A sketch of how this might 42 | work follows. 43 | 44 | Metrics provide: 45 | * List of shaped glyph metrics per fragment (fragment is a unit of glyphs that all share the 46 | same format/font/bidi/etc.). 47 | * Pointers back to the input characters for each glyph's bounds. 48 | 49 | Authors would use the `FormattedTextFragment` (holder of glyph's shaped information), 50 | and glyph info (index within that fragment or ID within the font) to render it. 51 | 52 | ## 4. Use Case: Editing: Rendering a selection over text (and placing/moving a caret) 53 | 54 | Many of the scenarios behind the chosen metrics are based on common editing use cases. An editing 55 | surface must provide a visual view and the means to move insertion points, selection ranges, etc., 56 | by responding to various input including pointing devices and keyboard. In order to support these 57 | input modalities, the metrics supplied by the explainer chiefly provide the means of understanding 58 | the relationships between parts of text as it was laid out (the glyphs that make up segments of 59 | like-formatted runs called "fragments" in this proposal, lines, etc.) and the relatiionship between 60 | these metrics and their source text objects. 61 | 62 | Metrics provide: 63 | 64 | * Position objects that map line/fragment/glyph indexes to input data model runs/offsets. 65 | (Map from text metrics to data model.) 66 | * API for obtaining position objects given input data model runs/offsets. (Map from 67 | data model to text metrics.) 68 | * API for obtaining position objects given (x,y) offsets relative to the `FormattedText`. 69 | (Map for mouse/touch/pen input to text metrics and data model.) 70 | * Access to formatted line objects with width/height (bounding box) and offsets from their 71 | container. 72 | * Access to formatted fragments within lines with width/height (bounding box) and offset from 73 | their container (if a selection needs to be tightly bound around the formatted glyph runs 74 | inside of lines). 75 | * Access to glyph width/height (bounding box) and offset from the fragment container. 76 | 77 | 78 | # Overview: data model to metrics to rendering 79 | 80 | The `FormattedText` constructor's static method `format` (described in the 81 | [data model](explainer-datamodel.md)) returns an instance of a `FormattedText` object, which is 82 | a container object for all the lines of text that have been formatted. The `FormattedText` 83 | constructor's static method `lines` returns an iterator used to get individual `FormattedTextLine` 84 | objects, which are "container-less" lines. Both the `FormattedText` and `FormattedTextLine` instances 85 | are metrics objects, retaining the formatted structure of the input text and offer various APIs for 86 | getting metrics and bounds (described later). 87 | 88 | The `FormattedText` is a container for all the input data model's metrics. It contains the APIs 89 | to get additional lines, fragments, and glyph information. The object hierarchy is shown below (note 90 | the image shows lines in a horizontal writing mode--but vertical writing modes are supported): 91 | 92 | ![A FormattedText box contains four horizontal FormattedTextLine objects. Each line object contains one or more FormattedTextFragment objects. Each fragment object is a container for glyph information.](explainerresources/metrics-structure.png) 93 | 94 | The `FormattedText` and `FormattedTextLine` objects may be rendered independently. We propose APIs 95 | to render them in the [Rendering explainer](explainer-rendering.md). 96 | 97 | ## Thinking ahead: future integration into DOM or Houdini Layout API 98 | 99 | ⚠🚧 WARNING: This section is entirely speculative, and out of scope for now. We include it here 100 | to ponder extended use cases in which these metrics could be applicable in the wider web platform. 101 | (And not to lose track of them in the design process.) 102 | 103 | The opportunity to get detailed metrics for formatted text is not exclusively tied to scenarios 104 | where DOM is potentially unavailable or impractical to use. We would like to ensure that we design 105 | for the possibility of integration into both DOM and Layout API scenarios as well. 106 | 107 | We envision APIs similar to `format`, that could also extract formatted text metrics but for 108 | Elements. In the DOM, a given `Node` already has a layout (when attached to the tree) that includes 109 | a Layout box model and is already constrained by the viewport and the hierarchy of nested layouts in 110 | which it resides. A new API something like `getFormattedText()` would return the formatted 111 | text metrics for an Element (acting like the `innerText` or `textContent` getters, but with context 112 | of the line formatting, relative placement of the lines and other metrics). Another approach might be 113 | extending a related API such as [`getClientRects()`](https://drafts.csswg.org/cssom-view/#dom-element-getclientrects) 114 | with line metric information. 115 | 116 | In the [Houdini Layout API](https://drafts.css-houdini.org/css-layout-api-1/), while processing a 117 | `layout`, `LayoutFragment` objects can represent a line of text (or a fragment from a line). 118 | In these situations, it might make sense to extend the `LayoutFragment` by combining it with a 119 | `FormattedTextLine` metrics object. This would provide extra information about the intra-line fragments 120 | and glyph information, potentially allowing advanced positioning of glyphs within a line-layout pass. 121 | 122 | | ⚠🚧 Ideas for integration into other parts of the platform | Description | 123 | |---|---| 124 | | myElement.`getFormattedText`() | Similar to `format`. TBD on scope of how this would work 😊 | 125 | | `extDOMRect`.`textFragments`[`i`] | Alternative DOM integration point that extends `getClientRects()` such that each rectangle gets the `FormattedTextLine` mixin or some such. | 126 | | `extLayoutFrag`.`textFragments`[`i`] | Array of `FormattedTextFragments` (see equivalent functionality in a `FormattedTextLine` object). | 127 | 128 | # Description of objects 129 | 130 | ## Line container - `FormattedText` 131 | 132 | Obtained as the result of `FormattedText.format`. It provides: 133 | 134 | * width/height (total bounding box for all lines). 135 | * array of `FormattedTextLine` objects. 136 | * a coordinate system for its lines (see section below). 137 | * Input-to-output mapping APIs 138 | * Getting a position for a given character/text run of the input text (position 139 | objects described below). 140 | * Pointer-to-output mapping APIs (hit testing) 141 | * Getting a position from an x/y coordinate pair (where the x/y coordinates 142 | are relative to the `FormattedText` object's coordinate system. 143 | 144 | | APIs on `FormattedText` | Description | 145 | |---|---| 146 | | .`width` | Returns the bounding-box width for all lines. This value may represent the bounds of the contained line's longest length or total height depending on the `writing-mode` and `direction` used to format the line. | 147 | | .`height` | Returns the bounding-box height for all lines. Independent of `writing-mode` as `width` above. | 148 | | .`lines`[] | An array of `FormattedTextLine` objects representing a sequence of heterogeneous formatted text in the inline flow. | 149 | | .`getPosition`(`textIndex`, `charIndex`) | Given a reference to an in input character (and the text object offset; 0 if only one string of text provided), returns a `FormattedTextPosition` of the associated place in the output metrics. | 150 | | .`getPositionFromPoint`(`x`,`y`,`findNearest`) | For mapping pointer positions into glyphs. `findNearest` might ensure that a `FormattedTextPosition` is always returned regardless of the coordinate value, whereas otherwise, `null` might be returned if not strictly over a glyph. | 151 | 152 | ### Coordinate systems 153 | 154 | Some APIs on the `FormattedText` and `FormattedTextLine` objects depend on coordinate systems for 155 | hit-testing, such as the `getPositionFromPoint` API, which takes an x/y pair. To simplify the logic 156 | for developers when translating between input pointer coordinates in viewport space and the coordinate 157 | space used by `FormattedText` objects, each `FormattedText` object has a *pointer origin* or *pointer 158 | coordinate space* with its origin in the upper-left corner of the bounding box rectangle that contains 159 | all the lines of text. For pointer tracking, all `x`/`y`/`width`/`height` values for the 160 | `FormattedTextLine`, etc. objects it contains are absolute values relative to the origin of the 161 | pointer-origin coordinate space. Because `FormattedTextLine`s that come from the `lines` iterator do 162 | not have a `FormattedText` container, the pointer-origin for each such line is the upper-left corner 163 | of the bounding box rectangle of the individual line itself. 164 | 165 | The pointer-origin coordinate space is not the most developer friendly for computing line offsets 166 | and positions in orthogonal writing modes. For that reason, `FormattedTextLine`, `FormattedTextFragment`, 167 | etc., objects also have a *writing mode origin* or *writing mode coordinate space* which is the 168 | origin of the block and inline progression directions for the chosen writing mode. 169 | `inlineSize`/`blockSize` provide sizing that is relative to the writing mode. For lines obtained from 170 | `format` (that have a `FormattedText` container) their `inlineOffset`/`blockOffset` values are relative 171 | to the writing mode origin for the set of lines formatted using that writing mode. Lines obtained from the 172 | `lines` iterator will always have a zero `inlineOffset` and `blockOffset` value. 173 | 174 | For example, in a series of lines formatted with `writing-mode: vertical-rl` and `direction: ltr`, the 175 | first line is on the right side of the pointer coordinate space, vertically oriented. The first 176 | line's origin in pointer coordinate space is the upper-left corner of the line's bounding box at 177 | position (`x`, `y`). The first line's origin in writing mode coordinate space is the upper-right 178 | corner of the line's bounding box. (Its `inlineOffset`, `blockOffset` values may be non-zero if the line 179 | is centered or justified.) The line's `blockSize` (i.e., line height) corresponds with the `width` value in the 180 | pointer coordinate space, while its `inlineSize` (i.e., line length) corresponds with the `height` 181 | value in pointer coordinate space. 182 | 183 | ![illustration of the various offset and size values for vertical text](explainerresources/coordinates.png) 184 | 185 | 186 | ## Positions – `FormattedTextPosition` 187 | 188 | Inspired by the 189 | [TextPosition](https://github.com/google/skia/blob/main/site/docs/dev/design/text_shaper.md#access-the-results-of-shaping-and-formatting) 190 | design, a `FormattedTextPosition` object provides the mapping between the formatted snapshot of the data 191 | model (e.g., the root, lines, fragments and glyphs) and the data model itself, which contains the 192 | source characters (Unicode code points) and their CSS-styled text runs. Positions always connect glyphs 193 | with characters and contain all necessary indexes to navigate the object structures to get between glyphs 194 | and characters. Like the rest of the formatted objects, positions are snapshots, and not updated as the 195 | model changes. (So don't change the model until you're done using a `FormattedTextPosition`!) 196 | 197 | ![A FormattedTextPosition bridges between the data model (with its potential array of text objects) and the root, line, fragment and glyph metrics objects. The position has text run references and character offsets to identify a precise location in the data model, and index values for the line, fragment, and glyph referenced by the associated character.](explainerresources/text-position-relationship.png) 198 | 199 | To find the relevant character(s) in the data model, positions have: 200 | 201 | * Text index which text object is being referenced (index of zero when only one text object was specified in `format`) 202 | * Character offset (start and end) since some glyphs are formed from a sequence of characters 203 | (Unicode combining characters, ligatures, etc., which vary by Font). 204 | 205 | To find the relevant glyph in the metrics objects, positions have: 206 | * Line index (within the `FormattedText`) 207 | * Fragment index (within the line) 208 | * Glyph index (within the fragment) 209 | 210 | There are some interesting cases to explore when it comes to Glyph mapping from source characters. 211 | These are covered below in the fragments section. 212 | 213 | | APIs on `FormattedTextPosition` | Description | 214 | |---|---| 215 | | .`sourceIndex` | An index of the input text object or string that contains the relevant character(s). The term `source` is generic in order to potentially support `Text` nodes as well in the future. **Note**: will need to resolve what the index means for DOM trees with hierarchies. | 216 | | .`characterOffsetStart` | The offset (unsigned long) within `source` that the associated glyph derives from (the start of the range if more than one character). | 217 | | .`characterOffsetEnd` | See above. If only one character maps to a single glyph, the start and end offsets will be the same. | 218 | | .`lineIndex` | The index into the `FormattedText`'s `lines` array where the associated glyph can be found. | 219 | | .`fragmentIndex` | The index into the `FormattedTextLine`'s `textFragments` array where the associated glyph can be found. | 220 | | .`glyphIndex` | The index into the `FormattedTextFragment`'s `glyphs` array where the associated glyph can be found. | 221 | 222 | ## Lines – `FormattedTextLine` 223 | 224 | The line is the bounding box of all the formatted fragments contained within the line (it has no 225 | formatting itself). All fragments contained within the line are wholly contained within (no fragments 226 | exist simultaneously in multiple lines). Note: due to justification or other inline alignment properties 227 | of the line, the line sizes and offsets may vary. If contained by a `FormattedText` instance, the line's 228 | offsets are relative to the origin of its "parent" `FormattedText` object. 229 | 230 | The line provides: 231 | 232 | * coordinate positions: x, y, inlineOffset, blockOffset 233 | * bounding boxes: width, height, inlineSize, blockSize 234 | * array of `FormattedTextFragment` objects. 235 | * ⚠🚧TODO: add dominant baseline information? 236 | * Utility functions for getting the start position and end position of the characters that bookend 237 | the line (e.g., the ability to identify where the line starts and ends in the data model) 238 | 239 | | APIs on `FormattedTextLine` | Description | 240 | |---|---| 241 | | .`width` | The line's width in pointer coordinate space (upper-left corner of the `FormattedText` object's bounding box containing all lines). This value may represent the line's width or height depending on the `writing-mode` and `direction` used to format the line. | 242 | | .`height` | The line's height in pointer coordinate space. This value may represent the line's width or height depending on the `writing-mode` and `direction` used to format the line. | 243 | | .`x` | The line's x-offset in pointer coordinate space (if held by a `FormattedText` container). Otherwise zero. | 244 | | .`y` | The line's y-offset in pointer coordinate space (if held by a `FormattedText` container). Otherwise zero. | 245 | | .`inlineSize` | The line's length in the inline direction (i.e., the direction of adjacent characters in the line of text). This value will always be the line's "width" regardless of `writing-mode` and `direction`. | 246 | | .`blockSize` | The line's height in the block direction (opposite of inline direction). This value will always be the line's "height" regardless of `writing-mode` and `direction`. | 247 | | .`inlineOffset` | Returns the inline-direction offset from the line's writing mode origin (if held by a `FormattedText` container). Otherwise zero. | 248 | | .`blockOffset` | Returns the block-direction offset from the line's writing mode origin (if held by a `FormattedText` container). Otherwise zero. | 249 | | .`textFragments`[] | An array of `FormattedTextFragment` objects. | 250 | | .`getStartPosition`() | Returns a `FormattedTextPosition` of the glyph forming the "start" end of the line (left side for horizontal LTR). | 251 | | .`getEndPosition`() | Returns a `FormattedTextPosition` of the glyph forming the "end" of the line (right side for horizontal LTR). | 252 | 253 | ## Fragments – `FormattedTextFragment` 254 | 255 | The fragment is the smallest unit of same-formatted text in a line. Fragments always have consistent 256 | directionality (they are post-BIDI algorithm processed, where alternating bidi sequences are split 257 | into separate fragments). All glyphs within have the same font shaping properties applied. Because 258 | of these properties, the fragment metrics are quite similar to the information exposed through 259 | [`measureText()`](https://html.spec.whatwg.org/multipage/canvas.html#dom-context-2d-measuretext). 260 | Consequently, the `FormattedTextFragment` extends the 261 | [`TextMetrics`](https://html.spec.whatwg.org/multipage/canvas.html#textmetrics) interface. 262 | 263 | The fragment also holds all the glyph information. The glyph information is to be understood in the 264 | context of the fragment's format (its font metrics). 265 | 266 | Note: the `FormattedTextFragment` and future 267 | [Font Metrics API](https://drafts.css-houdini.org/font-metrics-api/#fontmetrics) could be combined, 268 | though in this context the fragment is guaranteed to have consistent font metrics, so things like 269 | the FontMetrics' `fonts` (a list) wouldn't apply. 270 | 271 | A Fragment object provides: 272 | * coordinate positions: x, y, inlineOffset, blockOffset 273 | * bounding boxes: width, height, inlineSize, blockSize 274 | * Everything on HTML's 275 | [`TextMetrics`](https://html.spec.whatwg.org/multipage/canvas.html#textmetrics) interface 276 | * ⚠🚧 Formatting result values (for font, etc.). Note: we would like to understand the use cases 277 | for some of these better. 278 | * Array of glyph information 279 | * ⚠🚧 Potentially add baseline information (relative to the line) depending on use case? 280 | * Utility functions for getting the start position and end position of the characters that bookend 281 | the fragment (e.g., the ability to identify where the fragment starts and ends in the data model). 282 | * Utility function for getting a position for an arbitrary glyph in the fragment 283 | 284 | | APIs on `FormattedTextFragment` | Description | 285 | |---|---| 286 | | .`x`, `y`, `inlineOffset`, and `blockOffset` | Same as `FormattedTextLine` definition, but with offsets referring to this fragment. | 287 | | .`inlineSize`, `blockSize`, `width`, and `height` | Same as `FormattedTextLine` definition, but with sizes referring to this fragment. | 288 | | .`glyphs`[] | An array of dictionary objects containing glyph info (see next section). | 289 | | .`getStartPosition`() | Returns a `FormattedTextPosition` of the glyph forming the "start" end of the fragment (direction-dependent). | 290 | | .`getEndPosition`() | Returns a `FormattedTextPosition` of the glyph forming the "end" of the fragment (right side for horizontal LTR). | 291 | | .`getGlyphPosition`(`index`) | Returns a `FormattedTextPosition` for the glyph at the given index (in the glyphs array). | 292 | | APIs from [`TextMetrics`](https://html.spec.whatwg.org/multipage/canvas.html#textmetrics) | `width` (total advance width for the fragment; duplicate of `inlineSize` for horizontal writing modes), `actualBoundingBoxLeft`, `fontBoundingBoxAscent`, etc. | 293 | | Formatting result information (fragment specific) | ⚠🚧 To discuss use cases for this data (and how to protect it for installed fonts `fontFamily`, `fontSize`, `direction`, `writingMode`). | 294 | 295 | ### Fun with source characters and glyph mappings 296 | 297 | To form glyphs, font files have processing rules that are font-specific that determine how 298 | characters are mapped to particular glyphs. The presentation of a glyph may be processed differently 299 | depending on the font. For base ASCII characters the mapping is typically trivial--one character is 300 | associated with one glyph. However, for combining characters (and similar), font behavior may vary. 301 | For example, a font may map `á` (U+00E1 Latin Small Letter A with Acute) to a single glyph geometry 302 | (statically or through a processing table). Another font might cause two glyphs (the decomposition of 303 | the glyph `á` into `a` U+0061 Latin Small Letter A and `◌́ ` U+0301 Combining Acute Accent) to be used 304 | and will alter the advance logic so that the comining character glyph is painted in the same 305 | location as the `a`. 306 | 307 | ⚠🚧 We expect that additional experimentation is required to work out the best API given the range of 308 | possibilities and font combinations that exist. 309 | 310 | Our early thoughts are to model all cases where multiple glyphs are used for combining characters 311 | as unique fragments. Thus, within a fragment we can assert that the set of glyphs with will always 312 | have positive advances. When a combining character (or string of them) is encountered, such a character 313 | would be placed into its own fragment object, and the offset of that fragment would overlap the previous 314 | fragment. 315 | 316 | In cases where combining characters result in a single glyph, the API is designed to naturally express 317 | a `FormattedTextPosition` that includes the start and end range of the contributing characters. 318 | 319 | ## Glyphs – JS Object (dictionary) 320 | 321 | Plain JS object with keys/values: 322 | 323 | * advance – to be understood in the writing mode direction. 324 | * id - this glyph's font-specific index code (into the font). May be `null` for installed fonts. 325 | 326 | ⚠🚧 There are lots of interesting metrics for glyphs; however, we'll want to understand use cases 327 | that necessitate exposing more information (e.g., `x`/`y` origin, `bearing`, bounding box geometry, 328 | applied kerning, actual geometry like Path2D data, raw `ImageData`, etc.). Modeling a glyph with a 329 | JavaScript object allows for easy extensibility. Alternatively, with only `advance` and `id`, this 330 | information could be expressed as arrays. 331 | 332 | ## Additional Accessibility Considerations for Metrics 333 | 334 | **Word bounds/breaks** - meta-data about "word" breaks opportunities are not exposed to the developer 335 | today. In the current propsoal, the identification of word breaks is left as an activity to be 336 | supported by author code. Note that word-breaking for the purpose of line wrapping and formatting 337 | is in scope for this feature. However once the layout has been calculated, author code will need 338 | to use heuristics in langugues that have natural word breaks (e.g., via spaces between words). 339 | Developer feedback on whether native metrics should be support for word-breaks is sought and may 340 | motivate additional work. 341 | 342 | ## Contributors: 343 | (in alphabetical order) 344 | 345 | [dlibby-](https://github.com/dlibby-), 346 | [ijprest](https://github.com/ijprest), 347 | [sushraja-msft](https://github.com/sushraja-msft), 348 | [travisleithead](https://github.com/travisleithead) 349 | -------------------------------------------------------------------------------- /explainer-rendering.md: -------------------------------------------------------------------------------- 1 | # Formatted Text - Rendering 2 | 3 | This explainer describes how to render formatted text. See also: 4 | [specifying the data model](explainer-datamodel.md) for formatting, and the 5 | [output results/ metrics](explainer-metrics.md). 6 | 7 | There may be multiple rendering targets. This document describes rendering formatted text to 8 | a 2D canvas. 9 | 10 | ## Prerequisites 11 | 12 | Rendering formatted text first requires raw text and formats to be laid out via the `format` API 13 | (see the [data model explainer](explainer-datamodel.md)). The results of `format` are then potential 14 | objects for rendering. As noted in the [output results/ metrics explainer](explainer-metrics.md), 15 | `format` results consist of `FormattedText`, `FormattedTextLine`, and `FormattedTextFragment` objects. 16 | Each of these objects can be rendered independently through the APIs described below. 17 | 18 | ## `drawFormattedText` 19 | 20 | A new API is added to the [CanvasText](https://html.spec.whatwg.org/multipage/canvas.html#canvastext) mixin. 21 | 22 | ```js 23 | // Draw formatted text accepts output from the format function 24 | context.drawFormattedText( FormattedText.format( "hello world!" ), 50, 50 ); 25 | ``` 26 | 27 | The second and third parameters are the `x` and `y` coordinates on the canvas to draw the formatted content. 28 | 29 | An example usage of `drawFormattedText`: 30 | 31 | ```js 32 | // Setup the data model 33 | let formattedText = FormattedText.format( [ "the quick ", 34 | { text: "brown", style: "font-weight:bold" }, 35 | "fox jumps over the lazy dog" 36 | ], "font:18pt Arial", 250 /* inline size constraint */ ); 37 | // Render to a canvas at (0, 50) 38 | context.drawFormattedText( formattedText, /*x*/0, /*y*/50 ); 39 | ``` 40 | 41 | This would produce the following output on the canvas: 42 | 43 | ![Wrapped text rendered in a canvas.](explainerresources/Example1.png) 44 | 45 | # QnA 46 | 47 | ## What are the initial values for CSS properties, esp. when used with the Canvas. Who wins? 48 | 49 | Initial values for CSS properties will be as-specified in CSS. 50 | 51 | ⚠🚧 Our current thinking is that the rendering of formatted text will completely ignore the current 52 | Canvas drawing state. Drawing formatted text is more similar to drawing a rectangular image than to 53 | stroking or filling text using `strokeText` and `fillText`. We welcome author feedback on this 54 | front and use cases where it might be useful to respect certain canvas state (i.e., current 55 | transform). 56 | 57 | ⚠🚧 Are there use cases for using `vh` units for paragraphs of text? 58 | 59 | ## What about a glyph drawing API? 60 | 61 | ⚠🚧 We are seeking to understand use cases for per-glyph drawing and what metrics information 62 | would be required to support it. 63 | 64 | ## Accessibility Considerations for Rendering 65 | 66 | Canvas is a persistant challenge for accessibility on the web today. Several efforts are underway, 67 | including a 68 | [promising solution](https://github.com/WICG/aom/blob/gh-pages/explainer.md#use-case-4-adding-non-dom-nodes-virtual-nodes-to-the-accessibility-tree) 69 | as part of the Accessible Object Model (AOM) family of proposals. 70 | 71 | Meanwhile, web developers are encouraged to use the canvas element's fallback content 72 | to provide HTML markup that describes the canvas' current state until or unless a better 73 | solution that utilizes the `FormattedText` directly is established. For now, the aggregate 74 | text of a `FormattedText` object's text runs should be placed in block-styled 75 | [flow content](https://html.spec.whatwg.org/multipage/dom.html#flow-content-2) (such 76 | as a `

` element), with formatted sections wrapped in appropriate 77 | [phrasing content](https://html.spec.whatwg.org/multipage/dom.html#phrasing-content-2) 78 | (such as `` and styled to match the `FormattedTextRun` formatting.) The 79 | markup should make use of 80 | [ARIA Live Regions](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Live_Regions) 81 | to be sure assistive technologies (ATs) detect and announce any dynamic changes. 82 | 83 | ## Contributors: 84 | [sushraja-msft](https://github.com/sushraja-msft), 85 | [travisleithead](https://github.com/travisleithead) 86 | -------------------------------------------------------------------------------- /explainerresources/Available-Width.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/WICG/canvas-formatted-text/cf206c1b2bf2a6c6a01f3f6a75609b3a9f4858f7/explainerresources/Available-Width.png -------------------------------------------------------------------------------- /explainerresources/Example1.html: -------------------------------------------------------------------------------- 1 | CanvasFormattedText samples 2 | 3 | 4 | Text wrap width: 5 | 41 | -------------------------------------------------------------------------------- /explainerresources/Example1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/WICG/canvas-formatted-text/cf206c1b2bf2a6c6a01f3f6a75609b3a9f4858f7/explainerresources/Example1.png -------------------------------------------------------------------------------- /explainerresources/Example2.html: -------------------------------------------------------------------------------- 1 | CanvasFormattedText samples 2 | 3 | 4 | Text wrap width: 5 | -------------------------------------------------------------------------------- /explainerresources/Example2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/WICG/canvas-formatted-text/cf206c1b2bf2a6c6a01f3f6a75609b3a9f4858f7/explainerresources/Example2.png -------------------------------------------------------------------------------- /explainerresources/coordinates.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/WICG/canvas-formatted-text/cf206c1b2bf2a6c6a01f3f6a75609b3a9f4858f7/explainerresources/coordinates.png -------------------------------------------------------------------------------- /explainerresources/metrics-structure.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/WICG/canvas-formatted-text/cf206c1b2bf2a6c6a01f3f6a75609b3a9f4858f7/explainerresources/metrics-structure.png -------------------------------------------------------------------------------- /explainerresources/text-position-relationship.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/WICG/canvas-formatted-text/cf206c1b2bf2a6c6a01f3f6a75609b3a9f4858f7/explainerresources/text-position-relationship.png -------------------------------------------------------------------------------- /explainerresources/vertical-text-cn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/WICG/canvas-formatted-text/cf206c1b2bf2a6c6a01f3f6a75609b3a9f4858f7/explainerresources/vertical-text-cn.png -------------------------------------------------------------------------------- /explainerresources/vertical-text-en.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/WICG/canvas-formatted-text/cf206c1b2bf2a6c6a01f3f6a75609b3a9f4858f7/explainerresources/vertical-text-en.png -------------------------------------------------------------------------------- /impl-chromium-notes.md: -------------------------------------------------------------------------------- 1 | # 🚧 Implementation Report 🚧 2 | 3 | ## Chromium experimental implementation 4 | 5 | Implementation is being updated to reflect recent updates made to the data model explainer. 6 | 7 | No implementation of the metrics objects. 8 | 9 | ### Supported CSS values 10 | 11 | Summary of CSS subset in blink/renderer/core/css/css_properties.json5, starting from [commit 2788582](https://chromium-review.googlesource.com/c/chromium/src/+/2788582/14/third_party/blink/renderer/core/css/css_properties.json5) 12 | 13 | | CSS Property | FormattedText | FormattedTextRun | inherits | Notes | 14 | |--------------|---------------|------------------|----------|-------| 15 | | background | | | no | Currently only painting the foreground layer | 16 | | border | | | no | | 17 | | border-image | | | no | border-image-source will only supports `` functions | 18 | | border-radius | | | no | | 19 | | box-decoration-break | | | no | | 20 | | box-shadow | | | no | | 21 | | box-sizing | | | no | | 22 | | clip-path | | | no | | 23 | | color | ✔ | ✔ | yes | | 24 | | direction | ✔ | ✔ | yes | | 25 | | display | | | no | Generally not supported, but may make exceptions, e.g., ruby | 26 | | font | `font-family`, `font-size`, `font-stretch`, `font-style`, `font-variant-ligatures`, `font-variant-caps`, `font-variant-east-asian`, `font-variant-numeric`, `font-weight`, `font-variant-settings` | same as `FormattedText` column | yes | | 27 | | font-feature-settings | ✔ | ✔ | yes | | 28 | | font-kerning | ✔ | ✔ | yes | | 29 | | font-size-adjust | | | yes | | 30 | | height | ✔ | | no | | 31 | | hyphens | | | yes | | 32 | | letter-spacing | | | yes | | 33 | | line-break | ✔ (and `-webkit-` prefix) | ❌ | yes | | 34 | | line-height | | | yes | | 35 | | margin | | | no | | 36 | | mask | | | no | mask-border-source will only supports `` functions | 37 | | mask-border | | | no | | 38 | | max-height | | | no | | 39 | | max-width | | | no | | 40 | | min-height | | | no | | 41 | | min-width | | | no | | 42 | | opacity | | | no | | 43 | | outline | | | no | | 44 | | overflow-wrap | | | yes | | 45 | | padding | | | no | | 46 | | tab-size | | | yes | | 47 | | text-align | ✔ | | yes | | 48 | | text-align-all | | | yes | | 49 | | text-align-last | ✔ | | yes | | 50 | | text-combine-upright | | | yes | | 51 | | text-decoration | `text-decoration-color`, `text-decoration-line`, `text-decoration-skip-ink`, `text-decoration-style`, `text-decoration-thickness` | same as `FormattedText` column | no | | 52 | | text-emphasis | | | yes | | 53 | | text-indent | ✔ | | yes | | 54 | | text-justify | ✔ | ❌ | yes | | 55 | | text-orientation | ✔ | ❌ | yes | | 56 | | text-overflow | ✔ | | no | | 57 | | text-shadow | ✔ | ✔ | yes | | 58 | | text-transform | ✔ | ✔ | yes | | 59 | | text-underline-offset | ✔ | ✔ | yes | | 60 | | text-underline-position | ✔ | ✔ | yes | | 61 | | transform | | | no | | 62 | | transform-box | | | no | | 63 | | transform-origin | | | no | | 64 | | unicode-bidi | | | no | | 65 | | white-space | | | yes | | 66 | | width | ✔ | | no | | 67 | | word-break | | | yes | | 68 | | word-spacing | | | yes | | 69 | | word-wrap | | | yes | | 70 | | writing-mode | ✔ | ❌ | yes | | 71 | -------------------------------------------------------------------------------- /meetings/2021-06-17.md: -------------------------------------------------------------------------------- 1 | ## Canvas Formatted Text - Community Meeting 2 | ### 17 June 2021, 1700 UTC 3 | 4 | | Meeting info | | 5 | |---|----| 6 | | Video conference link | https://meet.google.com/ckm-bxvj-wfq | 7 | | IRC | **#formatted-text** (irc.w3.org) | 8 | 9 | ### Agenda 10 | 11 | * Meeting boilerplate (scribe, [CEPC](https://www.w3.org/Consortium/cepc/)) 12 | * Introductions 13 | * Overview of explainer status 14 | * Use cases review 15 | * Additional use cases to consider? 16 | * Data model overview 17 | * Data model new issues/concerns? 18 | * Next steps 19 | 20 | 21 | ### Present 22 | - Travis: PM @Microsoft 23 | - noamh: Excel online, Eng @Microsoft 24 | - Yego: Flutter, Eng @Google 25 | - Koji: Lead for text layout @Google 26 | - Fernando: Tech manager for canvas on Chrome 27 | - Sayna: PM @Microsoft 28 | - Sushanth: Eng Mgr @Microsoft 29 | - Mike: Graphics engineer works on SKIA @Google 30 | 31 | ### Summary 32 | Use cases: 33 | - layout and text flow 34 | - shaping: a lower level solution that requires knowledge of data about glyph shapes and geometries. 35 | 36 | Concerns about font-files: 37 | - how to access details with glyph id without parsing font-files 38 | 39 | Discussed the intersection of future metrics API and [Shaping API](https://github.com/google/skia/blob/main/site/docs/dev/design/text_shaper.md) previously proposed by Mike. 40 | 41 | ### Next steps 42 | 43 | Land the drawing API (on the dev side) and writing a draft of the measure API explainer. 44 | 45 | ### Meeting Minutes 46 | ``` 47 | [09:55] == Sayna_ [~Sayna@8604d15a.public.cloak] has joined #formatted-text 48 | [09:55] * Travis waves at Sayna_ 49 | [09:55] <@Travis> present+ Sayna_ 50 | [10:03] == koji [~sid53200@8604d15a.public.cloak] has joined #formatted-text 51 | [10:03] == noamh [~noamh@8604d15a.public.cloak] has joined #formatted-text 52 | [10:03] == yjbanov [~yjbanov@8604d15a.public.cloak] has joined #formatted-text 53 | [10:06] scribe: Sayna_ 54 | [10:06] noamh: excel online, plans on using formatted text 55 | [10:08] <@Travis> yjbanov: works on Flutter(?) team 56 | [10:08] <@Travis> sushanth: Microsoft, implementing the explainer 57 | [10:09] <@Travis> koji: lead for text layout at Google 58 | [10:09] <@Travis> fernando: tech manager for canvas on Chrome 59 | [10:09] Mike: Graphics engine at Google 60 | [10:09] <@Travis> mike: work on SKIA at Google 61 | [10:10] Flutter, correct (flutter.dev) 62 | [10:12] @Travis: We wrote an explainer a year ago, and after feedback for Canvas Formatted text in 3 parts: 63 | [10:12] ... data mode, rendering, and text metrics. 64 | [10:13] ... Many applications on the web starting to use canvas, we need to handle multi line text and formatted text. 65 | [10:14] Fernando: Let's talk about the use case. For some of the applications that we have, we have a low level use case that doesn't care about layout. 66 | [10:15] ... may be a performance implication when you care about shaping and when you want to put the text whereever you want and you dont' care about layout engine being exposed? 67 | [10:17] Travis: repeating the question - there is an additional, more low level use case where we don't care how the text will flow, but care more about how the text glyphs/shape/geometry 68 | [10:19] yjbanov: generate data for rendering directly and skipping the first two stages of rendering. 69 | [10:20] Fernando: these two cases can come together - It would be good to take both use cases into account. 70 | [10:21] ... glyphID isn't the issue, but unsure 71 | [10:22] ... advanced position is a metric that peeps might care about. 72 | [10:24] Sushanth: layout vs pure shaping: we did initial prototype with SKIA, we ran into issues like 73 | [10:24] ... format ranges of text and render with SKIA, it would have problems with boundaries of text 74 | [10:26] mike: Formatted text is a complete-looking spec from input to markup with many fields - perhaps still incomplete - we have a parallel looking set of explainers similar to what we have here. We have a simpler idea of marking up a rich paragraph, we propose explicitly exposing the result of the shaping. 75 | [10:27] ... (?), we expose result of concrete runs, and the concrete glyph IDs, allow people to take output and render to canvas 2D 76 | [10:28] ... with the exposed data, they can do hit testing. It is the data that is important. You captured kernings, and more. 77 | [10:29] ... ask: while you have a fast path proposed here, expose the underlying results. Apply color effects that can't be expressed in CSS. 78 | [10:30] Travis: question, do we have Mike's parallel explainer here? 79 | [10:30] Sushanth: no it is not. 80 | [10:32] Travis: second question for Mike, you start with raw input (or even what Travis has already), you want an in-between state where browser has done some work in shaping, positioning, kerning, and the unit that you are imagining is like ? 81 | [10:33] https://github.com/google/skia/blob/main/site/docs/dev/design/text_shaper.md 82 | [10:33] https://github.com/google/skia/blob/main/site/docs/dev/design/text_overview.md 83 | [10:33] https://github.com/google/skia/blob/main/site/docs/dev/design/text_c2d.md 84 | [10:34] * Travis thank you! 85 | [10:34] The above are the Skia proposals/explainers for a text shaping API 86 | [10:37] https://github.com/google/skia/blob/main/site/docs/dev/design/text_shaper.md 87 | [10:44] <@Travis> fernando: I'm wondering if all the detail shown in that spec is actually necessary. 88 | [10:44] <@Travis> (discussion of fonts and potential privacy concerns) 89 | [10:44] koji: when input is given, open text spec produces a set of data which can be used to measure before css renders line breaking and layout. Shaping is quite heavy, it's often 20-80 % of layout total time. 90 | [10:45] Mike: this is why I want the browser to do shaping. 91 | [10:45] <@Travis> So avoiding multiple shaping passes sounds like an ideal design. 92 | [10:48] <@Travis> fernando: may be a way to satisfy the use case without needing to get into the details of fonts and their insides... 93 | [10:49] <@Travis> .. would like to avoid creating a font object 94 | [10:50] <@Travis> noamh: https://github.com/foliojs/fontkit 95 | [10:52] <@Travis> yjbanov: script has to do so much today: pull down harfbuzz, ICU data, font-parsing logic... so much! 96 | [10:52] <@Travis> .. would love to get the 80% done. 97 | [10:52] <@Travis> fernando: if you can avoid the font-file parsing, the spec becomes easier. 98 | [10:53] <@Travis> .. we can't have an API that just exposes glyph ids (and nothing else) 99 | [10:54] <@Travis> mike: have another part of the proposal that takes glyph ids, and draws the text from those glyphs. 100 | [10:55] <@Travis> fernando: would love to remove the need to parse the font. If we can do this, this would be better for users. 101 | [10:55] <@Travis> yjbanov: just having a hard time imagining how to do this with glyph ids. 102 | [10:56] <@Travis> sushanth: in WebGL, server offers an SDF where the glyph ids are useful... 103 | [10:57] <@Travis> SDF / signed distance field 104 | [10:58] <@Travis> fernando: there are scenarios where you have external information about glyph ids that don't require going through a font. 105 | [10:59] <@Travis> sushanth: it was more expensive to get all the metrics out of the canvas formatted text... 106 | [10:59] <@Travis> mike: yes, make it optional to pull the metrics out. 107 | [11:00] <@Travis> sushanth: there was also complexity in exposing line objects (for rendering). Have to worry about caching, etc. 108 | [11:00] <@Travis> .. our idea is to have a measureFormattedText to then pull out the metrics. 109 | [11:01] <@Travis> mike: if the proposal can export the raw data, that's ideal. 110 | [11:01] <@Travis> fernando: having a styleMap makes impl in workers impossible... 111 | [11:01] <@Travis> Sushanth: I think I can make it work (will re-work the change) 112 | [11:02] <@Travis> fernando: today you use .style 113 | [11:02] <@Travis> fernando: proposal relies heavily on CSS parsing... 114 | [11:03] <@Travis> .. most common way to express style is .style object 115 | [11:06] <@Travis> .. may be a reason to do it the styleMap (from CSSWG) 116 | [11:07] <@Travis> .. styleMap doesn't necessarily help the user know if a property is supported. 117 | [11:14] Correction to my words: I meant "OpenType spec" instead of "open text spec". Great if you can fix it in the minutes. 118 | [11:15] <@Travis> Will do. 119 | [12:01] * Zakim excuses himself; his presence no longer seems to be needed 120 | [12:01] == Zakim [zakim@7facbc0a.team.cloak] has left #formatted-text [] 121 | [12:05] == yjbanov [~yjbanov@8604d15a.public.cloak] has quit [Ping timeout: 180 seconds] 122 | [12:06] <@Travis> rrsagent, please make logs world-visible 123 | [12:06] I have made the request, Travis 124 | [12:06] <@Travis> rrsagent, please format the minutes 125 | [12:06] I have made the request to generate https://www.w3.org/2021/06/17-formatted-text-minutes.html Travis 126 | [12:08] <@Travis> rrsagent, make logs public 127 | [12:08] I have made the request, Travis 128 | [12:13] == Travis changed the topic of #formatted-text to: Next meeting poll: https://doodle.com/poll/zz5uub9stikeweaa#table 129 | [12:13] <@Travis> RRSAgent: bye 130 | [12:13] I see no action items 131 | [12:13] == RRSAgent [rrsagent@7facbc0a.team.cloak] has left #formatted-text [] 132 | [12:59] == noamh [~noamh@8604d15a.public.cloak] has quit [Ping timeout: 180 seconds] 133 | 134 | ``` 135 | -------------------------------------------------------------------------------- /meetings/2021-08-18.md: -------------------------------------------------------------------------------- 1 | ## Canvas Formatted Text - Community Meeting 2 | ### 11 August 2021, 1500 UTC 3 | ### 18 August 2021, 1500 UTC 4 | 5 | | Meeting info | | 6 | |---|----| 7 | | Video conference link | meet.google.com/bxm-snaa-pgr | 8 | | IRC | **#formatted-text** (irc.w3.org) | 9 | 10 | ### Agenda 11 | 12 | * Meeting boilerplate (scribe, [CEPC](https://www.w3.org/Consortium/cepc/)) 13 | * Overview of current status 14 | * Review draft [text metrics API](../explainer-metrics.md) and [rendering](../explainer-rendering.md) 15 | * Talk about use cases for per-glyph rendering 16 | 17 | ### Present 18 | 19 | * Aaron Krajeski - Google 20 | * Fernando Serboncini - Google 21 | * Heather Miller - Google 22 | * Julia Lavrova - Google 23 | * Mike Reed - Google 24 | * Mouad Debbar - Google 25 | * Sushanth Rajasankar - Microsoft 26 | * Travis Leithead - Microsoft 27 | * Yegor Jbanov - Google 28 | 29 | ### Summary 30 | 31 | * Biggest concern: ensure the data model (discussed last meeting) can run in workers--this is core to various requirements. 32 | * Clarity sought on the necessary parameters to `.format()`; is `inlineSize` enough? 33 | * Debate about the lifecycle model for metrics; will want to prototype and wait and see what mitigating steps are necessary. Leaning toward create a new set of metrics objects each call. 34 | * Glyph relationships with respect to the source text is a many-to-many relationship. Graphemes not covered. This is a lot of complexity (especially for editing scenarios and caret movement logic). Consider simplifying and focusing on rendering needs only (e.g., size and position) if possible. 35 | * In most N-to-N mappings, knowing **why** they are mapped that way is important (Arabic decomposition different from emoji) 36 | * Clarify the order that things are put into the lists (especially at the fragment level): visual order or sourc order? Will matter for bidi formatted runs. 37 | * Be sure to note the focus on downloaded fonts for low-level glyph id/font info; will be not-available for installed fonts (for privacy reasons) 38 | * For "render your own glyph" cases, glyph ID and actual selected font (when the font is a downloaded font) are critical pieces. 39 | * Consider indicating when font-selection may have failed to resolve the character (which can happen). Glyph id 0 is usually reserved for this. 40 | 41 | ### Next steps 42 | 43 | * Work on explainer design changes to the metrics doc given above feedback. 44 | * Work on prototype code to enable worker scenarios. 45 | 46 | ### Meeting Minutes 47 | 48 | [HTML formatted minutes](https://www.w3.org/2021/08/18-formatted-text-minutes.html) 49 | -------------------------------------------------------------------------------- /w3c.json: -------------------------------------------------------------------------------- 1 | { 2 | "group": [80485] 3 | , "contacts": ["travisleithead"] 4 | , "repo-type": "cg-report" 5 | } 6 | --------------------------------------------------------------------------------