├── .github └── FUNDING.yml ├── .gitignore ├── README.md ├── index.bs └── index.html /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | # These are supported funding model platforms 2 | 3 | github: [Ygg01] 4 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | index.bk -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Draft for XML5 proposal. 2 | ========== 3 | 4 | The original version of this proposal is hosted on [XML5 code repository](https://github.com/annevk/xml5). 5 | 6 | What's XML5 7 | ========== 8 | 9 | XML5 is simply put, a more relaxed version of XML syntax. Basically, take the best aspects of HTML(5) syntax and apply it to XML. What this means in practice is following: 10 | 11 | * DOCTYPE is simplified and optional. Also no known laughing bomb exploits. 12 | * Creates a detailed rules for state transitions, to simplify implementations. 13 | * Allows mixed content everywhere `A bold new world` becomes: 14 | ``` 15 | Tag('text') 16 | | 17 | +---Text('A ') 18 | | 19 | +---Tag('b') 20 | | | 21 | | +---Text('bold') 22 | | 23 | +---Text(' new world') 24 | ``` 25 | * Writing one of escaped character inside tags automatically escapes them `Tom & Jerry` when parsed becomes: 26 | ``` 27 | Tag('tag') 28 | | 29 | +---Text('Tom & Jerry') 30 | ``` 31 | * XML comments are a lot more flexible (and a bit more difficult to parse), i.e. `--` is allowed in comments. Nested comments are prohibited. 32 | 33 | What XML5 strive to do is, to drop all notion of well-formedness out of the window and replace it with HTML-like error handling. XML5 parser will be able to parse any XML 1.0 and XML 1.1 documents, but a valid XML5 document wouldn't be parseable by a XML 1.0 or even XML 1.1 parser. 34 | Motivation behind this is to allow easier parsing of XML generated using string concatenation and used often on the web. 35 | 36 | How to read this? 37 | ================= 38 | 39 | Here is the [rendered version](https://ygg01.github.io/xml5_draft/). 40 | 41 | Otherwise you can download and compile the source yourself 42 | 43 | 1. [Install bikeshed](https://github.com/tabatkins/bikeshed/blob/master/docs/install.md) 44 | 2. Clone the the repository ````$ git clone https://github.com/Ygg01/xml5_draft```` 45 | 3. Change to xml5_draft the repository ````$ cd xml5_draft```` 46 | 4. Run bikeshed ````$ bikeshed```` 47 | -------------------------------------------------------------------------------- /index.bs: -------------------------------------------------------------------------------- 1 |

   2 | Title: XML5 Standard
   3 | H1: XML5
   4 | Status: LS
   5 | Logo: https://resources.whatwg.org/logo.svg
   6 | Shortname: xml5
   7 | Level:1
   8 | Editor: Anne van Kesteren, Mozilla, < annevk@annevk.nl >
   9 | Abstract: XML with well-defined error handling.
  10 | Editor: Daniel Fath, Unaffiliated,  < daniel.fath7@gmail.com >
  11 | Group: WGORWHATEVER
  12 |

13 | 66 | 67 |

68 | Parsing XML documents 69 |

70 | 71 | This section and its subsection define the XML parser. 72 | 73 |

This specification defines the parsing rules for XML documents, whether they are syntactically correct or not. 74 | Certain points in the parsing algorithm are said to be parse errors. The handling for 75 | parse errors is well-defined: user agents must either act as described below when encountering such problems, or 76 | must terminate processing at the first error that they encounter for which they do not wish to apply the rules 77 | described below.

78 | 79 |

80 | Overview 81 |

82 | 83 | The input to the XML parsing process consists of a stream of octets which is converted to a stream of code points, which in turn are tokenized, and finally those tokens are used to construct a tree. 84 | 85 |

86 | Parse Errors 87 |

88 | 89 | This specification defines the parsing rules for XML5 documents, whether they are syntactically correct or not. 90 | Certain points in the parsing algorithm are said to be parse errors. 91 | The error handling for parse errors is well-defined (that's the processing rules described throughout this specification), 92 | but user agents, while parsing an HTML document, may abort the parser at the first parse error that they encounter for 93 | which they do not wish to apply the rules described in this specification. 94 | 95 | 96 | 97 | 98 | 101 | 104 | 105 | 106 | 107 | 108 | 109 | 114 | 115 | 116 | 117 | 122 | 123 | 124 | 125 | 129 | 130 | 131 | 132 | 136 | 137 | 138 | 139 | 143 | 144 | 145 | 146 | 149 | 150 | 151 | 152 | 156 | 157 | 158 | 159 | 162 | 163 | 164 | 165 | 170 | 171 | 172 | 176 | 177 | 178 | 179 | 182 | 183 | 184 | 185 | 188 | 189 | 190 |

99 \| Code 100 \|	102 \| Description 103 \|
abrupt-closing-of-empty-comment	This error occurs if the parser encounters an empty comment that is abruptly closed by a U+003E 110 \| (`>`) code 111 \| point (i.e., `<!-->` or `<!--->`). The parser behaves as if the comment is 112 \| closed correctly. 113 \|
abrupt-closing-xml-declaration	118 \| This error occur if the parser encounters an unclosed quote in XML declaration. E.g. `119 \| <?xml version="1?> 120 \|` 121 \|
colon-before-attr	This error occurs if the parser encounters a U+003A COLON (`:`) in tag after name but before 126 \| attribute name (e.g. `<tag :attr`). Attributes can have namespaces but U+003A COLON but 127 \| namespaces can't be empty. 128 \|
eof-in-cdata	133 \| This error occurs if the parser encounters the end of the input stream in a CDATA section. 134 \| The parser treats such CDATA sections as if they are closed immediately before the end of the input stream.. 135 \|
eof-in-comment	140 \| This error occurs if the parser encounters the end of the input stream in a comment. 141 \| The parser treats such comments as if they are closed immediately before the end of the input stream. 142 \|
eof-in-doctype	147 \| This error occurs if the parser encounters the end of the input stream in a DOCTYPE section. 148 \|
eof-in-tag	153 \| This error occurs if the parser encounters the end of the input stream in a start tag or an end tag 154 \| (e.g.,`<div id=`). Such a tag is ignored. 155 \|
eof-in-xml-declaration	160 \| This error occurs if the parser encounters the end of the input stream in a XML Declaration e.g. `<?xml` 161 \|
incorrectly-opened-comment	This error occurs if the parser encounters the `<!` code point sequence that is not 166 \| immediately 167 \| followed by two U+002D (`-`) code points and that is not the start of a DOCTYPE or a CDATA 168 \| section. 169 \|
invalid-xml-declaration	This error occurs if the parser encounters any code point sequence other than "`PUBLIC`" 173 \| and "`SYSTEM`" keywords after a DOCTYPE name. In such a case, the parser ignores any following 174 \| public or system identifiers 175 \|
missing-whitespace-before-doctype-name	This error occurs if the parser encounters a DOCTYPE keyword and name are not separated by ASCII whitespace. 180 \| (e.g. `<!DOCTYPE`) In this case the parser behaves as if ASCII whitespace is present. 181 \|
missing-doctype-name	This error occurs if the parser encounters a DOCTYPE that is missing a name (e.g., 186 \| `<!DOCTYPE>`). 187 \|

191 | 192 |

193 | Input stream 194 |

195 | 196 | The stream of Unicode characters that consists the input to the tokenization stage will be initially seen by the user agent as a stream of octets (typically coming over the network or from the local file system). The octets encode Unicode code points according to a particular encoding, which the user agent must use to decode the octets into code points. 197 | 198 |

Define how to find the encoding

199 |

Decide how to deal with null values

200 | 201 | 202 |

203 | Tokenization 204 |

205 | 206 | Implementations must act as if they used the following state machine to tokenise 207 | HTML. The state machine must start in the data state. Most states consume a 208 | single character, which may have various side-effects, and either switches the 209 | state machine to a new state to reconsume the current input character, or 210 | switches it to a new state to consume the next character, or stays in the same 211 | state to consume the next character. Some states have more complicated behavior 212 | and can consume several characters before switching to another state. In some 213 | cases, the tokenizer state is also changed by the tree construction stage. 214 | 215 | When a state says to reconsume a matched character in a specified state, that 216 | means to switch to that state, but when it attempts to consume the next input 217 | character, provide it with the current input character instead. 218 | 219 | The next input character is the first character in the input stream that has 220 | not yet been consumed or explicitly ignored by the requirements in this 221 | section. Initially, the next input character is the first character in the 222 | input. The current input character is the last character to have been consumed. 223 | 224 |

Decide how to deal with namespaces

225 | 226 |

228 | Data state 229 |

232 | 233 | Consume the next input character: 234 | 235 |

U+0026 AMPERSAND (&) 237 |: Switch to character reference in data state.
U+003C LESSER-THAN SIGN (<): Switch to the tag open state.
EOF: Emit an end-of-file token.
Anything else: Emit the current input character as character. Stay in this state.

248 |

251 | Character reference in data state 252 |

255 | Switch to the data state. 256 | 257 | Attempt to consume a character reference. 258 | 259 | If nothing is returned emit a U+0026 AMPERSAND character (&) token. 260 | 261 | Otherwise, emit character tokens that were returned. 262 |

265 | Tag open state 266 |

269 | Consume the next input character: 270 |

U+002F SOLIDUS (/): Switch to the end tag open state.
U+003F QUESTION MARK(?): Switch to the pi state.
U+0021 (!): Switch to the markup declaration state.
U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space)
U+003A (:)
U+003C LESSER-THAN SIGN (<)
U+003E GREATER-THAN SIGN (>)
EOF: Parse error. Emit a U+003C LESSER-THAN SIGN (<) character. 290 | Reconsume the current input character in the data state. 291 |
Anything else: Create a new tag token, then reconsume current input character 296 | in tag name state. 297 |

299 |

302 | End tag open state 303 |

306 | 307 | Consume the next input character: 308 |

U+003E GREATER-THAN SIGN (>): Emit a short end tag token and then switch to the data 311 | state. 312 |
U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space)
U+003C LESSER-THAN SIGN (<)
U+003A (:)
EOF: Parse error. Emit a U+003C LESSER-THAN SIGN (<) character 321 | token and a U+002F SOLIDUS (/) character token. Reconsume the current 322 | input character in the data state. 323 |
Anything else: Create an end tag token, then reconsume the current input character in the end tag name 328 | state. 329 |

331 |

334 | End tag name state 335 |

338 | 339 | Consume the next input character: 340 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Switch to the end tag name after state.
U+002F SOLIDUS (/): Parse error. Switch to the end tag name after state.
EOF: Parse error. Emit the start tag token and then reprocess the 351 | current input character in the data state. 352 |
U+003E GREATER-THAN SIGN (>): Emit the end tag token and then switch to the data 356 | state. 357 |
Anything else: Append the current input character to the tag name and stay in the 361 | current state. 362 |

364 |

367 | End tag name after state 368 |

371 | 372 | Consume the next input character: 373 |

U+003E GREATER-THAN SIGN (>): Emit the end tag token and then switch to the data state.
U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Stay in the current state.
EOF: Parse error. Emit the current token and then reprocess the 384 | current input character in the data state. 385 |
Anything else: Parse error. Stay in the current state.

390 |

393 | Tag name state 394 |

397 | Consume the next input character: 398 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Switch to the tag attribute name before state.
U+003E GREATER-THAN SIGN (>): Emit the start tag token and then switch to the data state.
EOF: This an eof-in-tag parse error. Emit the current token and then reprocess the 409 | current input character in the data state. 410 |
U+002F SOLIDUS (/): Set current tag to empty tag. Switch to the empty tag state.
Anything else: Append the current input character to the tag name and stay in the 417 | current state. 418 |

420 |

424 | Empty tag state 425 |

428 | Consume the next input character: 429 | 430 |

U+003E GREATER-THAN SIGN (>): Emit the current tag token as empty tag token and then switch to the 433 | data state. 434 |
Anything else: Reconsume in tag attribute value before state. 438 |

440 |

444 | Tag attribute name before state 445 |

448 | 449 | Consume the next input character: 450 | 451 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Stay in the current state.
U+003E GREATER-THAN SIGN(>): Emit the current token and then switch to the data state.
U+002F SOLIDUS (/): Set current tag to empty tag. Switch to the empty tag state.
U+003A COLON (:): This is a colon-before-attr parse error. Stay in the current state.
EOF: This is an eof-in-tag parse error. Emit the current token and then reprocess the 469 | current input character in the data state. 470 |
Anything else: Start a new attribute in the current tag token. Set that attribute's 474 | name to the current input character and its value to the empty string and 475 | then switch to the tag attribute name state. 476 |

478 |

482 | Tag attribute name state 483 |

486 | 487 | Consume the next input character: 488 | 489 |

U+003D EQUALS SIGN (=): Switch to the tag attribute value before state.
U+003E GREATER-THEN SIGN (>): Emit the current token as start tag token. Switch to the data 495 | state. 496 |
U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Switch to the tag attribute name after state.
U+002F SOLIDUS (/): Set current tag to empty tag. Switch to the empty tag state.
EOF: This is an eof-in-tag parse error. Emit the current token as start tag token and 508 | then reprocess the current input character in the data 509 | state. 510 |
Anything else: Append the current input character to the current attribute's name. 514 | Stay in the current state. 515 |

517 | 518 | When the user agent leaves this state (and before emitting the tag token, 519 | if appropriate), the complete attribute's name must be 520 | compared to the other attributes on the same token; if there is already an 521 | attribute on the token with the exact same name, then this is a parse error 522 | and the new attribute must be dropped, along with the 523 | value that gets associated with it (if any). 524 | 525 |

529 | Tag attribute name after state 530 |

533 | 534 | Consume the next input character: 535 | 536 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Stay in the current state.
U+003D EQUALS SIGN(=): Switch to the tag attribute value before state.
U+003E GREATER-THEN SIGN(>): Emit the current token and then switch to the data state.
U+002F SOLIDUS (/): Set current tag to empty tag. Switch to the empty tag state.
EOF: This is an eof-in-tag parse error. Emit the current token and then reprocess the 553 | current input character in the data state. 554 |
Anything else: Start a new attribute in the current tag token. Set that attribute's 558 | name to the current input character and its value to the empty string and 559 | then switch to the tag attribute name state. 560 |

562 |

566 | Tag attribute value before state 567 |

570 | Consume the next input character: 571 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Stay in the current state.
U+0022 QUOTATION MARK ("): Switch to the tag attribute value double quoted state.
U+0027 APOSTROPHE ('): Switch to the tag attribute value single quoted state.
U+0026 AMPERSAND (&): 584 |: Reprocess the input character in the tag attribute value unquoted 585 | state. 586 |
U+003E GREATER-THAN SIGN(>): Emit the current token and then switch to the data state.
EOF: This is an eof-in-tag parse error. Emit the current token and then reprocess the 593 | current input character in the data state. 594 |
Anything else: Append the current input character to the current attribute's value and 598 | then switch to the tag attribute value unquoted state. 599 |

601 |

604 | Tag attribute value double quoted state 605 |

607 | Consume the next input character: 608 | 609 |

U+0022 QUOTATION MARK ("): Switch to the tag attribute name before state.
U+0026 AMPERSAND (&): Switch to character reference in attribute value state, with the 615 | additional allowed character being U+0022 QUOTATION MARK("). 616 |
EOF: This is an eof-in-tag parse error. Emit the current token and then reprocess the 620 | current input character in the data state. 621 |
Anything else: Append the input character to the current attribute's value. Stay in 625 | the current state. 626 |

628 |

632 | Tag attribute value single quoted state 633 |

635 | Consume the next input character: 636 |

U+0022 QUOTATION MARK ('): Switch to the tag attribute name before state.
U+0026 AMPERSAND (&): Switch to character reference in attribute value state, with the 642 | additional allowed character being APOSTROPHE ('). 643 |
EOF: This is an eof-in-tag parse error. Emit the current token and then reprocess the 647 | current input character in the data state. 648 |
Anything else: Append the input character to the current attribute's value. Stay in 652 | the current state. 653 |

655 |

659 | Tag attribute value unquoted state 660 |

662 | Consume the next input character: 663 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Switch to the tag attribute name before state.
U+0026 AMPERSAND (&): 670 |: 671 | Switch to character reference in attribute value state, with the 672 | additional allowed character being U+003E GREATER-THAN SIGN(>). 673 |
U+003E GREATER-THAN SIGN (>): Emit the current token as start tag token and then switch to the 677 | data state. 678 |
EOF: This is an eof-in-tag parse error. Emit the current token as start tag token and 682 | then reprocess the current input character in the 683 | data state. 684 |
Anything else: Append the input character to the current attribute's value. Stay in 688 | the current state. 689 |

691 |

695 | Pi state 696 |

699 | If the next few characters are: 700 |

Exact match for word "xml".: 703 | Consume those characters and switch to xml declaration state 704 |
U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space)
EOF: Parse error. Reconsume current input characters in the 711 | bogus comment state. 712 |
Anything else: Create a new processing instruction token. Reconsume current characters in pi 716 | target state. 717 |

719 |

722 | XML declaration state 723 |

726 | Consume the next input character: 727 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Stay in current state
U+0076 LATIN SMALL LETTER V (v)
U+0065 LATIN SMALL LETTER E (E)
U+0073 LATIN SMALL LETTER S (S): Reconsume current character in XML declaration attribute name state
737 | U+003F QUESTION MARK (?) 738 |: Switch to XML Declaration after state.
EOF: This is a eof-in-xml-declaration parse error. Append string "xml" 742 | to the processing instruction target, emit current processing instruction token and emit end-of-file 743 | token. 744 |
Anything else: This is an invalid-xml-declaration parse error. Append string "xml" 747 | to the processing instruction target, then reconsume current character in pi data state

749 |

752 | XML declaration attribute name state 753 |

756 | If the next few characters are: 757 |

Exact match for word "version".: 760 | Set current xml declaration attribute name to version. Switch to XML declaration attribute name 761 | after. 762 |
Exact match for word "encoding".: 766 | Set current xml declaration attribute name to encoding. Switch to XML declaration attribute name 767 | after. 768 |
Exact match for word "standalone".: 772 | Set current xml declaration attribute name to standalone. Switch to XML declaration attribute name 773 | after. 774 |
Anything else: This is an invalid-xml-declaration parse error. Switch to pi target state

779 |

782 | XML declaration attribute name after 783 |

786 | Consume the next input character: 787 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Stay in current state.
U+003D EQUALS SIGN (=): Switch to XML declaration attribute before value state.
EOF: This is an eof-in-xml-declaration parse error. Push to processing instruction target 798 | xml, then push to processing instruction data version=. 799 | Emit processing instruction token. 800 |
Anything else: This is an invalid-xml-declaration parse error. Push to processing instruction target 804 | xml, then push to processing instruction data version=. 805 | Reconsume in pi target state. 806 |

809 | 810 | 811 |

812 | XML declaration attribute before value state 813 |

814 | 815 |

816 | Consume the next input character: 817 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Stay in current state.
U+0027 APOSTROPHE ('): Switch to XML declaration attribute value (single-quoted) state.
U+0022 QUOTATION MARK ("): Switch to XML declaration attribute value (double-quoted) state.
EOF: This is an eof-in-xml-declaration parse error. Push to processing instruction target 831 | xml, then push to processing instruction data version=. 832 | Emit processing instruction token. 833 |
Anything else: This is an invalid-xml-declaration parse error. Push to processing instruction target 837 | xml, then push to processing instruction data version=. 838 | Reconsume in pi target state. 839 |

841 | 842 | 843 | 844 | 845 |

846 | XML declaration attribute value (single-quoted) state 847 |

848 | 849 |

850 | If the next few characters are: 851 |

U+0027 APOSTROPHE ('): Switch to XML declaration state.
U+003F QUESTION MARK (?): This is an abrupt-closing-xml-declaration parse error. 857 | Switch to XML Declaration after state. 858 |
EOF: This is an eof-in-xml-declaration parse error. Emit current xml declaration. 862 | Emit end-of-file token. 863 |
Anything else: This is an invalid-xml-declaration parse error. Switch to pi target state 867 |

869 |

870 | 871 |

872 | XML declaration attribute value (double-quoted) state 873 |

874 | 875 |

876 | If the next few characters are: 877 |

U+0022 QUOTATION MARK ("): Switch to XML declaration state.
U+003F QUESTION MARK (?): This is an abrupt-closing-xml-declaration parse error. 883 | Switch to XML Declaration after state. 884 |
EOF: This is an eof-in-xml-declaration parse error. Emit current xml declaration. 888 | Emit end-of-file token. 889 |
Anything else: This is an invalid-xml-declaration parse error. Switch to pi target state 893 |

895 |

896 | 897 |

898 | XML declaration after state 899 |

900 | 901 |

902 | If the next few characters are: 903 |

U+003E GREATER-THAN SIGN (>): Emit the xml declaration token and then switch to the data state.
U+003F QUESTION MARK(?): Append the current input character to the PI's data and stay in the 909 | current state. 910 |
Anything else: Reprocess the current input character in the pi data 914 | state. 915 |

917 |

918 | 919 |

920 | Pi target state 921 |

922 | 923 |

924 | 925 | Consume the next input character: 926 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Switch to the pi target after state.
EOF: Parse error. Emit the current processing instruction token and then reprocess the 934 | current input character in the data state. 935 |
U+003F QUESTION MARK(?): Switch to the pi after state.
Anything else: Append the current input character to the processing instruction target and stay in the 942 | current state. 943 |

945 |

946 | 947 |

948 | Pi target after state 949 |

950 | 951 |

952 | 953 | Consume the next input character: 954 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Stay in the current state.
Anything else: Reprocess the current input character in the pi data 962 | state. 963 |

965 |

966 | 967 |

968 | Pi data state 969 |

970 | 971 |

972 | 973 | Consume the next input character: 974 |

U+003F QUESTION MARK(?): Switch to the pi after state.
EOF: This is a eof-in-cdata parse error. Emit the current processing instruction token 980 | and then 981 | reprocess the 982 | current input character in the data state. 983 |
Anything else: Append the current input character to the pi's data and stay in the 987 | current state. 988 |

990 |

991 | 992 |

993 | Pi after state 994 |

995 | 996 |

997 | 998 | Consume the next input character: 999 |

U+003E GREATER-THAN SIGN (>): Emit the current token and then switch to the data state.
U+003F QUESTION MARK(?): Append the current input character to the PI's data and stay in the 1005 | current state. 1006 |
Anything else: Reprocess the current input character in the pi data 1010 | state. 1011 |

1013 |

1014 | 1015 |

1016 | Markup declaration state 1017 |

1018 | 1019 |

1020 | 1021 | If the next few characters are: 1022 | 1023 |

Two U+002D HYPEN-MINUS characters (-): Consume those two characters, create a comment token whose data is the empty string and switch 1026 | to comment 1027 | start state. 1028 |
Exact match for word "DOCTYPE": Consume those characters and switch to Doctype state
Exact match for word "[CDATA[" with a (the five uppercase letters 1034 | "CDATA" with a U+005B LEFT 1035 | SQUARE BRACKET character before and after) 1036 |: Consume those characters and switch to CDATA state
Anything else: Emit an incorrectly-opened-comment parse error. Create a comment token whose data 1041 | is an 1042 | empty string. 1043 | Switch to bogus comment state 1044 | (don't consume any characters) 1045 |

1047 |

1048 | 1049 |

1050 | Comment start state 1051 |

1052 | 1053 |

1054 | Consume the next input character: 1055 | 1056 |

U+002D HYPHEN-MINUS (-): Switch to comment start dash state
U+003E GREATER-THAN SIGN (>): This is an abrupt-closing-of-empty-comment parse error. Switch to data 1062 | state. 1063 | Emit the current comment token. 1064 |
Anything else: Reconsume in the comment state

1069 |

1070 | 1071 |

1072 | Comment start dash state 1073 |

1074 | 1075 |

1076 | Consume the next input character: 1077 | 1078 |

U+002D HYPHEN-MINUS (-): Switch to comment end state
U+003E GREATER-THAN SIGN (>): This is an abrupt-closing-of-empty-comment parse error. Switch to data 1084 | state. 1085 | Emit the current comment token. 1086 |
EOF: This is an eof-in-comment parse error. Emit the comment token. Emit an 1090 | end-of-file-token. 1091 |
Anything else: Append a U+002D HYPHEN-MINUS character (-) to the comment token's data. 1095 | Reconsume in the comment state. 1096 |

1098 |

1099 | 1100 | 1101 |

1102 | Comment state 1103 |

1104 | 1105 |

1106 | 1107 | Consume the next input character: 1108 | 1109 |

U+003C LESS-THAN SIGN (<): Append the current input character to the comment token's data. 1112 | Switch to the comment less-than sign state. 1113 |
U+002D HYPHEN-MINUS (-): Switch to the comment end dash state.
EOF: This is an eof-in-comment parse error. Emit the current comment token. Emit an 1120 | end-of-file 1121 | token. 1122 |
Anything else: Append the current input character to the comment token's data.

1127 |

1128 | 1129 | 1130 |

1131 | Comment less-than sign state 1132 |

1133 | 1134 |

1135 | 1136 | Consume the next input character: 1137 | 1138 |

U+0021 EXCLAMATION-MARK (!): Append the current input character to the comment token's data. 1141 | Switch to the comment less-than sign bang state. 1142 |
U+003C LESS-THAN SIGN (<): Append the current input character to the comment token's data. 1146 |
Anything else: Reconsume in the comment state.

1151 |

1152 | 1153 |

1154 | Comment less-than sign bang state 1155 |

1156 | 1157 |

1158 | Consume the next input character: 1159 | 1160 |

U+002D HYPHEN-MINUS (-): Switch to the comment less-than sign bang dash state.
Anything else: Reconsume in the comment state.

1167 |

1168 | 1169 |

1170 | Comment less-than sign bang dash state 1171 |

1172 | 1173 |

1174 | Consume the next input character: 1175 | 1176 |

U+002D HYPHEN-MINUS (-): Switch to the comment less-than sign bang dash dash state.
Anything else: Reconsume in the comment end dash state.

1183 |

1184 | 1185 |

1186 | Comment less-than sign bang dash dash state 1187 | 1188 |

1189 | 1190 |

1191 | Consume the next input character: 1192 | 1193 |

U+003E GREATER-THAN-SIGN (>)
EOF: Reconsume in the comment end state.
Anything else: Parse error.Reconsume in the comment end state. 1200 |

1202 |

1203 | 1204 |

1205 | Comment end dash state 1206 |

1207 | 1208 |

1209 | 1210 | Consume the next input character: 1211 | 1212 |

U+002D HYPHEN-MINUS (-): Switch to the comment end state.
EOF: Parse error. Emit the comment token. Emit an end-of-file token.
Anything else: Append a U+002D HYPHEN-MINUS (-) to the comment's token 1221 | data. Reconsume in the comment state. 1222 |

1224 |

1225 | 1226 |

1227 | Comment end state 1228 |

1229 | 1230 |

1231 | 1232 | Consume the next input character: 1233 | 1234 |

U+003E GREATER-THAN SIGN (>): Switch to the data state.Emit the comment token.
U+0021 EXCLAMATION MARK(!): Switch to the comment end bang state.
U+002D HYPHEN-MINUS (-): Append a U+002D HYPHEN-MINUS character (-) to the comment 1243 | token's data. 1244 |
EOF: Parse error. Emit the comment token. Emit an end-of-file token. 1248 |
Anything else: Append two U+002D (-) characters and the current input 1252 | character to the comment token's data. Reconsume in the comment 1253 | state. 1254 |

1256 |

1257 | 1258 |

1259 | Comment end bang state 1260 |

1261 | 1262 |

1263 | Consume the next input character: 1264 | 1265 |

U+002D HYPHEN-MINUS (-): Append a U+002D HYPHEN-MINUS character (-) and U+0021 EXCLAMATION MARK 1268 | character(!) to the comment token's data. Switch to the comment end dash 1269 | state. 1270 |
U+003E GREATER-THAN SIGN (>): Parse error. Switch to the data state.Emit the comment token.
EOF: Parse error. Emit the comment token. Emit an end-of-file token.
Anything else: Append two U+002D (-) characters and U+0021 EXCLAMATION MARK 1280 | character(!) to 1281 | the comment token's data. Reconsume in the comment 1282 | state. 1283 |

1285 |

1286 | 1287 |

1288 | CDATA state 1289 |

1290 | 1291 |

1292 | 1293 | Consume the next input character: 1294 | 1295 |

U+005D RIGHT SQUARE BRACKET (]): Switch to the CDATA bracket state.
EOF: Parse error. Reprocess the current input character in the 1301 | data state. 1302 |
Anything else: Emit the current input character as character token. Stay in the 1306 | current state. 1307 |

1309 |

1310 | 1311 |

1312 | CDATA bracket state 1313 |

1314 | 1315 |

1316 | 1317 | Consume the next input character: 1318 | 1319 |

U+005D RIGHT SQUARE BRACKET (]): Switch to the CDATA end state.
EOF: Parse error. Reprocess the current input character in the 1325 | data state. 1326 |
Anything else: Emit a U+005D RIGHT SQUARE BRACKET (]) character as character token and also 1330 | emit the current input character as character token. Switch to CDATA bracket state. 1331 |

1333 |

1334 | 1335 | 1336 |

1337 | CDATA end state 1338 |

1339 | 1340 |

1341 | 1342 | Consume the next input character: 1343 | 1344 |

U+003E GREATER-THAN SIGN (>): Switch to the data state.
U+005D RIGHT SQUARE BRACKET (]): Emit the current input character as character token. Stay in the 1350 | current state. 1351 |
EOF: Parse error. Reconsume the current input character in the 1355 | data state. 1356 |
Anything else: Emit two U+005D RIGHT SQUARE BRACKET (]) characters as character tokens and 1360 | also emit the current input character as character token. Switch to the 1361 | CDATA state. 1362 |

1364 |

1365 | 1366 |

1367 | Character reference in attribute value state 1368 |

1369 | 1370 |

1371 | 1372 | Attempt to consume a character reference. 1373 | 1374 | If nothing is returned, append a U+0026 AMPERSAND (&) character to current attribute's value. 1375 | 1376 | Otherwise, append returned character tokens to current attribute's value. 1377 | 1378 | Finally, switch back to attribute value state that switched to this state. 1379 |

1380 | 1381 | 1382 |

1383 | Bogus comment state 1384 |

1385 | 1386 |

1387 | Consume the next input character: 1388 |

U+003E GREATER-THAN SIGN (>): Switch to the data state. Emit the current comment token.
EOF: Emit the comment. Emit an end-of-file token
Anything else: 1397 | Append the current input character to the comment token's data. 1398 |

1400 |

1401 | 1402 | 1403 |

1404 | Tokenizing character references 1405 |

1406 | 1407 |

U+0009 CHARACTER TABULATION (Tab)

U+000A LINE FEED (LF)

U+0020 SPACE (Space)

U+003C LESS-THAN SIGN (<)

U+0025 PERCENT SIGN (%)

U+0026 AMPERSAND (&)

EOF

The additional allowed character if there is one

Not a character reference. No characters are consumed and nothing is returned (This is not an 1432 | error, 1433 | either). 1434 |

U+0023 NUMBER SIGN (#) 1436 |

1437 | 1438 | Consume the U+0023 NUMBER SIGN. 1439 | 1440 | The behaviour further depends on the character after the U+0023 NUMBER SIGN. 1441 | 1442 |

U+0078 LATIN SMALL LETTER X

U+0078 LATIN CAPITAL LETTER X

1446 |

Consume the X.

1447 |

Follow the steps below, but using ASCII hex digits.

1448 |

When it comes to interpreting the number, interpret it as a hexadecimal number.

1449 |

Anything else

1452 | Follow the steps below, but using ASCII digits. 1453 | 1454 | When it comes to interpreting the number, interpret it as a decimal number. 1455 |

Should we do HTML like replacement? At least for null?

1475 | 1476 | Otherwise, if the number is in the range 0xD800 to 0xDFFF or is greater than 0x10FFFF, then this 1477 | is a 1478 | parse error. Return a U+FFFD REPLACEMENT CHARACTER character token. 1479 | 1480 |

Should we refuse Unicode from ranges listed (0x0001 to 0x0008, 0x000D to 1481 | 0x001F, 1482 | 0x007F to 0x009F, 0xFDD0 to 0xFDEF, or is one of 0x000B, 0xFFFE, 0xFFFF, 0x1FFFE, 0x1FFFF, 1483 | 0x2FFFE, 1484 | 0x2FFFF, 0x3FFFE, 0x3FFFF, 0x4FFFE, 0x4FFFF, 0x5FFFE, 0x5FFFF, 0x6FFFE, 0x6FFFF, 0x7FFFE, 1485 | 0x7FFFF, 1486 | 0x8FFFE, 0x8FFFF, 0x9FFFE, 0x9FFFF, 0xAFFFE, 0xAFFFF, 0xBFFFE, 0xBFFFF, 0xCFFFE, 0xCFFFF, 1487 | 0xDFFFE, 1488 | 0xDFFFF, 0xEFFFE, 0xEFFFF, 0xFFFFE, 0xFFFFF, 0x10FFFE, or 0x10FFFF)?
1489 | 1490 | I've noted that Javascript implementation of XML5 is having to go around some characters in 1491 | its 1492 | version. 1493 |

1494 |

Anything else

1497 | 1498 | Consume characters until you reach a U+003B SEMICOLON character (;). 1499 | 1500 |

What happens if there is no semicolon? Does it read rest of the file? Maybe 1501 | better 1502 | solution is to read all characters that are part of name char according to XML 1.1. spec.

1517 | If the markup contains following attribute This is a &ref;, character 1518 | tokenizer 1519 | should return this as a reference named ref. However if the attribute defined is defined as 1520 |

This
1521 |                         is &notref

, then the tokenizer will interpret this as a text

This is
1522 |                     &notref

, while emitting a parse error. 1523 |

1524 | 1525 |

1527 |

1528 | 1529 |

1530 | DOCTYPE state 1531 |

1532 |

1533 | Consume the next input character: 1534 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Switch to the before DOCTYPE name state.
EOF: Emit an eof-in-doctype parse error. Switch to data state.
Anything else: Emit an missing-whitespace-before-doctype-name parse error parse error. 1546 | Reconsume character in before DOCTYPE name state. 1547 |

1549 |

1550 | 1551 |

1552 | Before DOCTYPE name state 1553 |

1554 | 1555 |

1556 | Consume the next input character: 1557 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Ignore the character.
Uppercase ASCII letter: Create a new DOCTYPE token. Set the token name to lowercase version 1565 | of the current input 1566 | character. 1567 | Switch to the DOCTYPE name state. 1568 |
U+003E GREATER-THAN SIGN(>): This is a missing-doctype-name parse error. Create a new DOCTYPE token. 1572 | Emit DOCTYPE token. Switch to data state. 1573 |
EOF: This is eof-in-doctype parse error. Switch to data state.
Anything else: Create new DOCTYPE token. Set the token's name to current input character. Switch to DOCTYPE 1581 | name state. 1582 |

1584 |

1585 | 1586 |

1587 | DOCTYPE name state 1588 |

1589 | 1590 |

1591 | Consume the next input character: 1592 | 1593 |

U+0009 CHARACTER TABULATION (Tab)
U+000A LINE FEED (LF)
U+0020 SPACE (Space): Set doctype depth to 0. Switch to the after DOCTYPE name state.
Uppercase ASCII letter: Append the lowercase of current input character to current DOCTYPE 1601 | token. 1602 |
U+003E GREATER-THAN SIGN(>): Create a new DOCTYPE token. Emit token. Switch to data state.
EOF: This is eof-in-doctype parse error. Emit the current DOCTYPE token. 1609 | Emit an end-of-file token. 1610 |
Anything else: Append the current input character to the current DOCTYPE token's name. 1614 | Reconsume the EOF character. 1615 |

1617 |

1618 | 1619 |

1620 | After DOCTYPE name state 1621 |

1622 | 1623 |

1624 | Consume the next input character: 1625 | 1626 |

U+005B LEFT SQUARE BRACKET ([): Increase doctype depth by 1. Remain in current state.
U+005D RIGHT SQUARE BRACKET (]): If current doctype depth is 0 switch to Bogus doctype state, 1632 | otherwise decrease doctype depth by 1. Remain in current state. 1633 |
U+003E GREATER-THAN SIGN(>): If current doctype depth is 0, emit current doctype and switch to data state.
EOF: This is eof-in-doctype parse error. Switch to the data state. Emit DOCTYPE 1640 | token. 1641 | Emit an end-of-file token. 1642 |
Anything else: Remain in current state

1647 |

1648 | 1649 |

1650 | Bogus DOCTYPE state 1651 |

1652 |

1653 | Consume the next input character: 1654 |

U+003E GREATER-THAN SIGN(>): Switch to data state. Emit DOCTYPE token.
EOF: Emit DOCTYPE token. Emit the end-of-file token.
Anything else: Ignore character.

1664 |

1665 | 1666 | 1667 |

1668 | Tree construction 1669 |

1670 | 1671 | The input to the tree construction stage is a sequence of tokens from the 1672 | tokenization stage. The output of this stage is a tree model 1673 | represented by a Document object. 1674 | 1675 | The tree construction stage passes through several phases. The initial 1676 | phase is the start phase. 1677 | 1678 | The stack of open elements contains all elements of which the 1679 | closing tag has not yet been encountered. Once the first start tag token in 1680 | the start phase is encountered it will contain one open element. 1681 | The rest of the elements are added during the main phase. 1682 | 1683 | The current element is the bottommost node in this stack. 1684 | 1685 | The stack of open elements is said to have an element in 1686 | scope if the target element is in the stack of open elements. 1687 | 1688 | When the steps below require the user agent to append a 1689 | character to a node, the user agent must collect it 1690 | and all subsequent consecutive characters that would be appended to that node 1691 | and insert one Text node whose data is the concatenation of all 1692 | those characters. 1693 | 1694 | Need to define create an element for the token... 1695 | 1696 | When the steps below require the user agent to insert an element 1697 | for a token the user agent must create an element 1698 | for the token and then append it to the current element 1699 | and push it into the stack of open elements so 1700 | that it becomes the new current element. 1701 | 1702 |

Start phase

A start tag token: 1714 | 1715 | Create an element for the token and then append it to the 1716 | Document node and push it into the stack of open elements. 1717 | 1718 | This element is the root element and the first current 1719 | element. Then switch to the main phase. 1720 |
An empty tag token: 1724 | 1725 | Create an element for the token and append it to the 1726 | Document node. Then switch to the end phase. 1727 | 1728 |
A comment token: 1732 | 1733 | Append a Comment node to the Document node 1734 | with the data attribute set to the data given in the 1735 | token. 1736 | 1737 |
A processing instruction token: 1741 | 1742 | Append a ProcessingInstruction node to the 1743 | Document node with the target and data 1744 | attributes set to the target and data given in the token. 1745 | 1746 |
An end-of-file token: 1750 | 1751 | Parse error. Reprocess the token in the end 1752 | phase. 1753 | 1754 |
Anything else: 1758 | Parse error. Ignore the token. 1759 |

1761 | 1762 |

Main phase

A character token: 1772 | 1773 | Append a character to the current 1774 | element. 1775 | 1776 |
A start tag token: Insert an element for the token.
An empty tag token: Create an element for the token and append it to the 1783 | current element.
An end tag token: 1787 | 1788 | If the tag name of the current node does not match the tag 1789 | name of the end tag token this is a parse error. 1790 | 1791 | If there is an element in scope with the same tag name as 1792 | that of the token pop nodes from the stack of open elements 1793 | until the first such element has been popped from the stack. 1794 | 1795 | If there are no more elements on the stack of open elements at this point 1796 | switch to the end phase. 1797 |
A short end tag token: 1801 | 1802 | Pop an element from the stack of open elements. If there 1803 | are no more elements on the stack of open elements switch to the end 1804 | phase. 1805 | 1806 |
A comment token: 1810 | 1811 | Append a Comment node to the current element 1812 | with the data attribute set to the data given in the 1813 | token. 1814 | 1815 |
A processing instruction token: 1819 | Append a ProcessingInstruction node to the current 1820 | element with the target and data attributes 1821 | set to the target and data given in the token. 1822 | 1823 |
An end-of-file token: 1827 | Parse error. Reprocess the token in the end phase. 1828 | 1829 |

1831 | 1832 |

End phase

1835 | Tokens in end phase must be handled as follows: 1836 | 1837 |

A comment token: Append a Comment node to the Document node 1840 | with the data attribute set to the data given in the 1841 | token. 1842 |
A processing instruction token: 1846 | 1847 | Append a ProcessingInstruction node to the 1848 | Document node with the target and data 1849 | attributes set to the target and data given in the token. 1850 | 1851 |
An end-of-file token: 1855 | 1856 | Stop parsing. 1857 |
Anything else: 1861 | 1862 | Parse error. Ignore the token. 1863 |

1865 | 1866 |

1867 | 1868 |

Once the user agent stops parsing the 1869 | document, it must follow these steps: 1870 | 1871 |

TODO

1872 | 1873 |

1874 | Writing XML documents 1875 |

1876 | 1877 |

1878 | Common parser idioms 1879 |

1880 | 1881 | The ASCII digits are the characters in the range U+0030 DIGIT ZERO (0) to U+0039 DIGIT 1882 | NINE ( 1883 | 9). 1884 | 1885 | The ASCII hex digits are the characters in the ranges U+0030 DIGIT ZERO ( 1886 | 0) to U+0039 DIGIT NINE ( 1887 | 9), U+0041 LATIN CAPITAL LETTER A to U+0046 LATIN CAPITAL LETTER F, and U+0061 LATIN SMALL LETTER A 1888 | to U+0066 LATIN SMALL LETTER F. 1889 | 1890 | The lowercase ASCII 1891 | letters are characters in the range between U+0061 LATIN SMALL LETTER A to U+007A LATIN SMALL LETTER Z. 1892 | 1893 | The uppercase ASCII 1894 | letters are characters in the range between U+0041 LATIN CAPITAL LETTER A to U+005A LATIN CAPITAL LETTER 1895 | Z. 1896 | 1897 | Comparing two strings in an ASCII case-insensitive manner means comparing them exactly, code point for code 1898 | point, except that the characters in the range U+0041 to U+005A (i.e. LATIN CAPITAL LETTER A to LATIN CAPITAL LETTER Z) 1899 | and the corresponding characters in the range U+0061 to U+007A (i.e. LATIN SMALL LETTER A to LATIN SMALL LETTER Z) are 1900 | considered to also match. 1901 | --------------------------------------------------------------------------------

99 \| Code 100 \|	102 \| Description 103 \|
abrupt-closing-of-empty-comment	This error occurs if the parser encounters an empty comment that is abruptly closed by a U+003E 110 \| (`>`) code 111 \| point (i.e., `<!-->` or `<!--->`). The parser behaves as if the comment is 112 \| closed correctly. 113 \|
abrupt-closing-xml-declaration	118 \| This error occur if the parser encounters an unclosed quote in XML declaration. E.g. `119 \| <?xml version="1?> 120 \|` 121 \|
colon-before-attr	This error occurs if the parser encounters a U+003A COLON (`:`) in tag after name but before 126 \| attribute name (e.g. `<tag :attr`). Attributes can have namespaces but U+003A COLON but 127 \| namespaces can't be empty. 128 \|
eof-in-cdata	133 \| This error occurs if the parser encounters the end of the input stream in a CDATA section. 134 \| The parser treats such CDATA sections as if they are closed immediately before the end of the input stream.. 135 \|
eof-in-comment	140 \| This error occurs if the parser encounters the end of the input stream in a comment. 141 \| The parser treats such comments as if they are closed immediately before the end of the input stream. 142 \|
eof-in-doctype	147 \| This error occurs if the parser encounters the end of the input stream in a DOCTYPE section. 148 \|
eof-in-tag	153 \| This error occurs if the parser encounters the end of the input stream in a start tag or an end tag 154 \| (e.g.,`<div id=`). Such a tag is ignored. 155 \|
eof-in-xml-declaration	160 \| This error occurs if the parser encounters the end of the input stream in a XML Declaration e.g. `<?xml` 161 \|
incorrectly-opened-comment	This error occurs if the parser encounters the `<!` code point sequence that is not 166 \| immediately 167 \| followed by two U+002D (`-`) code points and that is not the start of a DOCTYPE or a CDATA 168 \| section. 169 \|
invalid-xml-declaration	This error occurs if the parser encounters any code point sequence other than "`PUBLIC`" 173 \| and "`SYSTEM`" keywords after a DOCTYPE name. In such a case, the parser ignores any following 174 \| public or system identifiers 175 \|
missing-whitespace-before-doctype-name	This error occurs if the parser encounters a DOCTYPE keyword and name are not separated by ASCII whitespace. 180 \| (e.g. `<!DOCTYPE`) In this case the parser behaves as if ASCII whitespace is present. 181 \|
missing-doctype-name	This error occurs if the parser encounters a DOCTYPE that is missing a name (e.g., 186 \| `<!DOCTYPE>`). 187 \|