Jason's homepage

` or `

`, instead they're the any text content inside of other tags. So if we have a node like `

Hi there

`, this will get parsed out to a p `Node`, whose `content` is `.childNodes(children)`, and `children` will be an array with a single `Node`, whose `content` is `.text("Hi there")`. This structure *feels weird*, but it allows us to parse more complicated nodes like `

Hi there, friend

`. In short, we're wrapping otherwise un-tagged text into a pretend `` tag and then treating it as we do other nodes. 494 | 495 | To parse a text run, we first consume every token that's not an `<` character, which we assume might be the beginning of a tag. If we find any contents, we then join the contents' body together into one big string. 496 | 497 | Then, we do some quick and dirty text replacement, replacing encoded html entities with their display characters and non-space whitespaces with spaces for display (this doesn't follow the html standard for whitespaces perfectly, but it works well enough). With all the replacement done, we return the text run node. 498 | 499 | ``` 500 | { 501 | try context.consumeBetween(leftToken: .openAngleBracket, rightToken: .closeAngleBracket) { 502 | try context.consume(tokenKind: .bang, feedback: "Expected comment to begin with a bang") 503 | try context.consume(tokenKind: .hyphen, feedback: "Expected comment to have a hyphen after the bang") 504 | try context.consume(tokenKind: .hyphen, feedback: "Expected comment to have two hyphens after the bang") 505 | 506 | var done = false 507 | while done == false { 508 | 509 | if context.currentToken.kind == .hyphen && context.nextToken.kind == .hyphen && context.nextNextToken.kind == .closeAngleBracket { 510 | 511 | try context.consume(tokenKind: .hyphen, feedback: "-") 512 | try context.consume(tokenKind: .hyphen, feedback: "-") 513 | done = true 514 | } else { 515 | try context.consume(where: { _ in true }, skipWhitespaceTokens: false, feedback: "consuming comment contents") 516 | } 517 | } 518 | 519 | return Node(element: InternalElement.comment, content: .voidNode, attributes: []) 520 | } 521 | }])}) 522 | ``` 523 | 524 | Finally, if the child node wasn't a normal node, nor a text run node, we see if it was perhaps a comment node, which takes the form ``. Looking inside angle brackets, we first attempt to consume a bang, then 2 hyphen tokens. After that, we loop, peeking at the next 3 tokens looking for the ending `-->` pattern. If we don't find that pattern, we just consume and ignore whatever content was there. Once we're done munching tokens, we return the internal comment node. 525 | 526 | ``` 527 | .filter { 528 | if $0.element == InternalElement.comment { return false } 529 | if $0.element != InternalElement.textRun { return true } 530 | 531 | // filter out empty text run nodes 532 | return $0.textContent?.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty == false 533 | } 534 | ``` 535 | 536 | As a very last step of parsing child nodes, we remove nodes that are comments or nodes that are text runs with empty text. Everything else, we keep. And now we're done parsing child nodes. 537 | 538 | ``` 539 | 540 | let endTag = try Tag.parse(context: context) 541 | guard endTag.isEnd else { 542 | throw NodeParseError.closingTagWasActuallyOpening(tagName: endTag.element) 543 | } 544 | 545 | guard startTag.element == endTag.element else { 546 | throw NodeParseError.closingTagDidNotMatchOpeningTag(opening: startTag.element, closing: endTag.element) 547 | } 548 | 549 | return .init( 550 | element: startTag.element, 551 | content: .childNodes(children), 552 | attributes: startTag.attributes 553 | ) 554 | } 555 | ``` 556 | 557 | After the child nodes are parsed, all that's left is to parse the end tag, make sure it's really an end tag, and ensure that it matches the start tag. If all of that succeeded, we return the fully constructed `Node`. Most of what we just did was bookkeeping (checking tags, make sure start / end tags match), and then parsing the node's children, if any. 558 | 559 | ### Tag 560 | 561 | We've papered over `Tag` parsing, though, so let's look at that now: 562 | 563 | ``` 564 | struct Tag: Parsable { 565 | 566 | let element: String 567 | let isEnd: Bool 568 | let attributes: [Attribute] 569 | 570 | var isVoidElement: Bool { 571 | ["area", "base", "br", "col", "embed", "hr", "img", "input", "link", "meta", "source", "track", "wbr"].contains(element) 572 | } 573 | ``` 574 | 575 | We start our `Tag` type with some properties, alluding to the `Attribute` type we'll see shortly as well. We also list the known void elements to determine if our element should be considered void. Now on to the parsing: 576 | 577 | ``` 578 | static func parse(context: ParsingContext, options: ParsingOptions?) throws -> Tag { 579 | try context.consumeBetween(leftToken: .openAngleBracket, rightToken: .closeAngleBracket) { 580 | let slashToken = try? context.consume(tokenKind: .forwardSlash, feedback: "Expected a `/`") 581 | let _ = try? context.consume(tokenKind: .bang, feedback: "Expected a `!`") 582 | ``` 583 | 584 | A tag is wrapped in `<` and `>` angle brackets. Within those, we first look for an initial forward slash token, and if we find it we assume we're parsing an end tag (we use `try?` to optionally parse this — if we don't find the slash, we're not considering that an error worth bailing from). We also look for an optional bang token and just completely ignore it if we find it (this is for the `` tag). 585 | 586 | ``` 587 | let identifier = try context.consume(tokenKind: .text, feedback: "Expected a tag name") 588 | 589 | let attributes = context.untilThrowOrEndOfTokensReached(perform: { 590 | try context.attempt(action: { 591 | try Attribute.parse(context: context, options: options) 592 | }) 593 | }) 594 | ``` 595 | 596 | Next, we parse an identifier that we'll use for the tag's element. Then we attempt to parse as many attributes as we can (there may be 0). 597 | 598 | ``` 599 | // If there's a trailing slash (eg ), consume it but ignore it. this is invalid html 600 | _ = try? context.consume(tokenKind: .forwardSlash, feedback: "Expected a trailing `/`") 601 | return Tag(element: identifier.body, isEnd: slashToken != nil, attributes: attributes) 602 | } 603 | } 604 | } 605 | ``` 606 | 607 | Finally, optionally look for and ignore a trailing slash at the end of the tag, as it's not actually valid html (this was news to me when I started working on the browser). However, it's extremely common, so I thought it warranted handling here to make more of the web work. With that out of the way, we return our completed tag. 608 | 609 | ### Attribute 610 | 611 | Ok, last part of the parser! the attributes inside a tag. 612 | 613 | ``` 614 | struct Attribute: Hashable, Parsable { 615 | let key: String 616 | let value: String 617 | 618 | enum AttributeParseError: Error { 619 | case emptyAttributeValue(key: String) 620 | } 621 | 622 | static func parse(context: ParsingContext, options: ParsingOptions?) throws -> Attribute { 623 | 624 | let key = try context.consume(tokenKind: .text, feedback: "Expected an attribute name") 625 | 626 | guard let _ = try? context.consume(tokenKind: .equals, feedback: "Expected an equals sign") else { 627 | return Attribute(key: key.body, value: key.body) 628 | } 629 | ``` 630 | 631 | Attributes are (usually) key-value pairs, so those are our properties (for attributes that don't have explicit values, we'll just repeat the key for the value). 632 | 633 | Then, we start parsing. First we parse the key, then we look for an equals sign token. If we don't find it, we assume this attribute is the valueless kind and return it immediately. Otherwise, we parse the value, as a choice: 634 | 635 | ``` 636 | let value = try context.choose(from: [ 637 | { 638 | try context.consumeBetween(leftToken: .doubleQuote, rightToken: .doubleQuote) { 639 | let textContents = context.untilThrowOrEndOfTokensReached { 640 | try context.consume(where: { $0.kind != .doubleQuote }, skipWhitespaceTokens: false, feedback: "Expected a non quote token") 641 | } 642 | 643 | return textContents 644 | .map(\.body) 645 | .joined() 646 | } 647 | }, 648 | ``` 649 | 650 | First choice: the value is between double quotes, and we consume everything inside that isn't a double quote (and we don't skip whitespaces either). Then we join all those tokens together and return that as the value. 651 | 652 | ``` 653 | { 654 | try context.consumeBetween(leftToken: .singleQuote, rightToken: .singleQuote) { 655 | let textContents = context.untilThrowOrEndOfTokensReached { 656 | try context.consume(where: { $0.kind != .singleQuote }, skipWhitespaceTokens: false, feedback: "Expected a non single quote token") 657 | } 658 | 659 | return textContents 660 | .map(\.body) 661 | .joined() 662 | } 663 | }, 664 | ``` 665 | 666 | Second choice: same thing as before, except between single quotes. 667 | 668 | ``` 669 | { 670 | let textContents = context.untilThrowOrEndOfTokensReached { 671 | try context.consume( 672 | where: { 673 | $0.kind != .singleQuote && $0.kind != .doubleQuote && $0.kind != .whitespace && $0.kind != .closeAngleBracket 674 | }, 675 | skipWhitespaceTokens: false, 676 | feedback: "Expected non-whitespace, non-quote characters") 677 | } 678 | 679 | guard textContents.isEmpty == false else { 680 | throw AttributeParseError.emptyAttributeValue(key: key.body) 681 | } 682 | 683 | return textContents 684 | .map(\.body) 685 | .joined() 686 | } 687 | ``` 688 | 689 | Final choice: we look for a value that's *not* wrapped in any kind of quotes. These kinds of values are delimitted by whitespace (or an angle bracket), so we consume basically everything else, make sure we actually found something non-empty, and join those tokens together into a value. 690 | 691 | ``` 692 | ]) 693 | 694 | return Attribute(key: key.body, value: value) 695 | } 696 | } 697 | ``` 698 | 699 | Last, we return the completed attribute. 700 | 701 | ## End of Part 1 702 | 703 | This completes the end of part 1! We built ourselves some tools for breaking apart a program string into tokens and parsing them. And then we built some data types that know how to parse themselves using those tools. HTML is a kind of strange language, but we saw some familiar patterns repeated in multiple places (things being wrapped inside others, for example). 704 | 705 | In the next part, we'll take the data we just parsed and render it with SwiftUI. [Onward to part 2!](https://nearthespeedoflight.com/browser-2.html) 706 | 707 | 708 | # Part 2: Rendering in SwiftUI 709 | 710 | Welcome to part 2 of "Writing a web browser engine in Swift!" In [part 1](https://nearthespeedoflight.com/browser.html), we built a basic html parser from the ground up, learning about tokenizing, parsing, and syntax trees. We now have a fairly complete set of tools that can parse html into plain old Swift structs. 711 | 712 | In this part, we'll build our rendering engine with SwiftUI views. Let's get started. 713 | 714 | ## The Architecture 715 | 716 | The architecture of our rendering engine should look pretty familiar to anyone who's worked with SwiftUI before: we're more or less just going to have views which render our node hierarchy. It's almost exclusively composed of standard SwiftUI views, plus a controller object for loading HTML pages, and a few extensions on the `Node` type to more easily work with its properties. Here are the main pieces we'll be working with. 717 | 718 | - `PageController` is responsible for loading web urls asynchronously and parsing them into `Document`s. It also maintains the back / forward stacks of documents. 719 | - Some views: 720 | - `BrowserView` is the primary view, containing our chrome (back / forward / address bar) and the document view. 721 | - `WebDocumentView` displays either a homepage, error page, or the contents of the loaded page, depending on the page controller's state. 722 | - `BodyView` is the true beginnings of our rendering engine, it nests our page's content in a scroll view. 723 | - `BlocksView` displays views for 0 or more nodes in a vertical stack. It picks a different view depending on the node's element. 724 | - `InlineContentWrappingBlockView` combines the text of all its inline elements into one big `Text` for rendering. 725 | - `ListNodeView` renders ordered or unordered lists and their items. 726 | - `ImageView` asynchronously downloads and renders img nodes. 727 | - Extensions on `Node` for accessing its content. 728 | 729 | ### The Page Controller 730 | 731 | The `PageController` is our main controller object, responsible for loading pages, parsing them, and managing the back / forward stacks: 732 | 733 | ``` 734 | class PageController: ObservableObject { 735 | 736 | enum State { 737 | case notLoaded 738 | case loaded(Document, URL) 739 | case failed(Error) 740 | } 741 | 742 | private enum LoadingError: Error { 743 | case failedToLoad(URL) 744 | } 745 | 746 | @Published var state = State.notLoaded { 747 | didSet { 748 | if let currentlyLoadedDocument { 749 | address = currentlyLoadedDocument.1.absoluteString 750 | } 751 | } 752 | } 753 | var address = "https://nearthespeedoflight.com/browser.html" 754 | 755 | private var backStack: [(Document, URL)] = [] 756 | private var forwardStack: [(Document, URL)] = [] 757 | 758 | var canGoBack: Bool { backStack.isEmpty == false } 759 | var canGoForward: Bool { forwardStack.isEmpty == false } 760 | ``` 761 | 762 | First we set up some nested types. The controller can be in one of three `State`s: an initial unloaded state (maybe you show a homepage?), a loaded state with the parsed document and the URL it came from, and the failed error state. 763 | 764 | Then we have some properties, mainly the controller's current `state`, its current `address` string, and the back / forward stacks. 765 | 766 | ``` 767 | func loadPage(at url: URL) { 768 | Task { 769 | let newState: State 770 | do { 771 | let (data, response) = try await URLSession.shared.data(from: url) 772 | 773 | if let currentlyLoadedDocument { 774 | backStack.append(currentlyLoadedDocument) 775 | forwardStack = [] 776 | } 777 | 778 | let htmlString = String(data: data, encoding: .utf8) ?? "" 779 | let tokenizer = Tokenizer(programText: htmlString) 780 | let context = try ParsingContext(tokens: tokenizer.scanAllTokens()) 781 | 782 | newState = .loaded(try Document.parse(context: context, options: nil), response.url ?? url) 783 | } catch { 784 | print("error loading page: \(error)") 785 | newState = .failed(error) 786 | } 787 | 788 | await MainActor.run { 789 | state = newState 790 | } 791 | } 792 | } 793 | ``` 794 | 795 | To load a page, we kick off an async `Task`, await the loading of the given url, then we put the data through our parser pipeline. We also set the back / forward stacks to account for the state change that's about to happen. 796 | 797 | This is all made a little awkward due to error handling, as we want to catch any errors that happen here: there could be URL related errors, there could be an error in the parsing context, or there could be an error parsing the document. If there is an error, we want to record it. This wouldn't be so bad on its own, but we don't want to do any of this parsing on the main actor, where it could freeze the UI, *but* we must update our controller's `state` property on the main actor, as our view depends on that property to draw itself. 798 | 799 | ``` 800 | private var currentlyLoadedDocument: (Document, URL)? { 801 | switch state { 802 | case .notLoaded, .failed: return nil 803 | case let .loaded(document, url): return (document, url) 804 | } 805 | } 806 | 807 | func goBack() { 808 | guard let (previousDocument, previousURL) = backStack.popLast() else { return } 809 | if let currentlyLoadedDocument { 810 | forwardStack.append(currentlyLoadedDocument) 811 | } 812 | state = .loaded(previousDocument, previousURL) 813 | } 814 | 815 | func goForward() { 816 | guard let (nextDocument, nextURL) = forwardStack.popLast() else { return } 817 | if let currentlyLoadedDocument { 818 | backStack.append(currentlyLoadedDocument) 819 | } 820 | state = .loaded(nextDocument, nextURL) 821 | } 822 | } 823 | ``` 824 | 825 | Finally, we have a helper property for accessing the currently loaded document, if any, and methods for going back and forward. That wraps up our controller. Next, we'll see how the views make use of it while displaying our nodes. 826 | 827 | ### The Browser View 828 | 829 | As mentioned above, our `BrowserView` is the primary view for our browser window: it composes the "chrome" of our UI, plus the actual rendered content in another view. Our UI is going to be very simple, but you could extend it to use tabs, or even something more imaginative if you want :) 830 | 831 | ``` 832 | struct BrowserView: View { 833 | @ObservedObject var controller: PageController 834 | @FocusState private var addressIsFocused: Bool 835 | ``` 836 | 837 | All we need are 2 properties, an observed page controller and the focus state of the address textfield, so that focus works like you'd expect as we navigate. 838 | 839 | ``` 840 | var body: some View { 841 | VStack(spacing: 0) { 842 | HStack { 843 | HStack(spacing: 0) { 844 | Button(action: { controller.goBack() }) { 845 | Image(systemName: "arrowtriangle.left.fill") 846 | }.disabled(controller.canGoBack == false) 847 | Button(action: { controller.goForward() }) { 848 | Image(systemName: "arrowtriangle.right.fill") 849 | }.disabled(controller.canGoForward == false) 850 | } 851 | TextField("Address", text: $controller.address) 852 | .onSubmit { 853 | addressIsFocused = false 854 | guard let url = URL(string: controller.address) else { return } 855 | controller.loadPage(at: fullURL(forURLToLoad: url)) 856 | } 857 | .textFieldStyle(RoundedBorderTextFieldStyle()) 858 | .focused($addressIsFocused) 859 | } 860 | .padding() 861 | Divider() 862 | ``` 863 | 864 | The body of our view until this point is all about the chrome. We create our back / forward buttons and the address bar, and we bind their actions to our controller. 865 | 866 | ``` 867 | WebDocumentView(controller: controller) 868 | .background(.white) 869 | .environment(\.openURL, .init(handler: { url in 870 | controller.loadPage(at: fullURL(forURLToLoad: url)) 871 | addressIsFocused = false 872 | return .handled 873 | })) 874 | } 875 | ``` 876 | 877 | We configure the WebDocumentView and override SwiftUI's `openURL` environment value. When the user clicks a link in our app, SwiftUI invokes this callback, giving our app a chance to handle the URL. With the given URL, we construct an absolute URL (below), adjust the text field's focus, and tell the system we handled the url (we could also tell the system to handle it instead if the URL was eg `mailto:...`, but I'll leave that to you). 878 | 879 | ``` 880 | .environment(\.urlBuilder, fullURL(forURLToLoad:)) 881 | } 882 | 883 | private func fullURL(forURLToLoad urlToLoad: URL) -> URL { 884 | if urlToLoad.host != nil { return urlToLoad } 885 | 886 | switch controller.state { 887 | case .failed, .notLoaded: return urlToLoad 888 | case .loaded(_, let loadedURL): 889 | return URL(string: urlToLoad.path, relativeTo: loadedURL.deletingLastPathComponent()) ?? urlToLoad 890 | } 891 | } 892 | } // End of BrowserView 893 | 894 | private struct URLBuilderKey: EnvironmentKey { 895 | static let defaultValue: (URL) -> URL = { $0 } 896 | } 897 | 898 | extension EnvironmentValues { 899 | /// A function that takes a (potentially "relative") web url to load, and fleshes it out to a full url that includes a host. 900 | var urlBuilder: (URL) -> URL { 901 | get { self[URLBuilderKey.self] } 902 | set { self[URLBuilderKey.self] = newValue } 903 | } 904 | } 905 | ``` 906 | 907 | Finally, we use the environment modifier for a custom environment value. The `urlBuilder` is a closure / function responsible for taking a URL (one that's possibly relative, eg just `/page.html` vs `https://example.com/page.html`) and expanding it to an absolute URL so that pages and assets like images can be loaded. 908 | 909 | We do this as an environment value so that other views in the hierarchy can access the functionality. 910 | 911 | ### Web Document View 912 | 913 | The `WebDocumentView` takes up the majority of space in our browser window. What it shows depends on the `state` of the page controller, either showing a simple home page, error screen, or the loaded content. 914 | 915 | ``` 916 | struct WebDocumentView: View { 917 | @ObservedObject var controller: PageController 918 | 919 | var body: some View { 920 | switch controller.state { 921 | case .notLoaded: 922 | Text("Let's load a web page!") 923 | .frame(maxWidth: .infinity, maxHeight: .infinity) 924 | case .failed(let error): 925 | Text(verbatim: "Failed to load page. Error: \(error)") 926 | .frame(maxWidth: .infinity, maxHeight: .infinity) 927 | case .loaded(let document, _): 928 | BodyView(bodyNode: document.htmlNode.firstDirectChild(named: "body")!) 929 | .navigationTitle( 930 | document 931 | .htmlNode 932 | .firstDirectChild(named: "head")? 933 | .firstDirectChild(named: "title")? 934 | .firstDirectChild(named: Node.InternalElement.textRun)? 935 | .textContent ?? "Smol" 936 | ) 937 | .environment(\.font, Font.custom("Times", size: 16)) 938 | } 939 | } 940 | } 941 | ``` 942 | 943 | The `BodyView` accesses some properties on `Node` which we'll write shortly for accessing child nodes more easily. We drill down to find the page's title, if it has one, and set that as our window title. Finally, we set a default font on the document's text. "Times" is the font you see in most browsers with unstylized text (but you're allowed to choose any font you'd like here). 944 | 945 | ### Node extensions 946 | 947 | Before we go any further with our views, let's write those helpers in an extension on `Node`. 948 | 949 | ``` 950 | extension Node { 951 | var childNodes: [Node] { 952 | switch content { 953 | case .voidNode, .text: return [] 954 | case .childNodes(let nodes): return nodes 955 | } 956 | } 957 | 958 | var textContent: String? { 959 | switch content { 960 | case .childNodes, .voidNode: return nil 961 | case .text(let text): return text 962 | } 963 | } 964 | 965 | func firstDirectChild(named element: String) -> Node? { 966 | childNodes.first(where: { $0.element == element }) 967 | } 968 | ``` 969 | 970 | These properties help us access child nodes and text content more easily. 971 | 972 | ``` 973 | var childNodesSortedIntoBlocks: [Node] { 974 | var nodesToReturn = [Node]() 975 | var inlineElements = [Node]() 976 | 977 | func addInlineElementsAsGroupIfNeeded() { 978 | guard inlineElements.isEmpty == false else { return } 979 | // make a fake block element that has all these as children 980 | let wrapper = Node(element: "p", content: .childNodes(inlineElements), attributes: []) 981 | // and append it to our list to return 982 | nodesToReturn.append(wrapper) 983 | // then, empty the inlineElements list 984 | inlineElements = [] 985 | } 986 | 987 | for node in childNodes { 988 | if isInlineNode { 989 | inlineElements.append(node) 990 | } else { 991 | addInlineElementsAsGroupIfNeeded() 992 | nodesToReturn.append(node) 993 | } 994 | } 995 | addInlineElementsAsGroupIfNeeded() 996 | return nodesToReturn 997 | } 998 | 999 | var isInlineNode: Bool { 1000 | [InternalElement.textRun, "a", "abbr", "acronym", "audio", "b", "bdi", "bdo", "big", "br", "button", "canvas", "cite", "code", "data", "datalist", "del", "dfn", "em", "embed", "i", "iframe", "img", "input", "ins", "kbd", "label", "map", "mark", "meter", "noscript", "object", "output", "picture", "progress", "q", "ruby", "s", "samp", "script", "select", "slot", "small", "span", "strong", "sub", "sup", "svg", "template", "textarea", "time", "u", "tt", "var", "video", "wbr"].contains(element) 1001 | } 1002 | ``` 1003 | 1004 | This next property is a little more involved. When we're rendering nodes, we want block nodes, like `

`, `

`, etc. to flow one after another, vertically down the page, while things like ``, ``, etc. flow within the same line like words in a paragraph. 1005 | 1006 | The trouble for us is, in html those inline elements don't have exist inside of block elements at all, they can exist outside of them too. For example: 1007 | 1008 | ``` 1009 | 1010 | Some bold text 1011 |

A paragraph

1012 | 1013 | ``` 1014 | 1015 | The bold text is just kinda hanging out as inline, but inline relative *to what?* I'm not entirely sure how other browsers solve this, but we've solved it by grouping any inline elements as children of a fake, inserted `

` node. 1016 | 1017 | ``` 1018 | var attributeDictionary: [String: String] { 1019 | Dictionary(uniqueKeysWithValues: attributes.map({ ($0.key, $0.value) })) 1020 | } 1021 | } 1022 | ``` 1023 | 1024 | Lastly, we offer a way to access the node's attributes as a dictionary. 1025 | 1026 | Now we have enough tools at our disposal to write the rest of the views. 1027 | 1028 | ### The BodyView 1029 | 1030 | This view hosts our browser's scroll view, which then displays child nodes in another view. 1031 | 1032 | ``` 1033 | struct BodyView: View { 1034 | let bodyNode: Node 1035 | var body: some View { 1036 | ScrollView { 1037 | BlocksView(children: bodyNode.childNodesSortedIntoBlocks) 1038 | .padding(20) 1039 | } 1040 | .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading) 1041 | .background(Color.white) 1042 | } 1043 | } 1044 | ``` 1045 | 1046 | The hierarchy here is mostly straightforward: the `BlocksView` is initialized with the child nodes of the body and is given a global padding. Then we extend the frame of the scroll view to stretch as much as possible and align the content to the top leading edge, like other browsers do. 1047 | 1048 | ### BlocksView 1049 | 1050 | This one is kind of fun: it's a reusable view that vertically stacks the child nodes it was given, rendering them with the appropriate view depending on what element they are. It even recursively uses itself in a few cases. 1051 | 1052 | ``` 1053 | struct BlocksView: View { 1054 | let children: [Node] 1055 | 1056 | var body: some View { 1057 | VStack(alignment: .leading, spacing: 20) { 1058 | ForEach(children, id: \.self) { childNode in 1059 | switch childNode.element { 1060 | case "h1": 1061 | InlineContentWrappingBlockView(node: childNode) 1062 | .font(Font.custom("Times", size: 32).bold()) 1063 | case "h2": 1064 | InlineContentWrappingBlockView(node: childNode) 1065 | .font(Font.custom("Times", size: 28).bold()) 1066 | case "h3": 1067 | InlineContentWrappingBlockView(node: childNode) 1068 | .font(Font.custom("Times", size: 24).bold()) 1069 | case "p": 1070 | InlineContentWrappingBlockView(node: childNode) 1071 | case "div", "section", "main", "footer", "article", "header", "nav", "aside": 1072 | BlocksView(children: childNode.childNodesSortedIntoBlocks) 1073 | case "pre": 1074 | BlocksView(children: childNode.childNodesSortedIntoBlocks) 1075 | .font(Font.system(size: 13, design: .monospaced)) 1076 | case "blockquote": 1077 | BlocksView(children: childNode.childNodesSortedIntoBlocks) 1078 | .padding(.leading, 20) 1079 | case "ul": ListNodeView(node: childNode, style: .unordered) 1080 | case "ol": ListNodeView(node: childNode, style: .ordered) 1081 | case "hr": Divider() 1082 | case "script": EmptyView() 1083 | case "br": Color.clear.padding(20) 1084 | default: Text("unknown block element: <\(childNode.element)>") 1085 | } 1086 | } 1087 | } 1088 | } 1089 | } 1090 | ``` 1091 | 1092 | We don't support special rendering for every element under the sun, so if we find an element we don't know about, we just render that we found an unknown block. You could default it to behaving like a `

` if you wanted, but I like calling them out like this instead because I'm more motivated to give it a proper view that way. 1093 | 1094 | ### Inline nodes 1095 | 1096 | Inline nodes are interesting, because to render them we can't just use views placed in some kind of stack. Instead, we want them to be rendered one after another like text, wrapping to the next line as needed. And indeed, that's how we're going to do it in SwiftUI, by combining (or in Swift terms, using `reduce()`) inline contents into an `AttributedString` and rendering it in a single `Text` view per inline "block." 1097 | 1098 | ``` 1099 | struct InlineContentWrappingBlockView: View { 1100 | let node: Node 1101 | @Environment(\.font) var font 1102 | 1103 | var body: some View { 1104 | Text( 1105 | node 1106 | .childNodes 1107 | .map { $0.attributedText(defaultFont: font ?? Font.custom("Times", size: 16)) } 1108 | .reduce(AttributedString(), +) 1109 | ) 1110 | .lineSpacing(4) 1111 | .fixedSize(horizontal: false, vertical: true) 1112 | } 1113 | } 1114 | ``` 1115 | 1116 | In the body of our body, we return a single `Text`, initialized with an attributed string. The attributed string is created by mapping the node's child nodes and calling the `attributedText(defaultFont:)` method on each (we'll see that property in a moment). This mapping gives us an array of attribute strings, so we `reduce()` them into a single attributed string. 1117 | 1118 | ``` 1119 | extension Node { 1120 | func attributedText(defaultFont: Font) -> AttributedString { 1121 | switch element { 1122 | case InternalElement.textRun: 1123 | var attributes = AttributeContainer() 1124 | attributes.font = defaultFont 1125 | 1126 | return AttributedString(textContent ?? "", attributes: attributes) 1127 | ``` 1128 | 1129 | To get the attributed text for a node, we switch over its `element` to see how we should format it. Here we have the base case: a text run. We create an attribute container, use the font that was passed in, and return an attributed string with the node's text content and those attributes. 1130 | 1131 | ``` 1132 | case "em", "i": 1133 | var attributes = AttributeContainer() 1134 | attributes.font = defaultFont.italic() 1135 | 1136 | return childNodes 1137 | .map { $0.attributedText(defaultFont: defaultFont.italic()) } 1138 | .reduce(AttributedString(), +) 1139 | .mergingAttributes(attributes, mergePolicy: .keepCurrent) 1140 | ``` 1141 | 1142 | The rest of the cases are similar, in that we create some attributes, modifying the passed in font as needed. But in order to create the final attributed string, we actually need to recursively call ourselves so that we can handle multiple overlapping styles (eg a link node wrapped inside an italics node). 1143 | 1144 | ``` 1145 | case "strong", "b": 1146 | var attributes = AttributeContainer() 1147 | attributes.font = defaultFont.bold() 1148 | 1149 | return childNodes 1150 | .map { $0.attributedText(defaultFont: defaultFont.bold()) } 1151 | .reduce(AttributedString(), +) 1152 | .mergingAttributes(attributes, mergePolicy: .keepCurrent) 1153 | case "code": 1154 | var attributes = AttributeContainer() 1155 | let monospaced = Font.system(size: 13, design: .monospaced) 1156 | attributes.font = monospaced 1157 | 1158 | return childNodes 1159 | .map { $0.attributedText(defaultFont: monospaced) } 1160 | .reduce(AttributedString(), +) 1161 | .mergingAttributes(attributes, mergePolicy: .keepCurrent) 1162 | case "a": 1163 | var attributes = AttributeContainer() 1164 | attributes.link = URL(string: attributeDictionary["href"] ?? "") 1165 | attributes.underlineStyle = .single 1166 | 1167 | return childNodes 1168 | .map { $0.attributedText(defaultFont: defaultFont) } 1169 | .reduce(AttributedString(), +) 1170 | .mergingAttributes(attributes, mergePolicy: .keepCurrent) 1171 | default: 1172 | var attributes = AttributeContainer() 1173 | attributes.font = defaultFont 1174 | 1175 | return childNodes 1176 | .map { $0.attributedText(defaultFont: defaultFont) } 1177 | .reduce(AttributedString(), +) 1178 | .mergingAttributes(attributes, mergePolicy: .keepCurrent) 1179 | } 1180 | } 1181 | } 1182 | ``` 1183 | 1184 | It's all a little boilerplatey but it gets the job done. 1185 | 1186 | ### ListNodeView 1187 | 1188 | Our last node view is the `ListNodeView`, which we'll use for displaying both ordered and unordered lists (`

` and `

`). 1189 | 1190 | ``` 1191 | struct ListNodeView: View { 1192 | enum Style { 1193 | case ordered, unordered 1194 | 1195 | func listMarker(for index: Int) -> String { 1196 | switch self { 1197 | case .ordered: return "\(index + 1)." 1198 | case .unordered: return "•" 1199 | } 1200 | } 1201 | } 1202 | ``` 1203 | 1204 | We start with an enum for the two list styles, and a function for picking the right list item marker for the given index. 1205 | 1206 | ``` 1207 | let node: Node 1208 | let style: Style 1209 | 1210 | var body: some View { 1211 | VStack(alignment: .leading, spacing: 8) { 1212 | ForEach(Array(zip(node.childNodes.indices, node.childNodes)), id: \.1) { (index, childNode) in 1213 | HStack(alignment: .firstTextBaseline, spacing: 8) { 1214 | Text(verbatim: style.listMarker(for: index)) 1215 | BlocksView(children: childNode.childNodesSortedIntoBlocks) 1216 | } 1217 | } 1218 | } 1219 | } 1220 | } 1221 | ``` 1222 | 1223 | The body of the list node should look pretty straightforward at this point: a vertical stack wherein we iterate all our child nodes and render them as list items (using a kind of noisy `Array/zip` dance to get the index). A list item uses a horizontal stack to display the marker, followed by a `BlocksView` for the grouped children (it could be, for example, multiple paragraphs). 1224 | 1225 | ### Preserving Whitespace 1226 | 1227 | You may notice at this point, if you load up a webpage with a `

` tag, it uses a monospaced font but it does not preserve whitespace. That's because we're currently discarding tabs and newlines in our parser. But for `` tags we should be preserving it in child nodes (and all of their descendents). So let's make some modifications to the parser.
1228 | 
1229 | In the parse method of `Node`, immediately before parsing child nodes, let's add the following line:
1230 | 
1231 | ```
1232 | let shouldPreserveWhiteSpace = startTag.element == "pre"
1233 | ```
1234 | 
1235 | Then, in the 2nd choice closure (text runs), we want to replace whitespaces only when we're *not* preserving whitespace. Change our entity decoding code to the following:
1236 | 
1237 | ```
1238 | let entityDecodedContents = textContents
1239 | 	.map(\.body)
1240 | 	.joined()
1241 | 		.replacingOccurrences(of: "
", with: "")
1242 | 		.replacingOccurrences(of: "<", with: "<")
1243 | 		.replacingOccurrences(of: ">", with: ">")
1244 | 		.replacingOccurrences(of: "'", with: "'")
1245 | 		.replacingOccurrences(of: """, with: "\"")
1246 | 		.replacingOccurrences(of: "’", with: "’")
1247 | 		.replacingOccurrences(of: "‘", with: "’")
1248 | 		.replacingOccurrences(of: "”", with: "”")
1249 | 		.replacingOccurrences(of: "“", with: "“")
1250 | 		.replacingOccurrences(of: "&", with: "&")
1251 | 
1252 | let contentRun = shouldPreserveWhitespace ? entityDecodedContents : entityDecodedContents
1253 | 	.replacingOccurrences(of: "\n", with: " ")
1254 | 	.replacingOccurrences(of: "\t", with: " ")
1255 | ```
1256 | 
1257 | This preserves whitespace great when the text is a direct child of the node, but doesn't yet handle deeper nestings. To do that, we'll have to pass `shouldPreserveWhitespace` as a flag to child node parsing. To keep things simple, let's add the flag to our `Parsable` protocol requirement:
1258 | 
1259 | ```
1260 | static func parse(context: ParsingContext, shouldPreserveWhitespace: Bool) throws -> Self
1261 | ```
1262 | 
1263 | After you make the change you'll have to go through all the call sites where we implement the `parse(...)` method and update them to include the new flag. At pretty much every callsite, just give a value of `false`. However, let's return to `Node`, specifically where we're parsing a child node (the first choice). Change the parse call to:
1264 | 
1265 | ```
1266 | try Node.parse(context: context, shouldPreserveWhitespace: shouldPreserveWhitespace)
1267 | ```
1268 | 
1269 | so that we can pass it down the line. Finally, where we declare our local variable for preserving whitespace, we update that to factor in the parameter passed in:
1270 | 
1271 | ```
1272 | let shouldPreserveWhitespace = startTag.element == "pre" || shouldPreserveWhitespace
1273 | ```
1274 | 
1275 | Now if we run the browser, we should see that whitespace preservation works as expected.
1276 | 
1277 | ### Displaying images
1278 | 
1279 | Let's add support for displaying images using the `` tag. By default, image elements are considered to be "inline" (ie, `display: inline` in CSS terms), but inline images are a little complicated, so I'll leave those as an exercise for the reader. However, supporting block images will be a little more straightforward, so let's add support for that.
1280 | 
1281 | While supporting all of CSS is way, way out of scope for this tutorial, it would still be nice to leave us with a starting point, so I'd like to add support to the `style` attribute, which we'll parse in a rather crude way. Once we have the style attribute parsed, we'll add support for `display: block` as well. Let's start with a type representing a style.
1282 | 
1283 | ```
1284 | struct Style {
1285 | 	
1286 | 	enum DisplayStyle { case inline, block }
1287 | 	
1288 | 	var display: DisplayStyle? {
1289 | 		switch rawValue["display"] {
1290 | 		case "inline": return .inline
1291 | 		case "block": return .block
1292 | 		default:
1293 | 			return nil
1294 | 		}
1295 | 	}
1296 | 
1297 | 	private let rawValue: [String: String]
1298 | 
1299 | 	init(rawPairs: [String: String]) {
1300 | 		self.rawValue = rawPairs
1301 | 	}
1302 | }
1303 | ```
1304 | 
1305 | `Style` wraps an underlying dictionary of keys and values, and adds a helper property that looks for a `display` key, returning the value interpreted as either `inline` or `block` (or `nil`, if missing or something else). Now let's create a style instance from a node's style attribute, if it exists. In an extension on `Node`, put the following:
1306 | 
1307 | ```
1308 | var styleFromAttributes: Style? {
1309 | 	guard let styleAttribute = attributeDictionary["style"] else { return nil }
1310 | 	let stylePairs = styleAttribute.components(separatedBy: ";")
1311 | ```
1312 | 
1313 | First, we check to see if we even have a style attribute, otherwise we bail. Then, we break up the value string into substrings, which are separated by a semicolon.
1314 | 
1315 | ```
1316 | 	return Style(
1317 | 		rawPairs: .init(
1318 | 			uniqueKeysWithValues: stylePairs
1319 | 				.map { $0.components(separatedBy: ":") }
1320 | 				.map { ($0.first?.trimmingCharacters(in: .whitespacesAndNewlines), $0.last?.trimmingCharacters(in: .whitespacesAndNewlines)) }
1321 | 				.compactMap {
1322 | 					guard let key = $0, let value = $1 else { return nil }
1323 | 					return (key, value)
1324 | 				}
1325 | 		)
1326 | 	)
1327 | }
1328 | ```
1329 | 
1330 | Finally, we initialize the `Style` with a dictionary, whose keys and values are found by splitting up those substrings from earlier on colons, trimming out whitespace, and finally returning them as a non-nil tuple. This code is kinda fragile would definitely be made more powerful (and extensible!) if we wrote a parser like we did for html, but I'll leave that as an exercise for the reader :)
1331 | 
1332 | Now that we can parse a `Style` from a node's attributes, the last thing we need to do is modify how we're sorting / grouping nodes. In the `Node.childNodesSortedIntoBlocks` property, edit the for loop to look like this:
1333 | 
1334 | ```
1335 | for node in childNodes {
1336 | 	let defaultDisplayStyle: Style.DisplayStyle = node.isInlineNode ? .inline : .block 
1337 | 	let display = node.styleFromAttributes?.display ?? defaultDisplayStyle
1338 | 	if display == .inline {
1339 | 		inlineElements.append(node)
1340 | 	} else {
1341 | 		addInlineElementsAsGroupIfNeeded()
1342 | 		nodesToReturn.append(node)
1343 | 	}
1344 | }
1345 | ```
1346 | 
1347 | With that modification, `` nodes that have a style attribute declaring they should be `display: block` will now be properly considered block views in our renderer, and appear accordingly. All that's left is to create a view that can load and display images. SwiftUI already has the a perfect view for us: `AsyncImage`, which we'll wrap in our own `ImageView` to customize it a little.
1348 | 
1349 | ```
1350 | struct ImageView: View {
1351 | 	let node: Node
1352 | 	@Environment(\.urlBuilder) var urlBuilder
1353 | 
1354 | 	var body: some View {
1355 | 		AsyncImage(url: urlBuilder(URL(string: node.attributeDictionary["src"] ?? "")!), content: { image in
1356 | 			image
1357 | 				.resizable()
1358 | 				.aspectRatio(contentMode: .fit)
1359 | 				.frame(
1360 | 					width: node.attributeDictionary["width"].flatMap(WebSize.init(rawValue:))?.dimension,
1361 | 					height: node.attributeDictionary["height"].flatMap(WebSize.init(rawValue:))?.dimension
1362 | 				)
1363 | 		}, placeholder: {
1364 | 			Color(white: 0.9).cornerRadius(4)
1365 | 		})
1366 | 	}
1367 | }
1368 | ```
1369 | 
1370 | From the environment, we pull out the `urlBuilder` function we declared earlier in our view hierarchy, so that we can make sure the image's `src` url is an absolute url, that we'll then hand off to SwiftUI to load asynchronously for us. When the image is ready, we resize it and constrain it as necessary, depending on any width or height attributes of the `` node.
1371 | 
1372 | ```
1373 | struct WebSize {
1374 | 	let rawValue: String
1375 | 
1376 | 	var dimension: CGFloat {
1377 | 		// trim anything that isn't a digit, then try to parse that into an int. this ignores things like "px"
1378 | 		CGFloat(Int(rawValue.prefix(while: \.isWholeNumber)) ?? 0)
1379 | 	}
1380 | }
1381 | ```
1382 | 
1383 | `WebSize` is a small little type for extracting number values out of sizing values in html. We're assuming everything is measured in `px` for simplicity's sake. Sizing in html is a complicated topic, but you could go deep here if you wanted.
1384 | 
1385 | Finally, let's add a case for `img` nodes in our `BlocksView` body switch statement that uses the new image view we created:
1386 | 
1387 | ```
1388 | case "img":
1389 | 	ImageView(node: childNode)
1390 | ```
1391 | 
1392 | With that, we should finally be able to see image tags with their `display` set to `block`.
1393 | 
1394 | ### One last thing
1395 | 
1396 | You may have noticed that most browsers, when given an unstyled html page, will render the body using the entire width of the browser window, and our browser does this exact same thing. However, on modern monitors, this can result in extremely long lines of text that are kind of hard to read due to their length, so the nice thing to do is to style the container with a maximum width.
1397 | 
1398 | Now that we have support for the `style` attribute, let's parse out the `max-width` style, if it's present. In the `Style` struct, add the following property:
1399 | 
1400 | ```
1401 | var maxWidth: CGFloat? {
1402 | 	rawValue["max-width"].map(WebSize.init(rawValue:)).map(\.dimension)
1403 | }
1404 | ```
1405 | 
1406 | Let's use this property on our `BodyView`. Change the body view's `body` scroll view to this:
1407 | 
1408 | ```
1409 | ScrollView {
1410 | 	HStack(spacing: 0) {
1411 | 		BlocksView(children: bodyNode.childNodesSortedIntoBlocks)
1412 | 			.frame(maxWidth: bodyNode.styleFromAttributes?.maxWidth)
1413 | 		Spacer()
1414 | 	}
1415 | 	.padding(20)
1416 | }
1417 | ```
1418 | 
1419 | This makes the `BlocksView` respect the `max-width` from the `` tag, if it exists.
1420 | 
1421 | ## The End
1422 | 
1423 | This concludes our browser engine. We wrote a simple html parser from scratch with Swift, and then wrote a rendering engine using SwiftUI. In all, the browser should be capable of rendering this very tutorial, and it should look pretty much identical to how it looks in Safari, Chrome, or other Big Browsers.
1424 | 
1425 | You could extend this foundation in so many ways:
1426 | 
1427 | - You could add more block or inline elements types
1428 | - You could expand what style elements are supported
1429 | - You could even write your own CSS parser! or if you have limitless ambition, you could write a javascript engine too
1430 | 
1431 | But most of all, I hope you enjoyed yourself and learned a thing or two.
1432 | 
1433 | Thanks for reading! And special thanks to Dalton Claybrook, Dean Silfen, Drew Wyatt, and Kate Brennan for all their feedback and encouragement.
1434 | 
1435 | (ps: I'm looking for work, so if you're looking to hire someone to work on browsers, programming languages, dev tools, or Swift apps, I'm your guy! [Please reach out](mailto:i.jasonbrennan@gmail.com), I'd love to hear from you)
1436 | 


--------------------------------------------------------------------------------
/Smol/output.html:
--------------------------------------------------------------------------------
   1 | 
   2 | 	
   3 | 		Smol page
   4 | 	
   5 | 	
   6 | Let’s Write a Web Browser from Scratch in Swift!
   7 | 
   8 | There’s a rumour that Apple’s going to start allowing custom, non-WebKit based browser engines on iOS starting later this year. While that most likely means Chrome, Firefox, and the other big browsers could start using custom engines, it also means you could write your own too. So why not try it?
   9 | 
  10 | In this 2 part series, I’ll take you through how to write a basic web browser, from parsing HTML in Swift to rendering the pages with SwiftUI, displaying them with a simple, but familiar interface.
  11 | 
  12 | 
  13 | 
  14 | You might be thinking “Aren’t web browsers huge, incredibly complicated pieces of software?” and yes, the big ones we use every day are huge and complicated. But even huge and complicated pieces of software are still “just software” at their core, written by normal programmers just doing their job or following their passion.
  15 | 
  16 | What we’re attempting in this series is a very simple browser, and the end result is actually a little under 1000 lines of fairly straightforward Swift code. We’ll focus solely on rendering a subset of HTML, leaving CSS and Javascript as exercises for the reader :). We’ll take many shortcuts and liberties, but in the end you should have an app that can render unstyled, standard HTML pages. And you’ll also have some tools for writing programming language parsers by hand, which you could use to write your own custom language.
  17 | 
  18 | The feature set of our browser is going to be small, but the goal is this: you should be able to render this very browser tutorial web page in the browser itself. Fun, right?
  19 | 
  20 | The Architecture
  21 | 
  22 | Before we dive in to code, let’s look at the overall archicture of what we’re building, to make the challenge ahead more managable. Since HTML is a programming language, we’ll follow a similar architecture to that of most compilers / interpreters, which is a sort of pipeline, where each part of the pipeline takes input and spits something else out for the next component to work with. So what are our inputs and components?
  23 | 
  24 | 
  25 | We start with the raw html, as a String. This either comes from the network or a local file, but it doesn’t really matter.
  26 | We then digest the html string into an array of tokens, in a process known as tokenizing or lexing. This essentially chews up the raw string into common pieces that are easier to digest, such as punctuation characters (<, >, ", etc), whitespaces (newlines, tabs, spaces), digits, or just regular letter characters.
  27 | The tokens are then passed to the Parsing Context, a class whose core purpose is letting other types consume tokens they recognize, while also keeping track of which tokens have already been consumed.
  28 | Next we have structs representing the data we’re parsing, things like, the Document, a tree of Nodes, which consist of Tags and Attributes. We’ll write little parsers for each of these things, that will call into the ParsingContext to consume the tokens they need for their construction.
  29 | Finally, when parsing is complete, we have data we can then use to display our SwiftUI pages with. In a traditional programming language, this might be the point where you output compiled code into an executable or evaluate your data with an interpreter, but here our “interpreter” will simply display a UI.
  30 | 
  31 | 
  32 | With that general archictecture in mind, let’s fire up Xcode and get started.
  33 | 
  34 | Quick Tips
  35 | 
  36 | If you’re coding along as you read this tutorial, I highly recommend typing out all the code yourself, instead of copying and pasting. In my experience, I find this forces you to slow down and work more deliberately, and I think it’ll help you understand things better in the process.
  37 | 
  38 | I’d also recommend changing things as you move along. None of what I’ve written is the definitive way to write this code, and you could probably put your own spin on it. Or extend it to do even more!
  39 | 
  40 | The code I’ll be showing in this tutorial is more or less “finished” as is (we won’t be building to much of it iteratively, because that would take up a whole book!), but please know my browser was built iteratively (you can check out the git history if you’d like to see my stumbles as I went!). Some bits of code in the tutorial will depend upon code we haven’t written yet, so please use your imagination if things don’t compile at every stage.
  41 | 
  42 | Finally, I won’t be providing any unit tests in the tutorial, but you may very well like to include some, especially if you decide to extend your browser after you’re done. I find programming languages lend themselves very well to unit testing, as they have well defined inputs and outputs.
  43 | 
  44 | Starting the project
  45 | 
  46 | Create a new Xcode project using a SwiftUI template. I called my browser Smol because it’s very tiny, but feel free to let your creativity shine here. I made my browser be a Mac app just for ease of playing around with, but you could make yours an iOS app if you wanted, everything will work more or less the same. 
  47 | 
  48 | In your project settings Info tab, add key for “App Transport Security Settings,” and inside that add a key for “Allow arbitrary loads,” setting its value to Yes. This will let us load http and https content from anywhere on the internet and it’s not enabled by default.
  49 | 
  50 | Tokenizing
  51 | 
  52 | Tokenizing is the process of breaking down our program from a String into an array of Token elements, by scanning through the program character by character to build up different tokens. We’ll make 3 types: Token, ScanningCursor, and Tokenizer.
  53 | 
  54 | Token will be a small data struct that combines a token Kind with the text that makes it up. You could also include other data like where in the program this token is located (which would be helpful in showing errors to someone writing html), but it’s not strictly necessary here.
  55 | 
  56 | We define the type along with some initializers that’ll help us as we’re tokenizing.
  57 | 
  58 | struct Token: Equatable, CustomDebugStringConvertible {
  59 | 
  60 |     enum Kind: Equatable {
  61 |         case text, openAngleBracket, closeAngleBracket, forwardSlash, equals, hyphen, singleQuote, doubleQuote, whitespace, bang
  62 |     }
  63 | 
  64 |     let kind: Kind
  65 |     let body: String
  66 | 
  67 |     init(kind: Kind, body: String) {
  68 |         self.kind = kind
  69 |         self.body = body
  70 |     }
  71 | 
  72 |     init?(symbol: Character) {
  73 |         switch symbol {
  74 |         case "<": self.init(kind: .openAngleBracket, body: "<")
  75 |         case ">": self.init(kind: .closeAngleBracket, body: ">")
  76 |         case "/": self.init(kind: .forwardSlash, body: "/")
  77 |         case "=": self.init(kind: .equals, body: "=")
  78 |         case "-": self.init(kind: .hyphen, body: "-")
  79 |         case "'": self.init(kind: .singleQuote, body: "'")
  80 |         case "\"": self.init(kind: .doubleQuote, body: "\"")
  81 |         case "!": self.init(kind: .bang, body: "!")
  82 |         default: return nil
  83 |         }
  84 |     }
  85 | 
  86 |     var debugDescription: String { body }
  87 | }
  88 | 
  89 | 
  90 | Next, the ScanningCursor class will help us keep track of what character we’re looking at at any given moment. This could theoretically just be a part of Tokenizer, but I’ve pulled it out into its own type for possible testability and to keep the tokenizer simple.
  91 | 
  92 | class ScanningCursor {
  93 |     private let programText: String
  94 |     var currentIndex: String.Index
  95 | 
  96 |     var isNotAtEnd: Bool { currentIndex < programText.endIndex }
  97 | 
  98 |     init(programText: String) {
  99 |         self.programText = programText
 100 |         self.currentIndex = programText.startIndex
 101 |     }
 102 | 
 103 |     @discardableResult
 104 |     func advance() -> Character {
 105 |         guard isNotAtEnd else { fatalError() }
 106 | 
 107 |         let currentCharacter = currentCharacter()
 108 |         currentIndex = programText.index(after: currentIndex)
 109 | 
 110 |         return currentCharacter
 111 |     }
 112 | 
 113 |     func currentCharacter() -> Character {
 114 |         programText[currentIndex]
 115 |     }
 116 | 
 117 |     func previousCharacter() -> Character {
 118 |         programText[programText.index(before: currentIndex)]
 119 |     }
 120 | }
 121 | 
 122 | 
 123 | Finally, the Tokenizer itself:
 124 | 
 125 | class Tokenizer {
 126 |     private let cursor: ScanningCursor
 127 |     var scannedTokens = [Token]()
 128 | 
 129 |     init(programText: String) {
 130 |         cursor = ScanningCursor(programText: programText)
 131 |     }
 132 | 
 133 |     func scanAllTokens() -> [Token] {
 134 |         while cursor.isNotAtEnd {
 135 |             scanNextToken()
 136 |         }
 137 | 
 138 |         return scannedTokens
 139 |     }
 140 | 
 141 |     private func scanNextToken() {
 142 |         let next = cursor.advance()
 143 | 
 144 |         if let token = Token(symbol: next) {
 145 |             return scannedTokens.append(token)
 146 |         } else if next.isWhitespace {
 147 |             return scannedTokens.append(Token(kind: .whitespace, body: String(next)))
 148 |         } else {
 149 |             scanText()
 150 |         }
 151 |     }
 152 | 
 153 |     private func scanText() {
 154 |         var body = String(cursor.previousCharacter())
 155 |         while cursor.isNotAtEnd {
 156 |             let next = cursor.currentCharacter()
 157 |             if Token(symbol: next) != nil {
 158 |                 break
 159 |             }
 160 |             if next.isWhitespace { break }
 161 | 
 162 |             body.append(next)
 163 |             cursor.advance()
 164 |         }
 165 |         scannedTokens.append(Token(kind: .text, body: body))
 166 |     }
 167 | }
 168 | 
 169 | 
 170 | The tokenizer’s primary public function runs a loop, attempting to parse tokens until the cursor says we’ve reached the end of the program string.
 171 | 
 172 | The scanNextToken() method tells the cursor to pop off its next character and advance its internal position. With that next character, it then tries to decide what kind of token to make:
 173 | 
 174 | 
 175 | if the character matches one of the punctuation token types, we append that token to our list and return
 176 | if the character is whitespace, we add a single whitespace token
 177 | otherwise, we assume the token will be any other text, so we start scanning that.
 178 | 
 179 | 
 180 | scanText() grabs the most recently popped-off character and starts its own loop, accumulating text characters into a single string. Here we’re considering “text” to be “anything that’s neither whitespace nor one of our recognized punctuation tokens.” This is a kind of strange way to tokenize text, but html is a strange kind of programming language! and we break things up this way to make parsing easier for us later on.
 181 | 
 182 | The Parsing Context
 183 | 
 184 | As mentioned earlier, the Parsing Context, is a class whose core purpose is letting other types consume tokens they recognize, while also keeping track of which tokens have already been consumed. It’s similar to the scanning cursor from earlier, but a little more tailored moving forward (and at times, backward) through a list of tokens.
 185 | 
 186 | You can think of this type as similar to a graphics context, like an OpenGL or Core Graphics context. A graphics context is kind of like a canvas, where you call drawing methods on it (stroke this path, fill this rectangle) or set properties (the current font, the current transform matrix, etc). These calls manipulate the internal state of the context, until you’re ready for it to spit out a final rendered image.
 187 | 
 188 | The parsing context is kind of like that, but instead of adding to an eventual image, we’re subtracting bits of the internal token state while we parse out types. When we’re all done parsing, the context should ideally be at the end of its list of tokens and we should have all our parsed data.
 189 | 
 190 | class ParsingContext {
 191 | 
 192 |     private let tokens: [Token]
 193 |     private var tokenIndexStack = [0]
 194 |     private var currentTokenIndex: Int {
 195 |         get { tokenIndexStack.last! }
 196 |         set { tokenIndexStack[tokenIndexStack.endIndex - 1] = newValue }
 197 |     }
 198 | 
 199 |     var isNotAtEnd: Bool {
 200 |         currentTokenIndex < tokens.count
 201 |     }
 202 | 
 203 |     // These might be whitespace tokens.
 204 |     var currentToken: Token { isNotAtEnd == false ? tokens.last! : tokens[currentTokenIndex] }
 205 |     var nextToken: Token { tokens[currentTokenIndex + 1] }
 206 |     var nextNextToken: Token { tokens[currentTokenIndex + 2] }
 207 |     var previousToken: Token { tokens[currentTokenIndex - 1] }
 208 | 
 209 |     init(tokens: [Token]) {
 210 |         self.tokens = tokens
 211 |     }
 212 | 
 213 |     enum ParseError: Error {
 214 |         case unexpectedToken(Token, feedback: String)
 215 |         case failedToParse
 216 |     }
 217 | 
 218 | 
 219 | We start with some properties around accessing the tokens. We store a list of all tokens and access them by an index, which is our current parsing location. Instead of storing a single index, we instead have a stack of indexes, with the current index being the top of this index stack. We’ll look into this more below, but it allows us to move forward and backward through the list of tokens as we’re parsing.
 220 | 
 221 |     private func advance(when predicate: (Token) -> Bool, skipWhitespaceTokens: Bool = true) -> Bool {
 222 | 
 223 |         var skippedWhitespaceCount = 0
 224 |         if skipWhitespaceTokens {
 225 |             while isNotAtEnd && currentToken.kind == .whitespace {
 226 |                 currentTokenIndex += 1
 227 |                 skippedWhitespaceCount += 1
 228 |             }
 229 |         }
 230 | 
 231 |         guard isNotAtEnd else { return false }
 232 | 
 233 |         if predicate(tokens[currentTokenIndex]) {
 234 |             currentTokenIndex += 1
 235 |             return true
 236 |         } else {
 237 |             currentTokenIndex -= skippedWhitespaceCount
 238 |             return false
 239 |         }
 240 |     }
 241 | 
 242 |     @discardableResult
 243 |     func consume(tokenKind kind: Token.Kind, feedback: String) throws -> Token {
 244 |         try consume(where: { $0.kind == kind }, feedback: feedback)
 245 |     }
 246 | 
 247 |     @discardableResult
 248 |     func consume(where predicate: (Token) -> Bool, skipWhitespaceTokens: Bool = true, feedback: String) throws -> Token {
 249 |         let oldCurrentToken = self.currentToken
 250 |         guard advance(when: predicate, skipWhitespaceTokens: skipWhitespaceTokens) else {
 251 |             throw ParseError.unexpectedToken(oldCurrentToken, feedback: feedback)
 252 |         }
 253 |         return previousToken
 254 |     }
 255 | 
 256 | 
 257 | Next, we have the primary methods used for updating the token state, by advancing the cursor when we finding (advance(when:...)) and consuming matching tokens. The advance() method more or less just checks to see if the given predicate closure matches the current token. Most of the time in programming languages, we ignore whitespace tokens, so this method does that by default, but it has a flag to not skip, since we’ll need that later on for some of our parsing.
 258 | 
 259 | The consume(...) methods build upon advance(...), but will throw an error if matching fails. From this point onward in the parser architecture, we use Swift errors as a means of control flow to indicate more or less that parsing a certain token or syntax node was unsucessful. This doesn’t necessarily mean there is an error, only that we weren’t able to interpret a specific part a certain way (it might mean it should be interpreted another way).
 260 | 
 261 | The consume method takes a feedback string to make parsing failures a little clearer, and to make bugs in the parser a little easier to track down.
 262 | 
 263 |     // MARK: - Helpers
 264 | 
 265 |     /// Use this method when you want to accumulate results until parsing fails, but you want to keep what you've found so far.
 266 |     func untilThrowOrEndOfTokensReached<ConsumedType>(perform: () throws -> ConsumedType) -> [ConsumedType] {
 267 | 
 268 |         var results = [ConsumedType]()
 269 | 
 270 |         do {
 271 |             while isNotAtEnd {
 272 |                 results.append(try perform())
 273 |             }
 274 |         } catch {
 275 |             return results
 276 |         }
 277 |         return results
 278 |     }
 279 | 
 280 |     func attempt<ContentType>(action: () throws -> ContentType) throws -> ContentType {
 281 |         tokenIndexStack.append(currentTokenIndex)
 282 |         var shouldRevertIndexStack = true
 283 | 
 284 |         defer {
 285 |             // Pop the stack if `try action()` fails.
 286 |             // doing it this way, instead of catching + rethrowing
 287 |             // so that the error chain continues to the original error, not our rethrow
 288 |             if shouldRevertIndexStack {
 289 |                 _ = tokenIndexStack.popLast()
 290 |             }
 291 |         }
 292 | 
 293 |         let result = try action()
 294 | 
 295 |         // we succeeded, so pop the token index stack, and use THAT value as the new current index
 296 |         currentTokenIndex = tokenIndexStack.popLast()!
 297 |         shouldRevertIndexStack = false
 298 |         return result
 299 |     }
 300 | 
 301 |     func choose<ContentType>(from choices: [() throws -> ContentType]) throws -> ContentType {
 302 |         try attempt(action: {
 303 |             var mostRecentError: Error = ParseError.failedToParse
 304 | 
 305 |             for choice in choices {
 306 | 
 307 |                 do {
 308 |                     return try attempt(action: {
 309 |                         try choice()
 310 |                     })
 311 |                 } catch {
 312 |                     mostRecentError = error
 313 |                 }
 314 |             }
 315 | 
 316 |             throw mostRecentError
 317 |         })
 318 |     }
 319 | 
 320 |     @discardableResult
 321 |     func consumeBetween<ContentType>(leftToken: Token.Kind, rightToken: Token.Kind, content: () throws -> ContentType) throws -> ContentType {
 322 |         try consume(tokenKind: leftToken, feedback: "Expected a \(leftToken)")
 323 |         let consumedContent = try content()
 324 |         try consume(tokenKind: rightToken, feedback: "Expected a \(rightToken)")
 325 | 
 326 |         return consumedContent
 327 |     }
 328 | }
 329 | 
 330 | 
 331 | Finally, we have 4 helper methods that we’ll use while parsing.
 332 | 
 333 | untilThrowOrEndOfTokensReached(perform:) calls its perform closure in a loop, accumulating values returned from it in an array, which it eventually returns either when the end of tokens is reached or (more likely) when the closure throws an error. In practice, we’ll be calling other methods of the parsing context inside that closure while parsing syntax nodes. The point of this method is to essentially say “after a while, parsing failed, so I’m gonna give you what I’ve successfully parsed until that point.”
 334 | 
 335 | attempt(action:) is extremely useful, as it allows us to try multiple parsing actions in the hopes they succeed (and thus, move the token cursor forward), but if the action throws, we’re able to revert back to the previous cursor position. If we didn’t use attempt(action:) when parsing and e.g., called consume() twice successfully, and then a third time unsucessfully, we would have failed to parse a whole thing, but we would have also moved the cursor along with us, now in a spot unable to try finding something else. attempt(:) solves this for us, and is why we use a stack of token indexes instead of just a single index (this also works recursively).
 336 | 
 337 | choose(from:) takes an array of closures with parsing calls in them, each returning a value. It then runs through the array, calling each closure in order. If a closure successfully returns a value, choose will return that value. If a closure throws, then we move on to the next closure to try that. All of this is wrapped in an attempt(action:) call so that if parsing in one closure fails, the next one gets a fresh start before it parses. This method is useful when parsing could result in multiple possibilities in the same place in the program, and you frequently (but not always) would want your return type to be an enum with a choice for each of its cases.
 338 | 
 339 | Finally, consumeBetween(leftToken:, rightToken:, content:) helps us in the case when things are wrapped in certain tokens, for example quotes, parentheses, or angle brackets. It tries to consume the left token, then tries the content closure, and finally tries to consume the right token. If all of that succeeded, it returns whatever was returned by the closure.
 340 | 
 341 | And that completes the ParsingContext, which models common operations used throughout the HTML parsing process (and which could easily be reused with parsers for your own programming language too).
 342 | 
 343 | Parsing HTML
 344 | 
 345 | Now that we’ve built ourselves parsing tools, lets use them to parse out HTML into our own data types (in programming language theory, these are known as “abstract syntax trees / nodes,” which is a fancy way of saying a set of types that are usually arranged in some sort of hierarchy or graph). We’ll only make use of a few types, as most of HTML is fairly generic and has a similar structure all the way down.
 346 | 
 347 | To identify these syntax tree nodes, let’s make a protocol for anything that is Parsable:
 348 | 
 349 | protocol Parsable {
 350 |     static func parse(context: ParsingContext) throws -> Self
 351 | }
 352 | 
 353 | 
 354 | Types that conform to this protocol will have to implement the above static method and return a parsed version of themselves, or throw an error if they couldn’t be parsed out of the given ParsingContext. You could alternatively make this an initializer method instead, but that will shadow the auto-generated struct initializers, which is kind of annoying.
 355 | 
 356 | Document
 357 | 
 358 | Let’s start at the top, with the Document. An HTML document is our model that more or less lines up with the html “file” as a whole. We’ll keep ours very simple:
 359 | 
 360 | struct Document: Hashable, Parsable {
 361 | 
 362 |     enum DocumentError: Error {
 363 |         case unableToFindHTMLNode
 364 |     }
 365 | 
 366 |     let htmlNode: Node
 367 | 
 368 |     static func parse(context: ParsingContext) throws -> Document {
 369 |         let nodes = context.untilThrowOrEndOfTokensReached {
 370 |             try Node.parse(context: context, options: nil)
 371 |         }
 372 | 
 373 |         guard let htmlNode = nodes.first(where: { $0.element.lowercased() == "html" }) else {
 374 |             throw DocumentError.unableToFindHTMLNode
 375 |         }
 376 | 
 377 |         return Document(htmlNode: htmlNode)
 378 |     }
 379 | }
 380 | 
 381 | 
 382 | An html document has 0 or more “nodes” (tags) at the top level. It might have a <!doctype> tag, and it ideally should have an <html> tag too. Our Document.parse(:) implementation asks the given parsing context to parse out Nodes until an error is thrown or we’ve reached the end of the tokens. Then, we search through that array of nodes, looking for the html node, and finally, we return the document initialized with that found node (and we ignore any doctype or other nodes we might find). If we can’t find any html node, we throw an error indicating such. It might be that the program really didn’t contain an html tag, or more likely, that our Node parser failed to handle something inside the html node and errored out.
 383 | 
 384 | Our parser system is going to be kind of strict in what it accepts, which is contrary to how the Big Browsers tend to work, where they’ll accept pretty much anything you throw at them. Our approach favours simplicity of implementation to get concepts across, at the cost of compatibility with lots of websites. As you build out your browser, feel free to expand what your parser can handle :)
 385 | 
 386 | Node
 387 | 
 388 | Now it’s time for the real meat and potatoes of our syntax tree, the Node, which represents a “node” in the html document. It’s more or less the data model equivalent of a <tag>, any attributes inside of the tag itself, and any children nested between the tags (the distinction between a node, an element, and a tag is subtle, and you may be used to using the terms interchangeably, but I’ll try to keep them separate as best I can).
 389 | 
 390 | Let’s start the Node type with some internal types and properties:
 391 | 
 392 | struct Node: Hashable, Parsable {
 393 | 
 394 |     struct InternalElement {
 395 |         static let textRun = "__textRun"
 396 |         static let comment = "__comment"
 397 |     }
 398 | 
 399 |     enum Content: Hashable {
 400 |         case text(String)
 401 |         case childNodes([Node])
 402 |         case voidNode
 403 |     }
 404 | 
 405 |     enum NodeParseError: Error {
 406 |         case closingTagDidNotMatchOpeningTag(opening: String, closing: String)
 407 |         case openingTagWasActuallyClosing(tagName: String)
 408 |         case closingTagWasActuallyOpening(tagName: String)
 409 |         case didNotFindAnyText
 410 |     }
 411 | 
 412 |     let element: String
 413 |     let content: Content
 414 |     let attributes: [Attribute]
 415 | 
 416 | 
 417 | InternalElement lists some private element names we’ll use for bits of the html file that don’t fall under normal html tag rules (we’ll see more of them later).
 418 | 
 419 | Then we have the Content enum, which models the stuff inside of our node. This says, a node can either contain text, child nodes, or be a “void” node (that is, a node that only has a start tag, no end tag and no children. <img> is an example of a void node).
 420 | 
 421 | Next, we have an error type defined to list the things that can go wrong during parsing and which act as control flow.
 422 | 
 423 | Finally, we have Node’s properties: its element (or tag name), the aforementioned content, and any attributes that were in the start tag.
 424 | 
 425 | Now it’s on to parsing the node itself, which we’ll break down into some chunks:
 426 | 
 427 | static func parse(context: ParsingContext) throws -> Node {
 428 |     let startTag = try Tag.parse(context: context)
 429 | 
 430 |     guard startTag.isEnd == false else {
 431 |         throw NodeParseError.openingTagWasActuallyClosing(tagName: startTag.element)
 432 |     }
 433 | 
 434 |     if startTag.element.lowercased() == "doctype" {
 435 |         return Node(element: "doctype", content: .voidNode, attributes: startTag.attributes)
 436 |     }
 437 | 
 438 |     if startTag.isVoidElement {
 439 |         return Node(element: startTag.element, content: .voidNode, attributes: startTag.attributes)
 440 |     }
 441 | 
 442 | 
 443 | We begin by trying to parse a start tag (which we’ll get to in a bit). Then, we check some conditions to see if we can bail early:
 444 | 
 445 | 
 446 | If the tag that got parsed was an end tag (eg </something>), then we throw an error. Alternatively, we could break Tag into 2 types, StartTag and EndTag, and let start tags fail to parse end tags.
 447 | Then we check to see if our start tag is a doctype tag, in which case we return immediately.
 448 | Finally, we check to see if the start tag represents a void element, and if so we also return immediately.
 449 | 
 450 | 
 451 | If none of those conditions are met, we keep parsing. At this point, we have a start tag and we need to look for 0 or more children we might have, before reaching an end tag.
 452 | 
 453 | To parse child nodes, we’re going to ask the context to parse nodes  until we hit an error. This way, we’ll get 0 or more child nodes. Inside that loop, we’re going to ask the context to choose from a few possibilities:
 454 | 
 455 |     let children = context.untilThrowOrEndOfTokensReached(perform: {
 456 |         try context.choose(from: [
 457 |             { try Node.parse(context: context) },
 458 | 
 459 | 
 460 | The child might be a normal Node of some kind, so we recursively call Node.parse().
 461 | 
 462 | {
 463 |     let textContents = context.untilThrowOrEndOfTokensReached {
 464 |         try context.consume(where: { $0.kind != .openAngleBracket }, skipWhitespaceTokens: false, feedback: "Expected a non `<` token")
 465 |     }
 466 |     guard textContents.isEmpty == false else {
 467 |         throw NodeParseError.didNotFindAnyText
 468 |     }
 469 | 
 470 |     let contentRun = textContents
 471 |             .map(\.body)
 472 |             .joined()
 473 |             .replacingOccurrences(of: "&#x000A;", with: "")
 474 |             .replacingOccurrences(of: "\n", with: " ")
 475 |             .replacingOccurrences(of: "\t", with: " ")
 476 |             .replacingOccurrences(of: "&lt;", with: "<")
 477 |             .replacingOccurrences(of: "&gt;", with: ">")
 478 |             .replacingOccurrences(of: "&quot;", with: "\"")
 479 |             .replacingOccurrences(of: "&amp;", with: "&")
 480 | 
 481 |     return Node(element: InternalElement.textRun, content: .text(contentRun), attributes: [])
 482 | },
 483 | 
 484 | 
 485 | If it’s not a standard node, it might be a text run node. Text runs in html are not real nodes like <div> or <p>, instead they’re the any text content inside of other tags. So if we have a node like <p>Hi there</p>, this will get parsed out to a p Node, whose content is .childNodes(children), and children will be an array with a single Node, whose content is .text("Hi there"). This structure feels weird, but it allows us to parse more complicated nodes like <p>Hi there, <bold>friend</bold></p>. In short, we’re wrapping otherwise un-tagged text into a pretend <text-run> tag and then treating it as we do other nodes.
 486 | 
 487 | To parse a text run, we first consume every token that’s not an < character, which we assume might be the beginning of a tag. If we find any contents, we then join the contents' body together into one big string.
 488 | 
 489 | Then, we do some quick and dirty text replacement, replacing encoded html entities with their display characters and non-space whitespaces with spaces for display (this doesn’t follow the html standard for whitespaces perfectly, but it works well enough). With all the replacement done, we return the text run node.   
 490 | 
 491 | {
 492 |     try context.consumeBetween(leftToken: .openAngleBracket, rightToken: .closeAngleBracket) {
 493 |             try context.consume(tokenKind: .bang, feedback: "Expected comment to begin with a bang")
 494 |             try context.consume(tokenKind: .hyphen, feedback: "Expected comment to have a hyphen after the bang")
 495 |             try context.consume(tokenKind: .hyphen, feedback: "Expected comment to have two hyphens after the bang")
 496 | 
 497 |             var done = false
 498 |             while done == false {
 499 | 
 500 |                 if context.currentToken.kind == .hyphen && context.nextToken.kind == .hyphen && context.nextNextToken.kind == .closeAngleBracket {
 501 | 
 502 |                     try context.consume(tokenKind: .hyphen, feedback: "-")
 503 |                     try context.consume(tokenKind: .hyphen, feedback: "-")
 504 |                     done = true
 505 |                 } else {
 506 |                     try context.consume(where: { _ in true }, skipWhitespaceTokens: false, feedback: "consuming comment contents")
 507 |                 }
 508 |             }
 509 | 
 510 |             return Node(element: InternalElement.comment, content: .voidNode, attributes: [])
 511 |         }
 512 | }])})
 513 | 
 514 | 
 515 | Finally, if the child node wasn’t a normal node, nor a text run node, we see if it was perhaps a comment node, which takes the form <!-- your comment here -->. Looking inside angle brackets, we first attempt to consume a bang, then 2 hyphen tokens. After that, we loop, peeking at the next 3 tokens looking for the ending --> pattern. If we don’t find that pattern, we just consume and ignore whatever content was there. Once we’re done munching tokens, we return the internal comment node.
 516 | 
 517 |         .filter {
 518 |             if $0.element == InternalElement.comment { return false }
 519 |             if $0.element != InternalElement.textRun { return true }
 520 | 
 521 |             // filter out empty text run nodes
 522 |             return $0.textContent?.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty == false
 523 |         }
 524 | 
 525 | 
 526 | As a very last step of parsing child nodes, we remove nodes that are comments or nodes that are text runs with empty text. Everything else, we keep. And now we’re done parsing child nodes.
 527 | 
 528 | 
 529 |     let endTag = try Tag.parse(context: context)
 530 |     guard endTag.isEnd else {
 531 |         throw NodeParseError.closingTagWasActuallyOpening(tagName: endTag.element)
 532 |     }
 533 | 
 534 |     guard startTag.element == endTag.element else {
 535 |         throw NodeParseError.closingTagDidNotMatchOpeningTag(opening: startTag.element, closing: endTag.element)
 536 |     }
 537 | 
 538 |     return .init(
 539 |         element: startTag.element,
 540 |         content: .childNodes(children),
 541 |         attributes: startTag.attributes
 542 |     )
 543 | }
 544 | 
 545 | 
 546 | After the child nodes are parsed, all that’s left is to parse the end tag, make sure it’s really an end tag, and ensure that it matches the start tag. If all of that succeeded, we return the fully constructed Node. Most of what we just did was bookkeeping (checking tags, make sure start / end tags match), and then parsing the node’s children, if any.
 547 | 
 548 | Tag
 549 | 
 550 | We’ve papered over Tag parsing, though, so let’s look at that now:
 551 | 
 552 | struct Tag: Parsable {
 553 | 
 554 |     let element: String
 555 |     let isEnd: Bool
 556 |     let attributes: [Attribute]
 557 | 
 558 |     var isVoidElement: Bool {
 559 |         ["area", "base", "br", "col", "embed", "hr", "img", "input", "link", "meta", "source", "track", "wbr"].contains(element)
 560 |     }
 561 | 
 562 | 
 563 | We start our Tag type with some properties, alluding to the Attribute type we’ll see shortly as well. We also list the known void elements to determine if our element should be considered void. Now on to the parsing:
 564 | 
 565 |     static func parse(context: ParsingContext, options: ParsingOptions?) throws -> Tag {
 566 |         try context.consumeBetween(leftToken: .openAngleBracket, rightToken: .closeAngleBracket) {
 567 |             let slashToken = try? context.consume(tokenKind: .forwardSlash, feedback: "Expected a `/`")
 568 |             let _ = try? context.consume(tokenKind: .bang, feedback: "Expected a `!`")
 569 | 
 570 | 
 571 | A tag is wrapped in < and > angle brackets. Within those, we first look for an initial forward slash token, and if we find it we assume we’re parsing an end tag (we use try? to optionally parse this — if we don’t find the slash, we’re not considering that an error worth bailing from). We also look for an optional bang token and just completely ignore it if we find it (this is for the <!doctype> tag).
 572 | 
 573 |             let identifier = try context.consume(tokenKind: .text, feedback: "Expected a tag name")
 574 | 
 575 |             let attributes = context.untilThrowOrEndOfTokensReached(perform: {
 576 |                 try context.attempt(action: {
 577 |                     try Attribute.parse(context: context, options: options)
 578 |                 })
 579 |             })
 580 | 
 581 | 
 582 | Next, we parse an identifier that we’ll use for the tag’s element. Then we attempt to parse as many attributes as we can (there may be 0).
 583 | 
 584 |             // If there's a trailing slash (eg <img />), consume it but ignore it. this is invalid html
 585 |             _ = try? context.consume(tokenKind: .forwardSlash, feedback: "Expected a trailing `/`")
 586 |             return Tag(element: identifier.body, isEnd: slashToken != nil, attributes: attributes)
 587 |         }
 588 |     }
 589 | }
 590 | 
 591 | 
 592 | Finally, optionally look for and ignore a trailing slash at the end of the tag, as it’s not actually valid html (this was news to me when I started working on the browser). However, it’s extremely common, so I thought it warranted handling here to make more of the web work. With that out of the way, we return our completed tag.
 593 | 
 594 | Attribute
 595 | 
 596 | Ok, last part of the parser! the attributes inside a tag.
 597 | 
 598 | struct Attribute: Hashable, Parsable {
 599 |     let key: String
 600 |     let value: String
 601 | 
 602 |     enum AttributeParseError: Error {
 603 |         case emptyAttributeValue(key: String)
 604 |     }
 605 | 
 606 |     static func parse(context: ParsingContext, options: ParsingOptions?) throws -> Attribute {
 607 | 
 608 |         let key = try context.consume(tokenKind: .text, feedback: "Expected an attribute name")
 609 | 
 610 |         guard let _ = try? context.consume(tokenKind: .equals, feedback: "Expected an equals sign") else {
 611 |             return Attribute(key: key.body, value: key.body)
 612 |         }
 613 | 
 614 | 
 615 | Attributes are (usually) key-value pairs, so those are our properties (for attributes that don’t have explicit values, we’ll just repeat the key for the value). 
 616 | 
 617 | Then, we start parsing. First we parse the key, then we look for an equals sign token. If we don’t find it, we assume this attribute is the valueless kind and return it immediately. Otherwise, we parse the value, as a choice:
 618 | 
 619 | let value = try context.choose(from: [
 620 | {
 621 |     try context.consumeBetween(leftToken: .doubleQuote, rightToken: .doubleQuote) {
 622 |         let textContents = context.untilThrowOrEndOfTokensReached {
 623 |             try context.consume(where: { $0.kind != .doubleQuote }, skipWhitespaceTokens: false, feedback: "Expected a non quote token")
 624 |         }
 625 | 
 626 |         return textContents
 627 |             .map(\.body)
 628 |             .joined()
 629 |     }
 630 | },
 631 | 
 632 | 
 633 | First choice: the value is between double quotes, and we consume everything inside that isn’t a double quote (and we don’t skip whitespaces either). Then we join all those tokens together and return that as the value.
 634 | 
 635 | {
 636 |     try context.consumeBetween(leftToken: .singleQuote, rightToken: .singleQuote) {
 637 |         let textContents = context.untilThrowOrEndOfTokensReached {
 638 |             try context.consume(where: { $0.kind != .singleQuote }, skipWhitespaceTokens: false, feedback: "Expected a non single quote token")
 639 |         }
 640 | 
 641 |         return textContents
 642 |             .map(\.body)
 643 |             .joined()
 644 |     }
 645 | },
 646 | 
 647 | 
 648 | Second choice: same thing as before, except between single quotes.
 649 | 
 650 | {
 651 |     let textContents = context.untilThrowOrEndOfTokensReached {
 652 |         try context.consume(
 653 |             where: {
 654 |                 $0.kind != .singleQuote && $0.kind != .doubleQuote && $0.kind != .whitespace && $0.kind != .closeAngleBracket
 655 |         },
 656 |             skipWhitespaceTokens: false,
 657 |             feedback: "Expected non-whitespace, non-quote characters")
 658 |     }
 659 | 
 660 |     guard textContents.isEmpty == false else {
 661 |         throw AttributeParseError.emptyAttributeValue(key: key.body)
 662 |     }
 663 | 
 664 |     return textContents
 665 |         .map(\.body)
 666 |         .joined()
 667 | }
 668 | 
 669 | 
 670 | Final choice: we look for a value that’s not wrapped in any kind of quotes. These kinds of values are delimitted by whitespace (or an angle bracket), so we consume basically everything else, make sure we actually found something non-empty, and join those tokens together into a value.
 671 | 
 672 |         ])
 673 | 
 674 |         return Attribute(key: key.body, value: value)
 675 |     }
 676 | }
 677 | 
 678 | 
 679 | Last, we return the completed attribute.
 680 | 
 681 | End of Part 1
 682 | 
 683 | This completes the end of part 1! We built ourselves some tools for breaking apart a program string into tokens and parsing them. And then we built some data types that know how to parse themselves using those tools. HTML is a kind of strange language, but we saw some familiar patterns repeated in multiple places (things being wrapped inside others, for example).
 684 | 
 685 | In the next part, we’ll take the data we just parsed and render it with SwiftUI.
 686 | 
 687 | Part 2: Rendering in SwiftUI
 688 | 
 689 | The Architecture
 690 | 
 691 | The architecture of our rendering engine should look pretty familiar to anyone who’s worked with SwiftUI before: we’re more or less just going to have views which render our node hierarchy. It’s almost exclusively composed of standard SwiftUI views, plus a controller object for loading HTML pages, and a few extensions on the Node type to more easily work with its properties. Here are the main pieces we’ll be working with.
 692 | 
 693 | 
 694 | PageController is responsible for loading web urls asynchronously and parsing them into Documents. It also maintains the back / forward stacks of documents.
 695 | Some views:
 696 | 
 697 | 
 698 | BrowserView is the primary view, containing our chrome (back / forward / address bar) and the document view.
 699 | WebDocumentView displays either a homepage, error page, or the contents of the loaded page, depending on the page controller’s state.
 700 | BodyView is the true beginnings of our rendering engine, it nests our page’s content in a scroll view.
 701 | BlocksView displays views for 0 or more nodes in a vertical stack. It picks a different view depending on the node’s element.
 702 | InlineContentWrappingBlockView combines the text of all its inline elements into one big Text for rendering.
 703 | ListNodeView renders ordered or unordered lists and their items.
 704 | ImageView asynchronously downloads and renders img nodes.
 705 | 
 706 | Extensions on Node for accessing its content.
 707 | 
 708 | 
 709 | The Page Controller
 710 | 
 711 | The PageController is our main controller object, responsible for loading pages, parsing them, and managing the back / forward stacks:
 712 | 
 713 | class PageController: ObservableObject {
 714 | 
 715 |     enum State {
 716 |         case notLoaded
 717 |         case loaded(Document, URL)
 718 |         case failed(Error)
 719 |     }
 720 | 
 721 |     private enum LoadingError: Error {
 722 |         case failedToLoad(URL)
 723 |     }
 724 | 
 725 |     @Published var state = State.notLoaded {
 726 |         didSet {
 727 |             if let currentlyLoadedDocument {
 728 |                 address = currentlyLoadedDocument.1.absoluteString
 729 |             }
 730 |         }
 731 |     }
 732 |     var address = "https://nearthespeedoflight.com/smol.html"
 733 | 
 734 |     private var backStack: [(Document, URL)] = []
 735 |     private var forwardStack: [(Document, URL)] = []
 736 | 
 737 |     var canGoBack: Bool { backStack.isEmpty == false }
 738 |     var canGoForward: Bool { forwardStack.isEmpty == false }
 739 | 
 740 | 
 741 | First we set up some nested types. The controller can be in one of three States: an initial unloaded state (maybe you show a homepage?), a loaded state with the parsed document and the URL it came from, and the failed error state.
 742 | 
 743 | Then we have some properties, mainly the controller’s current state, its current address string, and the back / forward stacks.
 744 | 
 745 |     func loadPage(at url: URL) {
 746 |         Task {
 747 |             let newState: State
 748 |             do {
 749 |                 let (data, response) = try await URLSession.shared.data(from: url)
 750 | 
 751 |                 if let currentlyLoadedDocument {
 752 |                     backStack.append(currentlyLoadedDocument)
 753 |                     forwardStack = []
 754 |                 }
 755 | 
 756 |                 let htmlString = String(data: data, encoding: .utf8) ?? ""
 757 |                 let tokenizer = Tokenizer(programText: htmlString)
 758 |                 let context = try ParsingContext(tokens: tokenizer.scanAllTokens())
 759 | 
 760 |                 newState = .loaded(try Document.parse(context: context, options: nil), response.url ?? url)
 761 |             } catch {
 762 |                 print("error loading page: \(error)")
 763 |                 newState = .failed(error)
 764 |             }
 765 | 
 766 |             await MainActor.run {
 767 |                 state = newState
 768 |             }
 769 |         }
 770 |     }
 771 | 
 772 | 
 773 | To load a page, we kick off an async Task, await the loading of the given url, then we put the data through our parser pipeline. We also set the back / forward stacks to account for the state change that’s about to happen.
 774 | 
 775 | This is all made a little awkward due to error handling, as we want to catch any errors that happen here: there could be URL related errors, there could be an error in the parsing context, or there could be an error parsing the document. If there is an error, we want to record it. This wouldn’t be so bad on its own, but we don’t want to do any of this parsing on the main actor, where it could freeze the UI, but we must update our controller’s state property on the main actor, as our view depends on that property to draw itself.
 776 | 
 777 |     private var currentlyLoadedDocument: (Document, URL)? {
 778 |         switch state {
 779 |         case .notLoaded, .failed: return nil
 780 |         case let .loaded(document, url): return (document, url)
 781 |         }
 782 |     }
 783 | 
 784 |     func goBack() {
 785 |         guard let (previousDocument, previousURL) = backStack.popLast() else { return }
 786 |         if let currentlyLoadedDocument {
 787 |             forwardStack.append(currentlyLoadedDocument)
 788 |         }
 789 |         state = .loaded(previousDocument, previousURL)
 790 |     }
 791 | 
 792 |     func goForward() {
 793 |         guard let (nextDocument, nextURL) = forwardStack.popLast() else { return }
 794 |         if let currentlyLoadedDocument {
 795 |             backStack.append(currentlyLoadedDocument)
 796 |         }
 797 |         state = .loaded(nextDocument, nextURL)
 798 |     }
 799 | }
 800 | 
 801 | 
 802 | Finally, we have a helper property for accessing the currently loaded document, if any, and methods for going back and forward. That wraps up our controller. Next, we’ll see how the views make use of it while displaying our nodes.
 803 | 
 804 | The Browser View
 805 | 
 806 | As mentioned above, our BrowserView is the primary view for our browser window: it composes the “chrome” of our UI, plus the actual rendered content in another view. Our UI is going to be very simple, but you could extend it to use tabs, or even something more imaginative if you want :)
 807 | 
 808 | struct BrowserView: View {
 809 |     @ObservedObject var controller: PageController
 810 |     @FocusState private var addressIsFocused: Bool
 811 | 
 812 | 
 813 | All we need are 2 properties, an observed page controller and the focus state of the address textfield, so that focus works like you’d expect as we navigate.
 814 | 
 815 |     var body: some View {
 816 |         VStack(spacing: 0) {
 817 |             HStack {
 818 |                 HStack(spacing: 0) {
 819 |                     Button(action: { controller.goBack() }) {
 820 |                         Image(systemName: "arrowtriangle.left.fill")
 821 |                     }.disabled(controller.canGoBack == false)
 822 |                     Button(action: { controller.goForward() }) {
 823 |                         Image(systemName: "arrowtriangle.right.fill")
 824 |                     }.disabled(controller.canGoForward == false)
 825 |                 }
 826 |                 TextField("Address", text: $controller.address)
 827 |                     .onSubmit {
 828 |                         addressIsFocused = false
 829 |                         guard let url = URL(string: controller.address) else { return }
 830 |                         controller.loadPage(at: fullURL(forURLToLoad: url))
 831 |                     }
 832 |                     .textFieldStyle(RoundedBorderTextFieldStyle())
 833 |                     .focused($addressIsFocused)
 834 |             }
 835 |             .padding()
 836 |             Divider()
 837 | 
 838 | 
 839 | The body of our view until this point is all about the chrome. We create our back / forward buttons and the address bar, and we bind their actions to our controller.
 840 | 
 841 |             WebDocumentView(controller: controller)
 842 |                 .background(.white)
 843 |                 .environment(\.openURL, .init(handler: { url in
 844 |                     controller.loadPage(at: fullURL(forURLToLoad: url))
 845 |                     addressIsFocused = false
 846 |                     return .handled
 847 |                 }))
 848 |         }
 849 | 
 850 | 
 851 | We configure the WebDocumentView and override SwiftUI’s openURL environment value. When the user clicks a link in our app, SwiftUI invokes this callback, giving our app a chance to handle the URL. With the given URL, we construct an absolute URL (below), adjust the text field’s focus, and tell the system we handled the url (we could also tell the system to handle it instead if the URL was eg mailto:..., but I’ll leave that to you). 
 852 | 
 853 |         .environment(\.urlBuilder, fullURL(forURLToLoad:))
 854 |     }
 855 | 
 856 |     private func fullURL(forURLToLoad urlToLoad: URL) -> URL {
 857 |         if urlToLoad.host != nil { return urlToLoad }
 858 | 
 859 |         switch controller.state {
 860 |         case .failed, .notLoaded: return urlToLoad
 861 |         case .loaded(_, let loadedURL):
 862 |             return URL(string: urlToLoad.path, relativeTo: loadedURL.deletingLastPathComponent()) ?? urlToLoad
 863 |         }
 864 |     }
 865 | } // End of BrowserView
 866 | 
 867 | private struct URLBuilderKey: EnvironmentKey {
 868 |     static let defaultValue: (URL) -> URL = { $0 }
 869 | }
 870 | 
 871 | extension EnvironmentValues {
 872 |     /// A function that takes a (potentially "relative") web url to load, and fleshes it out to a full url that includes a host.
 873 |     var urlBuilder: (URL) -> URL {
 874 |         get { self[URLBuilderKey.self] }
 875 |         set { self[URLBuilderKey.self] = newValue }
 876 |     }
 877 | }
 878 | 
 879 | 
 880 | Finally, we use the environment modifier for a custom environment value. The urlBuilder is a closure / function responsible for taking a URL (one that’s possibly relative, eg just /page.html vs https://example.com/page.html) and expanding it to an absolute URL so that pages and assets like images can be loaded.
 881 | 
 882 | We do this as an environment value so that other views in the hierarchy can access the functionality.
 883 | 
 884 | Web Document View
 885 | 
 886 | The WebDocumentView takes up the majority of space in our browser window. What it shows depends on the state of the page controller, either showing a simple home page, error screen, or the loaded content.
 887 | 
 888 | struct WebDocumentView: View {
 889 |     @ObservedObject var controller: PageController
 890 | 
 891 |     var body: some View {
 892 |         switch controller.state {
 893 |         case .notLoaded:
 894 |             Text("Let's load a web page!")
 895 |                 .frame(maxWidth: .infinity, maxHeight: .infinity)
 896 |         case .failed(let error):
 897 |             Text(verbatim: "Failed to load page. Error: \(error)")
 898 |                 .frame(maxWidth: .infinity, maxHeight: .infinity)
 899 |         case .loaded(let document, _):
 900 |             BodyView(bodyNode: document.htmlNode.firstDirectChild(named: "body")!)
 901 |                 .navigationTitle(
 902 |                     document
 903 |                         .htmlNode
 904 |                         .firstDirectChild(named: "head")?
 905 |                         .firstDirectChild(named: "title")?
 906 |                         .firstDirectChild(named: Node.InternalElement.textRun)?
 907 |                         .textContent ?? "Smol"
 908 |                 )
 909 |                 .environment(\.font, Font.custom("Times", size: 16))
 910 |         }
 911 |     }
 912 | }
 913 | 
 914 | 
 915 | The BodyView accesses some properties on Node which we’ll write shortly for accessing child nodes more easily. We drill down to find the page’s title, if it has one, and set that as our window title. Finally, we set a default font on the document’s text. “Times” is the font you see in most browsers with unstylized text (but you’re allowed to choose any font you’d like here).
 916 | 
 917 | Node extensions
 918 | 
 919 | Before we go any further with our views, let’s write those helpers in an extension on Node.
 920 | 
 921 | extension Node {
 922 |     var childNodes: [Node] {
 923 |         switch content {
 924 |         case .voidNode, .text: return []
 925 |         case .childNodes(let nodes): return nodes
 926 |         }
 927 |     }
 928 | 
 929 |     var textContent: String? {
 930 |         switch content {
 931 |         case .childNodes, .voidNode: return nil
 932 |         case .text(let text): return text
 933 |         }
 934 |     }
 935 | 
 936 |     func firstDirectChild(named element: String) -> Node? {
 937 |         childNodes.first(where: { $0.element == element })
 938 |     }
 939 | 
 940 | 
 941 | These properties help us access child nodes and text content more easily.
 942 | 
 943 |     var childNodesSortedIntoBlocks: [Node] {
 944 |         var nodesToReturn = [Node]()
 945 |         var inlineElements = [Node]()
 946 | 
 947 |         func addInlineElementsAsGroupIfNeeded() {
 948 |             guard inlineElements.isEmpty == false else { return }
 949 |             // make a fake block element that has all these as children
 950 |             let wrapper = Node(element: "p", content: .childNodes(inlineElements), attributes: [])
 951 |             // and append it to our list to return
 952 |             nodesToReturn.append(wrapper)
 953 |             // then, empty the inlineElements list
 954 |             inlineElements = []
 955 |         }
 956 | 
 957 |         for node in childNodes {
 958 |             if isInlineNode {
 959 |                 inlineElements.append(node)
 960 |             } else {
 961 |                 addInlineElementsAsGroupIfNeeded()
 962 |                 nodesToReturn.append(node)
 963 |             }
 964 |         }
 965 |         addInlineElementsAsGroupIfNeeded()
 966 |         return nodesToReturn
 967 |     }
 968 | 
 969 |     var isInlineNode: Bool {
 970 |         [InternalElement.textRun, "a", "abbr", "acronym", "audio", "b", "bdi", "bdo", "big", "br", "button", "canvas", "cite", "code", "data", "datalist", "del", "dfn", "em", "embed", "i", "iframe", "img", "input", "ins", "kbd", "label", "map", "mark", "meter", "noscript", "object", "output", "picture", "progress", "q", "ruby", "s", "samp", "script", "select", "slot", "small", "span", "strong", "sub", "sup", "svg", "template", "textarea", "time", "u", "tt", "var", "video", "wbr"].contains(element)
 971 |     }
 972 | 
 973 | 
 974 | This next property is a little more involved. When we’re rendering nodes, we want block nodes, like <p>, <div>, etc. to flow one after another, vertically down the page, while things like <bold>, <a>, etc. flow within the same line like words in a paragraph.
 975 | 
 976 | The trouble for us is, in html those inline elements don’t have exist inside of block elements at all, they can exist outside of them too. For example:
 977 | 
 978 | <body>
 979 |     <bold>Some bold text</bold>
 980 |     <p>A paragraph</p>
 981 | </body>
 982 | 
 983 | 
 984 | The bold text is just kinda hanging out as inline, but inline relative to what? I’m not entirely sure how other browsers solve this, but we’ve solved it by grouping any inline elements as children of a fake, inserted <p> node.
 985 | 
 986 |     var attributeDictionary: [String: String] {
 987 |         Dictionary(uniqueKeysWithValues: attributes.map({ ($0.key, $0.value) }))
 988 |     }
 989 | }
 990 | 
 991 | 
 992 | Lastly, we offer a way to access the node’s attributes as a dictionary.
 993 | 
 994 | Now we have enough tools at our disposal to write the rest of the views.
 995 | 
 996 | The BodyView
 997 | 
 998 | This view hosts our browser’s scroll view, which then displays child nodes in another view.
 999 | 
1000 | struct BodyView: View {
1001 |     let bodyNode: Node
1002 |     var body: some View {
1003 |         ScrollView {
1004 |             BlocksView(children: bodyNode.childNodesSortedIntoBlocks)
1005 |             .padding(20)
1006 |         }
1007 |         .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading)
1008 |         .background(Color.white)
1009 |     }
1010 | }
1011 | 
1012 | 
1013 | The hierarchy here is mostly straightforward: the BlocksView is initialized with the child nodes of the body and is given a global padding. Then we extend the frame of the scroll view to stretch as much as possible and align the content to the top leading edge, like other browsers do.
1014 | 
1015 | BlocksView
1016 | 
1017 | This one is kind of fun: it’s a reusable view that vertically stacks the child nodes it was given, rendering them with the appropriate view depending on what element they are. It even recursively uses itself in a few cases.
1018 | 
1019 | struct BlocksView: View {
1020 |     let children: [Node]
1021 | 
1022 |     var body: some View {
1023 |         VStack(alignment: .leading, spacing: 20) {
1024 |             ForEach(children, id: \.self) { childNode in
1025 |                 switch childNode.element {
1026 |                 case "h1":
1027 |                     InlineContentWrappingBlockView(node: childNode)
1028 |                         .font(Font.custom("Times", size: 32).bold())
1029 |                 case "h2":
1030 |                     InlineContentWrappingBlockView(node: childNode)
1031 |                         .font(Font.custom("Times", size: 28).bold())
1032 |                 case "h3":
1033 |                     InlineContentWrappingBlockView(node: childNode)
1034 |                         .font(Font.custom("Times", size: 24).bold())
1035 |                 case "p":
1036 |                     InlineContentWrappingBlockView(node: childNode)
1037 |                 case "div", "section", "main", "footer", "article", "header", "nav", "aside":
1038 |                     BlocksView(children: childNode.childNodesSortedIntoBlocks)
1039 |                 case "pre":
1040 |                     BlocksView(children: childNode.childNodesSortedIntoBlocks)
1041 |                         .font(Font.system(size: 13, design: .monospaced))
1042 |                 case "blockquote":
1043 |                     BlocksView(children: childNode.childNodesSortedIntoBlocks)
1044 |                         .padding(.leading, 20)
1045 |                 case "ul": ListNodeView(node: childNode, style: .unordered)
1046 |                 case "ol": ListNodeView(node: childNode, style: .ordered)
1047 |                 case "hr": Divider()
1048 |                 case "script": EmptyView()
1049 |                 case "br": Color.clear.padding(20)
1050 |                 default: Text("unknown block element: <\(childNode.element)>")
1051 |                 }
1052 |             }
1053 |         }
1054 |     }
1055 | }
1056 | 
1057 | 
1058 | We don’t support special rendering for every element under the sun, so if we find an element we don’t know about, we just render that we found an unknown block. You could default it to behaving like a <div> if you wanted, but I like calling them out like this instead because I’m more motivated to give it a proper view that way.
1059 | 
1060 | Inline nodes
1061 | 
1062 | Inline nodes are interesting, because to render them we can’t just use views placed in some kind of stack. Instead, we want them to be rendered one after another like text, wrapping to the next line as needed. And indeed, that’s how we’re going to do it in SwiftUI, by combining (or in Swift terms, using reduce()) inline contents into an AttributedString and rendering it in a single Text view per inline “block.”
1063 | 
1064 | struct InlineContentWrappingBlockView: View {
1065 |     let node: Node
1066 |     @Environment(\.font) var font
1067 | 
1068 |     var body: some View {
1069 |         Text(
1070 |             node
1071 |                 .childNodes
1072 |                 .map { $0.attributedText(defaultFont: font ?? Font.custom("Times", size: 16)) }
1073 |                 .reduce(AttributedString(), +)
1074 |         )
1075 |         .lineSpacing(4)
1076 |         .fixedSize(horizontal: false, vertical: true)
1077 |     }
1078 | }
1079 | 
1080 | 
1081 | In the body of our body, we return a single Text, initialized with an attributed string. The attributed string is created by mapping the node’s child nodes and calling the attributedText(defaultFont:) method on each (we’ll see that property in a moment). This mapping gives us an array of attribute strings, so we reduce() them into a single attributed string.
1082 | 
1083 | extension Node {
1084 |     func attributedText(defaultFont: Font) -> AttributedString {
1085 |         switch element {
1086 |         case InternalElement.textRun:
1087 |             var attributes = AttributeContainer()
1088 |             attributes.font = defaultFont
1089 | 
1090 |             return AttributedString(textContent ?? "", attributes: attributes)
1091 | 
1092 | 
1093 | To get the attributed text for a node, we switch over its element to see how we should format it. Here we have the base case: a text run. We create an attribute container, use the font that was passed in, and return an attributed string with the node’s text content and those attributes.
1094 | 
1095 |         case "em", "i":
1096 |             var attributes = AttributeContainer()
1097 |             attributes.font = defaultFont.italic()
1098 | 
1099 |             return childNodes
1100 |                 .map { $0.attributedText(defaultFont: defaultFont.italic()) }
1101 |                 .reduce(AttributedString(), +)
1102 |                 .mergingAttributes(attributes, mergePolicy: .keepCurrent)
1103 | 
1104 | 
1105 | The rest of the cases are similar, in that we create some attributes, modifying the passed in font as needed. But in order to create the final attributed string, we actually need to recursively call ourselves so that we can handle multiple overlapping styles (eg a link node wrapped inside an italics node). 
1106 | 
1107 |         case "strong", "b":
1108 |             var attributes = AttributeContainer()
1109 |             attributes.font = defaultFont.bold()
1110 | 
1111 |             return childNodes
1112 |                 .map { $0.attributedText(defaultFont: defaultFont.bold()) }
1113 |                 .reduce(AttributedString(), +)
1114 |                 .mergingAttributes(attributes, mergePolicy: .keepCurrent)
1115 |         case "code":
1116 |             var attributes = AttributeContainer()
1117 |             let monospaced = Font.system(size: 13, design: .monospaced)
1118 |             attributes.font = monospaced
1119 | 
1120 |             return childNodes
1121 |                 .map { $0.attributedText(defaultFont: monospaced) }
1122 |                 .reduce(AttributedString(), +)
1123 |                 .mergingAttributes(attributes, mergePolicy: .keepCurrent)
1124 |         case "a":
1125 |             var attributes = AttributeContainer()
1126 |             attributes.link = URL(string: attributeDictionary["href"] ?? "")
1127 |             attributes.underlineStyle = .single
1128 | 
1129 |             return childNodes
1130 |                 .map { $0.attributedText(defaultFont: defaultFont) }
1131 |                 .reduce(AttributedString(), +)
1132 |                 .mergingAttributes(attributes, mergePolicy: .keepCurrent)
1133 |         default:
1134 |             var attributes = AttributeContainer()
1135 |             attributes.font = defaultFont
1136 | 
1137 |             return childNodes
1138 |                 .map { $0.attributedText(defaultFont: defaultFont) }
1139 |                 .reduce(AttributedString(), +)
1140 |                 .mergingAttributes(attributes, mergePolicy: .keepCurrent)
1141 |         }
1142 |     }
1143 | }
1144 | 
1145 | 
1146 | It’s all a little boilerplatey but it gets the job done.
1147 | 
1148 | ListNodeView
1149 | 
1150 | Our last node view is the ListNodeView, which we’ll use for displaying both ordered and unordered lists (<ol> and <ul>).
1151 | 
1152 | struct ListNodeView: View {
1153 |     enum Style {
1154 |         case ordered, unordered
1155 | 
1156 |         func listMarker(for index: Int) -> String {
1157 |             switch self {
1158 |             case .ordered: return "\(index + 1)."
1159 |             case .unordered: return "•"
1160 |             }
1161 |         }
1162 |     }
1163 | 
1164 | 
1165 | We start with an enum for the two list styles, and a function for picking the right list item marker for the given index.
1166 | 
1167 |     let node: Node
1168 |     let style: Style
1169 | 
1170 |     var body: some View {
1171 |         VStack(alignment: .leading, spacing: 8) {
1172 |             ForEach(Array(zip(node.childNodes.indices, node.childNodes)), id: \.1) { (index, childNode) in
1173 |                 HStack(alignment: .firstTextBaseline, spacing: 8) {
1174 |                     Text(verbatim: style.listMarker(for: index))
1175 |                     BlocksView(children: childNode.childNodesSortedIntoBlocks)
1176 |                 }
1177 |             }
1178 |         }
1179 |     }
1180 | }
1181 | 
1182 | 
1183 | The body of the list node should look pretty straightforward at this point: a vertical stack wherein we iterate all our child nodes and render them as list items (using a kind of noisy Array/zip dance to get the index). A list item uses a horizontal stack to display the marker, followed by a BlocksView for the grouped children (it could be, for example, multiple paragraphs).
1184 | 
1185 | Preserving Whitespace
1186 | 
1187 | You may notice at this point, if you load up a webpage with a <pre> tag, it uses a monospaced font but it does not preserve whitespace. That’s because we’re currently discarding tabs and newlines in our parser. But for <pre> tags we should be preserving it in child nodes (and all of their descendents). So let’s make some modifications to the parser.
1188 | 
1189 | In the parse method of Node, immediately before parsing child nodes, let’s add the following line:
1190 | 
1191 | let shouldPreserveWhiteSpace = startTag.element == "pre"
1192 | 
1193 | 
1194 | Then, in the 2nd choice closure (text runs), we want to replace whitespaces only when we’re not preserving whitespace. Change our entity decoding code to the following:
1195 | 
1196 | let entityDecodedContents = textContents
1197 |     .map(\.body)
1198 |     .joined()
1199 |        .replacingOccurrences(of: "&#x000A;", with: "")
1200 |        .replacingOccurrences(of: "&lt;", with: "<")
1201 |        .replacingOccurrences(of: "&gt;", with: ">")
1202 |        .replacingOccurrences(of: "&quot;", with: "\"")
1203 |        .replacingOccurrences(of: "&amp;", with: "&")
1204 | 
1205 | let contentRun = shouldPreserveWhitespace ? entityDecodedContents : entityDecodedContents
1206 |     .replacingOccurrences(of: "\n", with: " ")
1207 |     .replacingOccurrences(of: "\t", with: " ")
1208 | 
1209 | 
1210 | This preserves whitespace great when the text is a direct child of the node, but doesn’t yet handle deeper nestings. To do that, we’ll have to pass shouldPreserveWhitespace as a flag to child node parsing. To keep things simple, let’s add the flag to our Parsable protocol requirement:
1211 | 
1212 | static func parse(context: ParsingContext, shouldPreserveWhitespace: Bool) throws -> Self
1213 | 
1214 | 
1215 | After you make the change you’ll have to go through all the call sites where we implement the parse(...) method and update them to include the new flag. At pretty much every callsite, just give a value of false. However, let’s return to Node, specifically where we’re parsing a child node (the first choice). Change the parse call to:
1216 | 
1217 | try Node.parse(context: context, shouldPreserveWhitespace: shouldPreserveWhitespace)
1218 | 
1219 | 
1220 | so that we can pass it down the line. Finally, where we declare our local variable for preserving whitespace, we update that to factor in the parameter passed in:
1221 | 
1222 | let shouldPreserveWhitespace = startTag.element == "pre" || shouldPreserveWhitespace
1223 | 
1224 | 
1225 | Now if we run the browser, we should see that whitespace preservation works as expected.
1226 | 
1227 | Displaying images
1228 | 
1229 | Let’s add support for displaying images using the <img> tag. By default, image elements are considered to be “inline” (ie, display: inline in CSS terms), but inline images are a little complicated, so I’ll leave those as an exercise for the reader. However, supporting block images will be a little more straightforward, so let’s add support for that.
1230 | 
1231 | While supporting all of CSS is way, way out of scope for this tutorial, it would still be nice to leave us with a starting point, so I’d like to add support to the style attribute, which we’ll parse in a rather crude way. Once we have the style attribute parsed, we’ll add support for display: block as well. Let’s start with a type representing a style.
1232 | 
1233 | struct Style {
1234 | 
1235 |     enum DisplayStyle { case inline, block }
1236 | 
1237 |     var display: DisplayStyle? {
1238 |         switch rawValue["display"] {
1239 |         case "inline": return .inline
1240 |         case "block": return .block
1241 |         default:
1242 |             return nil
1243 |         }
1244 |     }
1245 | 
1246 |     private let rawValue: [String: String]
1247 | 
1248 |     init(rawPairs: [String: String]) {
1249 |         self.rawValue = rawPairs
1250 |     }
1251 | }
1252 | 
1253 | 
1254 | Style wraps an underlying dictionary of keys and values, and adds a helper property that looks for a display key, returning the value interpreted as either inline or block (or nil, if missing or something else). Now let’s create a style instance from a node’s style attribute, if it exists. In an extension on Node, put the following:
1255 | 
1256 | var styleFromAttributes: Style? {
1257 |     guard let styleAttribute = attributeDictionary["style"] else { return nil }
1258 |     let stylePairs = styleAttribute.components(separatedBy: ";")
1259 | 
1260 | 
1261 | First, we check to see if we even have a style attribute, otherwise we bail. Then, we break up the value string into substrings, which are separated by a semicolon.
1262 | 
1263 |     return Style(
1264 |         rawPairs: .init(
1265 |             uniqueKeysWithValues: stylePairs
1266 |                 .map { $0.components(separatedBy: ":") }
1267 |                 .map { ($0.first?.trimmingCharacters(in: .whitespacesAndNewlines), $0.last?.trimmingCharacters(in: .whitespacesAndNewlines)) }
1268 |                 .compactMap {
1269 |                     guard let key = $0, let value = $1 else { return nil }
1270 |                     return (key, value)
1271 |                 }
1272 |         )
1273 |     )
1274 | }
1275 | 
1276 | 
1277 | Finally, we initialize the Style with a dictionary, whose keys and values are found by splitting up those substrings from earlier on colons, trimming out whitespace, and finally returning them as a non-nil tuple. This code is kinda fragile would definitely be made more powerful (and extensible!) if we wrote a parser like we did for html, but I’ll leave that as an exercise for the reader :)
1278 | 
1279 | Now that we can parse a Style from a node’s attributes, the last thing we need to do is modify how we’re sorting / grouping nodes. In the Node.childNodesSortedIntoBlocks property, edit the for loop to look like this:
1280 | 
1281 | for node in childNodes {
1282 |     let defaultDisplayStyle: Style.DisplayStyle = node.isInlineNode ? .inline : .block 
1283 |     let display = node.styleFromAttributes?.display ?? defaultDisplayStyle
1284 |     if display == .inline {
1285 |         inlineElements.append(node)
1286 |     } else {
1287 |         addInlineElementsAsGroupIfNeeded()
1288 |         nodesToReturn.append(node)
1289 |     }
1290 | }
1291 | 
1292 | 
1293 | With that modification, <img> nodes that have a style attribute declaring they should be display: block will now be properly considered block views in our renderer, and appear accordingly. All that’s left is to create a view that can load and display images. SwiftUI already has the a perfect view for us: AsyncImage, which we’ll wrap in our own ImageView to customize it a little.
1294 | 
1295 | struct ImageView: View {
1296 |     let node: Node
1297 |     @Environment(\.urlBuilder) var urlBuilder
1298 | 
1299 |     var body: some View {
1300 |         AsyncImage(url: urlBuilder(URL(string: node.attributeDictionary["src"] ?? "")!), content: { image in
1301 |             image
1302 |                 .resizable()
1303 |                 .aspectRatio(contentMode: .fit)
1304 |                 .frame(
1305 |                     width: node.attributeDictionary["width"].flatMap(WebSize.init(rawValue:))?.dimension,
1306 |                     height: node.attributeDictionary["height"].flatMap(WebSize.init(rawValue:))?.dimension
1307 |                 )
1308 |         }, placeholder: {
1309 |             Color(white: 0.9).cornerRadius(4)
1310 |         })
1311 |     }
1312 | }
1313 | 
1314 | 
1315 | From the environment, we pull out the urlBuilder function we declared earlier in our view hierarchy, so that we can make sure the image’s src url is an absolute url, that we’ll then hand off to SwiftUI to load asynchronously for us. When the image is ready, we resize it and constrain it as necessary, depending on any width or height attributes of the <img> node.
1316 | 
1317 | struct WebSize {
1318 |     let rawValue: String
1319 | 
1320 |     var dimension: CGFloat {
1321 |         // trim anything that isn't a digit, then try to parse that into an int. this ignores things like "px"
1322 |         CGFloat(Int(rawValue.prefix(while: \.isWholeNumber)) ?? 0)
1323 |     }
1324 | }
1325 | 
1326 | 
1327 | WebSize is a small little type for extracting number values out of sizing values in html. We’re assuming everything is measured in px for simplicity’s sake. Sizing in html is a complicated topic, but you could go deep here if you wanted.
1328 | 
1329 | Finally, let’s add a case for img nodes in our BlocksView body switch statement that uses the new image view we created:
1330 | 
1331 | case "img":
1332 |     ImageView(node: childNode)
1333 | 
1334 | 
1335 | With that, we should finally be able to see image tags with their display set to block.
1336 | 
1337 | One last thing
1338 | 
1339 | You may have noticed that most browsers, when given an unstyled html page, will render the body using the entire width of the browser window, and our browser does this exact same thing. However, on modern monitors, this can result in extremely long lines of text that are kind of hard to read due to their length, so the nice thing to do is to style the container with a maximum width.
1340 | 
1341 | Now that we have support for the style attribute, let’s parse out the max-width style, if it’s present. In the Style struct, add the following property:
1342 | 
1343 | var maxWidth: CGFloat? {
1344 |     rawValue["max-width"].map(WebSize.init(rawValue:)).map(\.dimension)
1345 | }
1346 | 
1347 | 
1348 | Let’s use this property on our BodyView. Change the body view’s body scroll view to this:
1349 | 
1350 | ScrollView {
1351 |     HStack(spacing: 0) {
1352 |         BlocksView(children: bodyNode.childNodesSortedIntoBlocks)
1353 |             .frame(maxWidth: bodyNode.styleFromAttributes?.maxWidth)
1354 |         Spacer()
1355 |     }
1356 |     .padding(20)
1357 | }
1358 | 
1359 | 
1360 | This makes the BlocksView respect the max-width from the <body> tag, if it exists.
1361 | 
1362 | The End
1363 | 
1364 | This concludes our browser engine. We wrote a simple html parser from scratch with Swift, and then wrote a rendering engine using SwiftUI. In all, the browser should be capable of rendering this very tutorial, and it should look pretty much identical to how it looks in Safari, Chrome, or other Big Browsers.
1365 | 
1366 | You could extend this foundation in so many ways:
1367 | 
1368 | 
1369 | You could add more block or inline elements types
1370 | You could expand what style elements are supported
1371 | You could even write your own CSS parser! or if you have limitless ambition, you could write a javascript engine too
1372 | 
1373 | 
1374 | But most of all, I hope you enjoyed yourself and learned a thing or two.
1375 | 
1376 | Thanks for reading!
1377 | 
1378 | (ps: I’m looking for work, so if you’re looking to hire someone to work on browsers, programming languages, dev tools, or Swift apps, I’m your guy! Please reach out, I’d love to hear from you)
1379 | 
1380 | todo:
1381 | 
1382 | 
1383 | screenshots
1384 | 
1385 | 	
1386 | 


--------------------------------------------------------------------------------
/SmolTests/SmolTests.swift:
--------------------------------------------------------------------------------
  1 | //
  2 | //  SmolTests.swift
  3 | //  SmolTests
  4 | //
  5 | //  Created by Jason Brennan on 2/9/23.
  6 | //
  7 | 
  8 | import XCTest
  9 | @testable import Smol
 10 | 
 11 | final class SmolTests: XCTestCase {
 12 | 	
 13 | 	// MARK: - Tokenizer
 14 | 	
 15 | 	func testTokenizer() throws {
 16 | 		let program1 = " "
 17 | 		let tokenizer = Tokenizer(programText: program1)
 18 | 		
 19 | 		let tokens = try tokenizer.scanAllTokens()
 20 | 		
 21 | 		XCTAssertEqual(tokens, [
 22 | 			Token(kind: .openAngleBracket, body: "<"),
 23 | 			Token(kind: .text, body: "html"),
 24 | 			Token(kind: .closeAngleBracket, body: ">"),
 25 | 			Token(kind: .whitespace, body: " "),
 26 | 			Token(kind: .openAngleBracket, body: "<"),
 27 | 			Token(kind: .forwardSlash, body: "/"),
 28 | 			Token(kind: .text, body: "html"),
 29 | 			Token(kind: .closeAngleBracket, body: ">"),
 30 | 		])
 31 | 	}
 32 | 	
 33 | 	func testTokenizerSplitsOnWhitespace() throws {
 34 | 		let program1 = "img src"
 35 | 		let tokenizer = Tokenizer(programText: program1)
 36 | 		
 37 | 		let tokens = try tokenizer.scanAllTokens()
 38 | 		
 39 | 		XCTAssertEqual(tokens, [
 40 | 			Token(kind: .text, body: "img"),
 41 | 			Token(kind: .whitespace, body: " "),
 42 | 			Token(kind: .text, body: "src"),
 43 | 		])
 44 | 	}
 45 | 	
 46 | 	
 47 | 	// MARK: - Node parsing
 48 | 
 49 | 	func testEmptyHTMLTag() throws {
 50 | 		let program1 = ""
 51 | 		let tokenizer = Tokenizer(programText: program1)
 52 | 		
 53 | 		let tokens = try tokenizer.scanAllTokens()
 54 | 		
 55 | 		let context = ParsingContext(tokens: tokens)
 56 | 		let node = try Node.parse(context: context)
 57 | 		
 58 | 		XCTAssertEqual(node, Node(element: "html", content: .childNodes([]), attributes: [:]))
 59 |     }
 60 | 	
 61 | 	func testIncorrectlyMatchingTagFails() throws {
 62 | 		let program1 = ""
 63 | 		let tokenizer = Tokenizer(programText: program1)
 64 | 		
 65 | 		let tokens = try tokenizer.scanAllTokens()
 66 | 		
 67 | 		let context = ParsingContext(tokens: tokens)
 68 | 		XCTAssertThrowsError(try Node.parse(context: context))
 69 | 	}
 70 | 	
 71 | 	func testHTMLTagWithBody() throws {
 72 | 		let program1 = ""
 73 | 		let tokenizer = Tokenizer(programText: program1)
 74 | 		
 75 | 		let tokens = try tokenizer.scanAllTokens()
 76 | 		
 77 | 		let context = ParsingContext(tokens: tokens)
 78 | 		let node = try Node.parse(context: context)
 79 | 		
 80 | 		XCTAssertEqual(node, Node(element: "html", content: .childNodes([Node(element: "body", content: .childNodes([]), attributes: [:])]), attributes: [:]))
 81 | 	}
 82 | 	
 83 | 	func testDoubleNestedTag() throws {
 84 | 		let program1 = "\n\n\n\n"
 85 | 		let tokenizer = Tokenizer(programText: program1)
 86 | 		
 87 | 		let tokens = try tokenizer.scanAllTokens()
 88 | 		
 89 | 		let context = ParsingContext(tokens: tokens)
 90 | 		let node = try Node.parse(context: context)
 91 | 		
 92 | 		XCTAssertEqual(node, Node(element: "html", content: .childNodes([Node(element: "body", content: .childNodes([Node(element: "h1", content: .childNodes([]), attributes: [:])]), attributes: [:])]), attributes: [:]))
 93 | 	}
 94 | 	
 95 | 	func testTagWithJustText() throws {
 96 | 		let program1 = "hello there"
 97 | 		let tokenizer = Tokenizer(programText: program1)
 98 | 		
 99 | 		let tokens = try tokenizer.scanAllTokens()
100 | 		
101 | 		let context = ParsingContext(tokens: tokens)
102 | 		let node = try Node.parse(context: context)
103 | 		
104 | 		XCTAssertEqual(node, Node(element: "p", content: .childNodes([Node(element: Node.textRunElement, content: .text("hello there"), attributes: [:])]), attributes: [:]))
105 | 	}
106 | 	
107 | 	func testTagWithTextAndChildTags() throws {
108 | 		let program1 = "hello there"
109 | 		let tokenizer = Tokenizer(programText: program1)
110 | 		
111 | 		let tokens = try tokenizer.scanAllTokens()
112 | 		
113 | 		let context = ParsingContext(tokens: tokens)
114 | 		let node = try Node.parse(context: context)
115 | 		
116 | 		XCTAssertEqual(node, Node(element: "p", content: .childNodes([Node(element: Node.textRunElement, content: .text("hello "), attributes: [:]), Node(element: "em", content: .childNodes([Node(element: Node.textRunElement, content: .text("there"), attributes: [:])]), attributes: [:])]), attributes: [:]))
117 | 	}
118 | 	
119 | 	func testVoidElement() throws {
120 | 		let program1 = ""
121 | 		let tokenizer = Tokenizer(programText: program1)
122 | 		
123 | 		let tokens = try tokenizer.scanAllTokens()
124 | 		
125 | 		let context = ParsingContext(tokens: tokens)
126 | 		let node = try Node.parse(context: context)
127 | 		
128 | 		XCTAssertEqual(node, Node(element: "img", content: .voidNode, attributes: [:]))
129 | 	}
130 | 	
131 | 	func testVoidElementWithAttributes() throws {
132 | 		let program1 = ""
133 | 		let tokenizer = Tokenizer(programText: program1)
134 | 		
135 | 		let tokens = try tokenizer.scanAllTokens()
136 | 		
137 | 		let context = ParsingContext(tokens: tokens)
138 | 		let node = try Node.parse(context: context)
139 | 		
140 | 		XCTAssertEqual(node, Node(element: "img", content: .voidNode, attributes: ["src": "http://example.com/image.png", "width": "600px"]))
141 | 	}
142 | 	
143 | 	func testVoidElementWithSingleQuotedAttributes() throws {
144 | 		let program1 = ""
145 | 		let tokenizer = Tokenizer(programText: program1)
146 | 		
147 | 		let tokens = try tokenizer.scanAllTokens()
148 | 		
149 | 		let context = ParsingContext(tokens: tokens)
150 | 		let node = try Node.parse(context: context)
151 | 		
152 | 		XCTAssertEqual(node, Node(element: "img", content: .voidNode, attributes: ["src": "http://example.com/image.png", "width": "600px"]))
153 | 	}
154 | 	
155 | 	func testVoidElementWithNonQuotedAttributes() throws {
156 | 		let program1 = ""
157 | 		let tokenizer = Tokenizer(programText: program1)
158 | 		
159 | 		let tokens = try tokenizer.scanAllTokens()
160 | 		
161 | 		let context = ParsingContext(tokens: tokens)
162 | 		let node = try Node.parse(context: context)
163 | 		
164 | 		XCTAssertEqual(node, Node(element: "img", content: .voidNode, attributes: ["src": "home", "width": "600"]))
165 | 	}
166 | 	
167 | 	func testVoidElementWithKeyOnlyAttributes() throws {
168 | 		let program1 = ""
169 | 		let tokenizer = Tokenizer(programText: program1)
170 | 		
171 | 		let tokens = try tokenizer.scanAllTokens()
172 | 		
173 | 		let context = ParsingContext(tokens: tokens)
174 | 		let node = try Node.parse(context: context)
175 | 		
176 | 		XCTAssertEqual(node, Node(element: "img", content: .voidNode, attributes: ["src": "src", "autoplay": "autoplay", "me": "you", "blep": "blep"]))
177 | 	}
178 | 	
179 | //	func testVoidElementWithHyphenatedAttributeKey() throws {
180 | //		let program1 = ""
181 | //		let tokenizer = Tokenizer(programText: program1)
182 | //
183 | //		let tokens = try tokenizer.scanAllTokens()
184 | //
185 | //		let context = ParsingContext(tokens: tokens)
186 | //		let node = try Node.parse(context: context)
187 | //
188 | //		XCTAssertEqual(node, Node(element: "img", content: .voidNode, attributes: ["data-name": "hello"]))
189 | //	}
190 | 	
191 | 	func testVoidElementWithTrailingSlash() throws {
192 | 		let program1 = ""
193 | 		let tokenizer = Tokenizer(programText: program1)
194 | 		
195 | 		let tokens = try tokenizer.scanAllTokens()
196 | 		
197 | 		let context = ParsingContext(tokens: tokens)
198 | 		let node = try Node.parse(context: context)
199 | 		
200 | 		XCTAssertEqual(node, Node(element: "img", content: .voidNode, attributes: [:]))
201 | 	}
202 | 	
203 | 	func testVoidElementWithSpaceAndTrailingSlash() throws {
204 | 		let program1 = ""
205 | 		let tokenizer = Tokenizer(programText: program1)
206 | 		
207 | 		let tokens = try tokenizer.scanAllTokens()
208 | 		
209 | 		let context = ParsingContext(tokens: tokens)
210 | 		let node = try Node.parse(context: context)
211 | 		
212 | 		XCTAssertEqual(node, Node(element: "img", content: .voidNode, attributes: [:]))
213 | 	}
214 | 	
215 | 	func testNestedVoidElement() throws {
216 | 		let program1 = ""
217 | 		let tokenizer = Tokenizer(programText: program1)
218 | 		
219 | 		let tokens = try tokenizer.scanAllTokens()
220 | 		
221 | 		let context = ParsingContext(tokens: tokens)
222 | 		let node = try Node.parse(context: context)
223 | 		
224 | 		XCTAssertEqual(node, Node(element: "html", content: .childNodes([Node(element: "body", content: .childNodes([Node(element: "img", content: .voidNode, attributes: [:])]), attributes: [:])]), attributes: [:]))
225 | 	}
226 | 	
227 | 	func testSkipsCommentTags() throws {
228 | 		let program1 = "hi"
229 | 		let tokenizer = Tokenizer(programText: program1)
230 | 		
231 | 		let tokens = try tokenizer.scanAllTokens()
232 | 		
233 | 		let context = ParsingContext(tokens: tokens)
234 | 		let node = try Node.parse(context: context)
235 | 		
236 | 		XCTAssertEqual(node, Node(element: "html", content: .childNodes([Node(element: "head", content: .childNodes([Node(element: Node.textRunElement, content: .text("hi"), attributes: [:])]), attributes: [:])]), attributes: [:]))
237 | 	}
238 | 
239 | }
240 | 


--------------------------------------------------------------------------------
/readme.md:
--------------------------------------------------------------------------------
1 | # SmolHTML
2 | 
3 | A smol web browser engine written from scratch in Swift. You can read more about it here: https://nearthespeedoflight.com/browser.html
4 | 


--------------------------------------------------------------------------------

Welcome to Jason's homepage

Let’s Write a Web Browser from Scratch in Swift!

The Architecture

Quick Tips

Starting the project

Tokenizing

The Parsing Context

Parsing HTML

Document

Node

Tag

Attribute

End of Part 1

Part 2: Rendering in SwiftUI

The Architecture

The Page Controller

The Browser View

Web Document View

Node extensions

The BodyView

BlocksView

Inline nodes

ListNodeView

Preserving Whitespace

Displaying images

One last thing

The End