├── .gitignore ├── Introduction to compilers.playground ├── Pages │ ├── Finish.xcplaygroundpage │ │ └── Contents.swift │ ├── IRGeneration.xcplaygroundpage │ │ ├── Contents.swift │ │ ├── Sources │ │ │ └── DebuggerView.swift │ │ └── timeline.xctimeline │ ├── Introduction.xcplaygroundpage │ │ ├── Contents.swift │ │ └── timeline.xctimeline │ ├── Lexer.xcplaygroundpage │ │ ├── Contents.swift │ │ └── timeline.xctimeline │ ├── Optimisation.xcplaygroundpage │ │ ├── Contents.swift │ │ ├── Sources │ │ │ └── OptimisationExplorer.swift │ │ └── timeline.xctimeline │ ├── Parser.xcplaygroundpage │ │ ├── Contents.swift │ │ ├── Sources │ │ │ ├── ASTExplorer.swift │ │ │ └── ASTView.swift │ │ └── timeline.xctimeline │ └── ParserSolution.xcplaygroundpage │ │ └── Contents.swift ├── Resources │ ├── Fibonacci.swift │ ├── Program with else.swift │ ├── Program with syntax error.swift │ └── Simple program.swift ├── Sources │ ├── Compiler │ │ ├── AST.swift │ │ ├── ASTPrinter.swift │ │ ├── ASTWalker.swift │ │ ├── Common.swift │ │ ├── Compiler.swift │ │ ├── IR.swift │ │ ├── IRDebugger.swift │ │ ├── IRExecutor.swift │ │ ├── IRFunctionsGen.swift │ │ ├── IRGen.swift │ │ ├── Lexer.swift │ │ ├── Optimiser.swift │ │ ├── Parser.swift │ │ ├── Scanner.swift │ │ ├── SwiftFile.swift │ │ ├── SyntaxHighlighter.swift │ │ └── Typechecker.swift │ └── GUI │ │ ├── TokenHoverView.swift │ │ ├── TokensExplorer.swift │ │ └── ViewExtensions.swift ├── contents.xcplayground └── playground.xcworkspace │ ├── contents.xcworkspacedata │ └── xcshareddata │ └── IDEWorkspaceChecks.plist ├── LICENSE ├── README.md ├── Screenshot1.png ├── Screenshot2.png ├── Screenshot3.png └── Screenshot4.png /.gitignore: -------------------------------------------------------------------------------- 1 | xcuserdata -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Finish.xcplaygroundpage/Contents.swift: -------------------------------------------------------------------------------- 1 | /*: 2 | # What have we learned? 3 | 4 | Modern compilers go through multiple independent phases to reduce the complexity of compilation: 5 | 6 | - The **lexer** transforms the stream of characters in the source code into categorised tokens 7 | - The **parser** organises the tokens in an abstract syntax tree (AST) to understand the source code's semantics 8 | - The **type checker** will then verify that there are no type system violations in the AST 9 | - **Code generation** flattens the AST into a compiler-internal intermediate representation (IR) that consists of multiple basic blocks and uses `branch` and `jump` to transfer control flow 10 | - The **optimiser** optimises the IR to generate more efficient code and remove artifacts that resulted from previous compilation steps 11 | - Lastly, the IR is translated into **machine code** that depends on the architecture for which the program is being compiled 12 | 13 | * callout(Discover): 14 | The Swift compiler is open source. Have a look at how the `if` statement gets parsed [there](https://github.com/apple/swift/blob/f23ec8855d8f633d3bfb1b7c79ed0a0bf42dd57d/lib/Parse/ParseStmt.cpp#L1345). 15 | 16 | [❮ Back to Optimisation](Optimisation) 17 | 18 | --- 19 | */ 20 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/IRGeneration.xcplaygroundpage/Contents.swift: -------------------------------------------------------------------------------- 1 | /*: 2 | # IR Generation (Intermediate Representation) 3 | 4 | While the tree structure generated by the parser is very good at highlighting the semantics of Swift code, later phases of the compiler are simpler if they can operate on an unnested, linear stream of instructions. 5 | 6 | The compiler will thus transform the AST generated by the parser into an *Intermediate Representation (IR)*. Each funtion in the IR consists of multiple *basic blocks*. These basic blocks, in turn, consist of basic instructions like `add` or `compare` and terminate with `branch`, `jump`, or `return` that transfer control flow to a different block (conditionally or unconditionally respectively) or return from the current function. 7 | 8 | For example, the instruction `add %1, 2 -> %2` means that we want to add `2` to the value of variable `%1` and store the result in variable `%2`. In the context of IR and assembly, these variables are often called *registers*. 9 | 10 | * note: 11 | If you are familiar with Assembly code, the IR presented here may look very familiar. Compilers that are more advanced than the one we study right now have features in their IR that cannot be represented in assembly like assigning a variable based on which block was executed previously (the so-called ϕ-function). These simplify later analysis. 12 | 13 | */ 14 | let sourceFile: SwiftFile = #fileLiteral(resourceName: "Fibonacci.swift") 15 | /*: 16 | * callout(Discover): 17 | Use the IR debugger on the right to explore how the IR generated for the program above gets executed by stepping through the instructions. \ 18 | Change the source file to see the IR for different programs. 19 | 20 | [❮ Back to the parser](Parser) 21 | 22 | [❯ Continue with optimisation](Optimisation) 23 | 24 | --- 25 | */ 26 | // Setup for the live view 27 | import PlaygroundSupport 28 | PlaygroundPage.current.liveView = DebuggerView(forSourceFile: sourceFile) 29 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/IRGeneration.xcplaygroundpage/Sources/DebuggerView.swift: -------------------------------------------------------------------------------- 1 | import Cocoa 2 | 3 | fileprivate class TableViewCell: NSTableCellView { 4 | override var backgroundStyle: NSView.BackgroundStyle { 5 | didSet { 6 | let str = NSMutableAttributedString(attributedString: self.textField!.attributedStringValue) 7 | switch backgroundStyle { 8 | case .light: 9 | str.addAttribute(.foregroundColor, 10 | value: NSColor.black, 11 | range: NSRange(location: 0, length: str.length)) 12 | case .dark: 13 | str.addAttribute(.foregroundColor, 14 | value: NSColor.white, 15 | range: NSRange(location: 0, length: str.length)) 16 | default: 17 | break 18 | } 19 | self.textField?.attributedStringValue = str 20 | } 21 | } 22 | } 23 | 24 | fileprivate class RegisterValuesDataSource: NSObject, NSTableViewDataSource, NSTableViewDelegate { 25 | var registerValues: [Register: IRValue] = [:] 26 | var sortedRegisterValues: [(Register, IRValue)] { 27 | let sortedKeys = registerValues.keys.sorted(by: { $0.name < $1.name }) 28 | return sortedKeys.map({ ($0, registerValues[$0]!) }) 29 | } 30 | 31 | func numberOfRows(in tableView: NSTableView) -> Int { 32 | return registerValues.count 33 | } 34 | 35 | func tableView(_ tableView: NSTableView, viewFor tableColumn: NSTableColumn?, row: Int) -> NSView? { 36 | let (register, value) = sortedRegisterValues[row] 37 | 38 | let string = NSMutableAttributedString(string: "\(register) \(value)") 39 | string.addAttribute(.font, 40 | value: NSFont.boldSystemFont(ofSize: NSFont.systemFontSize), 41 | range: NSRange(location: 0, length: ("\(register)" as NSString).length)) 42 | string.addAttribute(.foregroundColor, 43 | value: NSColor.darkGray, 44 | range: NSRange(location: ("\(register)" as NSString).length, 45 | length: (" \(value)" as NSString).length)) 46 | 47 | let textView = NSTextField(labelWithAttributedString: string) 48 | let cell = TableViewCell() 49 | cell.textField = textView 50 | cell.addSubview(textView) 51 | return cell 52 | } 53 | } 54 | 55 | fileprivate class StackFramesDataSource: NSObject, NSTableViewDataSource, NSTableViewDelegate { 56 | var stackFrames: [StackFrame] = [] 57 | var selectionDidChangeCallback: ((Int) -> Void)? 58 | 59 | func numberOfRows(in tableView: NSTableView) -> Int { 60 | return stackFrames.count 61 | } 62 | 63 | func tableView(_ tableView: NSTableView, viewFor tableColumn: NSTableColumn?, row: Int) -> NSView? { 64 | let label = "#\(row): \(stackFrames[row].functionName)" 65 | let textView = NSTextField(labelWithString: label) 66 | let cell = TableViewCell() 67 | cell.textField = textView 68 | cell.addSubview(textView) 69 | return cell 70 | } 71 | 72 | fileprivate func tableViewSelectionDidChange(_ notification: Notification) { 73 | let tv = notification.object! as! NSTableView 74 | selectionDidChangeCallback?(tv.selectedRow) 75 | } 76 | 77 | } 78 | 79 | @available(OSX 10.12.2, *) 80 | extension NSTouchBarItem.Identifier { 81 | static let stepItem = NSTouchBarItem.Identifier("Step") 82 | static let continueItem = NSTouchBarItem.Identifier("Continue") 83 | static let resetItem = NSTouchBarItem.Identifier("Reset") 84 | } 85 | 86 | func +=(lhs: NSMutableAttributedString, rhs: NSAttributedString) { 87 | lhs.append(rhs) 88 | } 89 | 90 | /// A UI for a debugger that is able to step through IR instructions 91 | public class DebuggerView: NSSplitView, NSTouchBarDelegate { 92 | 93 | private var ir: IR? 94 | private var debugger: IRDebugger? 95 | private let irView = NSTextField() 96 | private let registerValuesView = NSTableView() 97 | private let registerValuesViewDataSource = RegisterValuesDataSource() 98 | private let stackFramesView = NSTableView() 99 | private let stackFramesDataSource = StackFramesDataSource() 100 | private let resultsView = NSTextField() 101 | private let footerView = NSSplitView() 102 | private var callStack: [StackFrame] = [] 103 | 104 | /// - Parameter sourceFile: The source file whose IR to debug 105 | public init(forSourceFile sourceFile: SwiftFile) { 106 | super.init(frame: CGRect(x: 0, y: 0, width: 500, height: 600)) 107 | 108 | self.wantsLayer = true 109 | self.layer!.backgroundColor = NSColor(white: 247/255, alpha: 1).cgColor 110 | 111 | self.dividerStyle = .thin 112 | 113 | let mainRegion = NSStackView() 114 | mainRegion.translatesAutoresizingMaskIntoConstraints = false 115 | mainRegion.orientation = .vertical 116 | 117 | // Create the header 118 | let header = NSTextField(labelWithString: "IR Debugger") 119 | header.font = .systemFont(ofSize: 33, weight: .semibold) 120 | mainRegion.addFullWidthView(header) 121 | 122 | // Compile the program and set up the debugger 123 | do { 124 | let ast = try Parser.parse(sourceFile: sourceFile) 125 | try Typechecker.typecheck(node: ast) 126 | 127 | self.ir = IRGen.generateIR(forAST: ast) 128 | if let ir = ir { 129 | self.debugger = IRDebugger(ir: ir) 130 | } 131 | } catch { 132 | // Set up source view 133 | let sourceView = NSTextField() 134 | sourceView.backgroundColor = NSColor.white 135 | sourceView.drawsBackground = true 136 | sourceView.isBordered = false 137 | sourceView.attributedStringValue = sourceFile.highlightedString 138 | sourceView.translatesAutoresizingMaskIntoConstraints = false 139 | mainRegion.addFullWidthView(sourceView) 140 | 141 | let error = error as! CompilationError 142 | let errorView = NSTextField() 143 | errorView.translatesAutoresizingMaskIntoConstraints = false 144 | errorView.stringValue = "Compilation error:\n\(error)" 145 | errorView.isBordered = false 146 | errorView.isEditable = false 147 | 148 | mainRegion.addFullWidthView(errorView) 149 | 150 | addArrangedSubview(mainRegion) 151 | return 152 | } 153 | 154 | // Create the IR view 155 | irView.backgroundColor = NSColor.white 156 | irView.drawsBackground = true 157 | irView.isBordered = false 158 | irView.isEditable = false 159 | irView.translatesAutoresizingMaskIntoConstraints = false 160 | 161 | let irScrollView = NSScrollView() 162 | irScrollView.documentView = irView 163 | irScrollView.hasVerticalScroller = true 164 | irScrollView.translatesAutoresizingMaskIntoConstraints = false 165 | irScrollView.addConstraint(NSLayoutConstraint(item: irView, attribute: .width, relatedBy: .equal, toItem: irScrollView, attribute: .width, multiplier: 1, constant: 0)) 166 | 167 | self.reset() 168 | 169 | mainRegion.addFullWidthView(irScrollView) 170 | 171 | addArrangedSubview(mainRegion) 172 | 173 | let footerStackView = NSStackView() 174 | footerStackView.translatesAutoresizingMaskIntoConstraints = false 175 | footerStackView.orientation = .vertical 176 | 177 | func createDebuggerButton(withTitle title: String, action: Selector) -> NSButton { 178 | let button = NSButton(title: title, target: self, action: action) 179 | button.font = .systemFont(ofSize: 20) 180 | button.isBordered = false 181 | button.translatesAutoresizingMaskIntoConstraints = false 182 | 183 | return button 184 | } 185 | 186 | // Create debugger buttons 187 | let stepButton = createDebuggerButton(withTitle: "⤼", action: #selector(self.step)) 188 | stepButton.toolTip = "Step" 189 | let runUntilEndButton = createDebuggerButton(withTitle: "↠", action: #selector(self.runUntilEnd)) 190 | runUntilEndButton.toolTip = "Run until end" 191 | let resetButton = createDebuggerButton(withTitle: "⟲", action: #selector(self.reset)) 192 | resetButton.toolTip = "Reset" 193 | 194 | let debuggerButtonsView = NSStackView() 195 | debuggerButtonsView.addView(stepButton, in: .leading) 196 | debuggerButtonsView.addView(runUntilEndButton, in: .leading) 197 | debuggerButtonsView.addView(resetButton, in: .leading) 198 | footerStackView.addFullWidthView(debuggerButtonsView) 199 | 200 | 201 | // Create the stack frames view 202 | stackFramesDataSource.selectionDidChangeCallback = { [weak self] (selectedStackFrame: Int) in 203 | self?.didSelectStackFrame(stackFrame: selectedStackFrame) 204 | } 205 | 206 | stackFramesView.dataSource = self.stackFramesDataSource 207 | stackFramesView.delegate = self.stackFramesDataSource 208 | let stackFrameColumn = NSTableColumn(identifier: .stackFrame) 209 | stackFrameColumn.title = "Call stack" 210 | stackFrameColumn.tableView = stackFramesView 211 | stackFramesView.addTableColumn(stackFrameColumn) 212 | 213 | let stackFramesScrollView = NSScrollView() 214 | stackFramesScrollView.documentView = stackFramesView 215 | stackFramesScrollView.translatesAutoresizingMaskIntoConstraints = false 216 | stackFramesScrollView.hasVerticalScroller = true 217 | 218 | // Create the registers view 219 | registerValuesView.dataSource = self.registerValuesViewDataSource 220 | registerValuesView.delegate = self.registerValuesViewDataSource 221 | let column = NSTableColumn(identifier: .registerValue) 222 | column.title = "Register values" 223 | column.tableView = registerValuesView 224 | registerValuesView.addTableColumn(column) 225 | 226 | let registerScrollView = NSScrollView() 227 | registerScrollView.documentView = registerValuesView 228 | registerScrollView.translatesAutoresizingMaskIntoConstraints = false 229 | registerScrollView.hasVerticalScroller = true 230 | 231 | // Create the results view 232 | resultsView.backgroundColor = NSColor.white 233 | resultsView.drawsBackground = true 234 | resultsView.isBordered = false 235 | resultsView.isEditable = false 236 | resultsView.translatesAutoresizingMaskIntoConstraints = false 237 | 238 | let resultsScrollView = NSScrollView() 239 | resultsScrollView.documentView = resultsView 240 | resultsScrollView.translatesAutoresizingMaskIntoConstraints = false 241 | resultsScrollView.hasVerticalScroller = true 242 | 243 | resultsScrollView.addConstraint(NSLayoutConstraint(item: resultsView, attribute: .width, relatedBy: .equal, toItem: resultsScrollView, attribute: .width, multiplier: 1, constant: 0)) 244 | 245 | // Aggregate results and register view to footer 246 | 247 | footerView.translatesAutoresizingMaskIntoConstraints = false 248 | footerView.addArrangedSubview(stackFramesScrollView) 249 | footerView.addArrangedSubview(registerScrollView) 250 | footerView.addArrangedSubview(resultsScrollView) 251 | footerView.isVertical = true 252 | footerView.dividerStyle = .thin 253 | 254 | footerStackView.addFullWidthView(footerView) 255 | 256 | addArrangedSubview(footerStackView) 257 | } 258 | 259 | required public init?(coder: NSCoder) { 260 | fatalError("init(coder:) has not been implemented") 261 | } 262 | 263 | public override func viewDidMoveToWindow() { 264 | self.window?.makeFirstResponder(self) 265 | 266 | self.setPosition(self.frame.size.height - 150, ofDividerAt: 0) 267 | footerView.setPosition(footerView.frame.size.width / 3, ofDividerAt: 0) 268 | footerView.setPosition(2 * footerView.frame.size.width / 3, ofDividerAt: 1) 269 | } 270 | 271 | @available(OSX 10.12.2, *) 272 | override public func makeTouchBar() -> NSTouchBar? { 273 | let touchBar = NSTouchBar() 274 | touchBar.delegate = self 275 | touchBar.defaultItemIdentifiers = [.stepItem, .continueItem, .resetItem] 276 | return touchBar 277 | } 278 | 279 | @available(OSX 10.12.2, *) 280 | public func touchBar(_ touchBar: NSTouchBar, makeItemForIdentifier identifier: NSTouchBarItem.Identifier) -> NSTouchBarItem? { 281 | switch identifier { 282 | case NSTouchBarItem.Identifier.stepItem: 283 | let customViewItem = NSCustomTouchBarItem(identifier: identifier) 284 | let button = NSButton(title: "⤼", target: self, action: #selector(step)) 285 | button.font = .systemFont(ofSize: 20) 286 | customViewItem.view = button 287 | return customViewItem 288 | case NSTouchBarItem.Identifier.continueItem: 289 | let customViewItem = NSCustomTouchBarItem(identifier: identifier) 290 | let button = NSButton(title: "↠", target: self, action: #selector(runUntilEnd)) 291 | button.font = .systemFont(ofSize: 20) 292 | customViewItem.view = button 293 | return customViewItem 294 | case NSTouchBarItem.Identifier.resetItem: 295 | let customViewItem = NSCustomTouchBarItem(identifier: identifier) 296 | let button = NSButton(title: "⟲", target: self, action: #selector(reset)) 297 | button.font = .systemFont(ofSize: 20) 298 | customViewItem.view = button 299 | return customViewItem 300 | default: 301 | return nil 302 | } 303 | } 304 | 305 | @objc func runUntilEnd() { 306 | guard let debugger = debugger else { 307 | return 308 | } 309 | 310 | var lastDebuggerState = debugger.debuggerState 311 | 312 | while debugger.debuggerState != nil { 313 | lastDebuggerState = debugger.debuggerState 314 | debugger.executeNextStep() 315 | } 316 | if let debuggerState = lastDebuggerState { 317 | populateViewsFromDebuggerState(debuggerState) 318 | } 319 | } 320 | 321 | @objc func step() { 322 | guard let debugger = debugger else { 323 | return 324 | } 325 | debugger.executeNextStep() 326 | 327 | if let debuggerState = debugger.debuggerState { 328 | populateViewsFromDebuggerState(debuggerState) 329 | } 330 | } 331 | 332 | private func populateViewsFromStackFrame(_ stackFrame: StackFrame) { 333 | setIRViewText(currentFunctionName: stackFrame.functionName, 334 | currentBlock: stackFrame.block, 335 | currentInstructionIndex: stackFrame.instructionIndex) 336 | 337 | self.registerValuesViewDataSource.registerValues = stackFrame.registers 338 | self.registerValuesView.reloadData() 339 | } 340 | 341 | private func didSelectStackFrame(stackFrame: Int) { 342 | populateViewsFromStackFrame(self.callStack[stackFrame]) 343 | } 344 | 345 | @objc func reset() { 346 | guard let ir = ir else { 347 | return 348 | } 349 | 350 | debugger = IRDebugger(ir: ir) 351 | 352 | setIRViewText(currentFunctionName: "main", 353 | currentBlock: ir.functions["main"]!.startBlock, 354 | currentInstructionIndex: 0) 355 | 356 | self.registerValuesViewDataSource.registerValues = [:] 357 | self.registerValuesView.reloadData() 358 | 359 | self.resultsView.attributedStringValue = NSAttributedString(string: "") 360 | 361 | self.callStack = debugger!.debuggerState!.callStack 362 | 363 | populateViewsFromDebuggerState(debugger!.debuggerState!) 364 | } 365 | 366 | private func populateViewsFromDebuggerState(_ debuggerState: DebuggerState) { 367 | self.stackFramesDataSource.stackFrames = debuggerState.callStack 368 | self.callStack = debuggerState.callStack 369 | self.stackFramesView.reloadData() 370 | self.stackFramesView.selectRowIndexes([0], byExtendingSelection: false) 371 | 372 | let resultsString = NSAttributedString(string: debuggerState.output, attributes: [ 373 | .font: NSFont(name: "Menlo Bold", size: 11)! 374 | ]) 375 | self.resultsView.attributedStringValue = resultsString 376 | } 377 | 378 | private func setIRViewText(currentFunctionName: String, currentBlock: BlockName, currentInstructionIndex: Int) { 379 | guard let ir = ir else { 380 | return 381 | } 382 | 383 | let result = NSMutableAttributedString() 384 | 385 | for (functionName, function) in ir.functions { 386 | result += (functionName + "(").monospacedString 387 | result += function.argumentRegisters.map({ $0.description }).joined(separator: ", ").monospacedString 388 | result += "): \n".monospacedString 389 | for blockName in function.blocks.keys.sorted(by: { $0.name < $1.name }) { 390 | let instructions = function.blocks[blockName]! 391 | result += " \(blockName):\n".monospacedString 392 | for (index, instruction) in instructions.enumerated() { 393 | let instructionString = NSMutableAttributedString(attributedString: (" " + instruction.debugDescription + "\n").monospacedString) 394 | if functionName == currentFunctionName && blockName == currentBlock && index == currentInstructionIndex { 395 | instructionString.addAttribute(.backgroundColor, 396 | value: #colorLiteral(red: 0.8431372549, green: 0.9098039216, blue: 0.8549019608, alpha: 1), 397 | range: NSRange(location: 0, length: instructionString.length)) 398 | } 399 | result.append(instructionString) 400 | } 401 | } 402 | result += "\n".monospacedString 403 | } 404 | 405 | self.irView.attributedStringValue = result 406 | } 407 | } 408 | 409 | private extension NSUserInterfaceItemIdentifier { 410 | static let stackFrame = NSUserInterfaceItemIdentifier("StackFrame") 411 | static let registerValue = NSUserInterfaceItemIdentifier("RegisterValue") 412 | } 413 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/IRGeneration.xcplaygroundpage/timeline.xctimeline: -------------------------------------------------------------------------------- 1 | 2 | 4 | 5 | 10 | 11 | 16 | 17 | 22 | 23 | 27 | 28 | 32 | 33 | 38 | 39 | 44 | 45 | 46 | 47 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Introduction.xcplaygroundpage/Contents.swift: -------------------------------------------------------------------------------- 1 | /*: 2 | - callout(Xcode issue): 3 | If you see the error message \ 4 | `Playground execution failed: error: Introduction.xcplaygroundpage:11:17: error: use of undeclared type 'SwiftFile'` \ 5 | wait for about 30 seconds to let the playground compile auxiliary classes that are needed for the execution in this pages. \ 6 | I have filed this bug as [rdar://30999038](rdar://30999038) 7 | 8 | 9 | # Do you know how compilers work? 10 | Your Mac by itself cannot understand Swift code, but it can only execute very low-level *assembly* instructions like `add`, `shift` or `branch`. It doesn't know about classes or even `if` statements. 11 | 12 | To make your Swift programs run on your Mac, we need a program that translates Swift into assembly code. This program is called a *compiler*. The compiler is always invoked when you build a program in Xcode or when you see `Compiling` in the status bar of a Playground. 13 | 14 | In the following pages you will have a chance to discover interactively how compilers work and which phases modern compilers go through to compile your program. 15 | 16 | As a high-level overview, consider you have written the following Swift code: 17 | */ 18 | let sourceFile: SwiftFile = #fileLiteral(resourceName: "Simple program.swift") 19 | /*: 20 | Then the Swift compiler will *compile* the Swift code into assembly code that may look like this: 21 | */ 22 | do { 23 | try Compiler.compile(swiftFile: sourceFile) 24 | } catch let error as CompilationError { 25 | error 26 | } 27 | /*: 28 | 29 | * callout(Discover): 30 | Choose different Swift programs above and see how the assembly code changes. 31 | 32 | 33 | * note: 34 | To change the program to compile, double-click on “Simple Program” on top of the page 35 | 36 | [❯ Start with the lexer](Lexer) 37 | 38 | --- 39 | */ 40 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Introduction.xcplaygroundpage/timeline.xctimeline: -------------------------------------------------------------------------------- 1 | 2 | 4 | 5 | 10 | 11 | 15 | 16 | 20 | 21 | 25 | 26 | 31 | 32 | 37 | 38 | 43 | 44 | 49 | 50 | 55 | 56 | 61 | 62 | 63 | 64 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Lexer.xcplaygroundpage/Contents.swift: -------------------------------------------------------------------------------- 1 | /*: 2 | # Lexer 3 | 4 | For a computer, even source code is just a stream of characters. 5 | 6 | To understand the code's semantics, it first *lexes* the stream of characters into tokens. A token is, for example, the keyword `if`, or the string literal `"hello world"`. These tokens are classified into categories like the different brackets, identifiers, and operators. 7 | 8 | * callout(Discover): 9 | Hover over the source code in the live view to see the different tokens it consists of. \ 10 | Select different example source files to see how they get lexed. 11 | 12 | 13 | * note: 14 | Make sure the live view is activated to see the source code. 15 | 16 | */ 17 | let sourceFile: SwiftFile = #fileLiteral(resourceName: "Simple program.swift") 18 | /*: 19 | [❮ Back to the introduction](Introduction) 20 | 21 | [❯ Continue with the parser](Parser) 22 | 23 | --- 24 | */ 25 | // Setup for the live view 26 | import PlaygroundSupport 27 | PlaygroundPage.current.liveView = TokensExplorer(forSourceFile: sourceFile) 28 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Lexer.xcplaygroundpage/timeline.xctimeline: -------------------------------------------------------------------------------- 1 | 2 | 4 | 5 | 6 | 7 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Optimisation.xcplaygroundpage/Contents.swift: -------------------------------------------------------------------------------- 1 | /*: 2 | # Optimisation 3 | 4 | You may have noticed that the IR generated before is often far from optimal. Instead of making the generation of IR more complex, another compiler phase *optimises* the IR. 5 | 6 | In the real Swift compiler more than 100 different optimisation passes are applied. 7 | 8 | We will explore some of the more basic ones in the following. 9 | 10 | * callout(Discover): 11 | Option-click on the different optimisation options below to see what they do. 12 | */ 13 | let optimisationOptions: OptimisationOptions = [.constantExprssionEvaluation, 14 | .constantPropagation, 15 | .deadStoreElimination, 16 | .deadCodeElimination, 17 | .emptyBlockElimination, 18 | .inlineJumpTargets, 19 | .deadBlockElimination 20 | ] 21 | 22 | let sourceFile: SwiftFile = #fileLiteral(resourceName: "Simple program.swift") 23 | 24 | let optimiser = Optimiser(options: optimisationOptions) 25 | /*: 26 | * callout(Experiment): 27 | Explore how the optimised code changes when you remove some options in the array above. \ 28 | You may notice that many options only produce good results when combined with other options like `.constantExprssionEvaluation` and `.constantPropagation`. 29 | 30 | After being optimised, the IR is converted into machine code that is specific to the architecture on which the code will be executed. This means that different machine code is generated depending on whether you compile your code for iPhone/iPad (which have ARM processors) or Mac (which have Intel processors). 31 | 32 | [❮ Back to IR Generation](IRGeneration) 33 | 34 | [❯ Finish](Finish) 35 | 36 | --- 37 | */ 38 | // Setup for the live view 39 | import PlaygroundSupport 40 | PlaygroundPage.current.liveView = OptimisationExplorer(withOptimiser: optimiser, forSourceFile: sourceFile) 41 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Optimisation.xcplaygroundpage/Sources/OptimisationExplorer.swift: -------------------------------------------------------------------------------- 1 | import Cocoa 2 | 3 | /// View to see the differences between unoptimised and optimised IR 4 | public class OptimisationExplorer: NSSplitView { 5 | 6 | private let irView = NSSplitView() 7 | 8 | /// - Parameters: 9 | /// - optimiser: The optimiser used to optimise the IR generated by the given Swift file 10 | /// - sourceFile: The Swift file to compile with and without optimisations enabled 11 | public init(withOptimiser optimiser: Optimiser, forSourceFile sourceFile: SwiftFile) { 12 | super.init(frame: CGRect(x: 0, y: 0, width: 500, height: 600)) 13 | 14 | self.wantsLayer = true 15 | self.layer!.backgroundColor = NSColor(white: 247/255, alpha: 1).cgColor 16 | 17 | // Set up source header 18 | let sourceCodeHeader = NSTextField(labelWithString: "Source Code") 19 | sourceCodeHeader.font = .systemFont(ofSize: 33, weight: .semibold) 20 | addArrangedSubview(sourceCodeHeader) 21 | 22 | // Set up source view 23 | let sourceView = NSTextField() 24 | sourceView.isEditable = false 25 | sourceView.backgroundColor = NSColor.white 26 | sourceView.drawsBackground = true 27 | sourceView.isBordered = false 28 | sourceView.attributedStringValue = sourceFile.highlightedString 29 | sourceView.translatesAutoresizingMaskIntoConstraints = false 30 | 31 | let sourceScrollView = NSScrollView() 32 | sourceScrollView.documentView = sourceView 33 | sourceScrollView.translatesAutoresizingMaskIntoConstraints = false 34 | sourceScrollView.hasVerticalScroller = true 35 | sourceScrollView.addConstraint(NSLayoutConstraint(item: sourceView, attribute: .width, relatedBy: .equal, toItem: sourceScrollView, attribute: .width, multiplier: 1, constant: 0)) 36 | 37 | 38 | let sourceWrapper = NSStackView() 39 | sourceWrapper.orientation = .vertical 40 | sourceWrapper.translatesAutoresizingMaskIntoConstraints = false 41 | sourceWrapper.addFullWidthView(sourceCodeHeader) 42 | sourceWrapper.addFullWidthView(sourceScrollView) 43 | 44 | self.addArrangedSubview(sourceWrapper) 45 | 46 | // Compile the program 47 | let ast: ASTRoot 48 | do { 49 | ast = try Parser.parse(sourceFile: sourceFile) 50 | try Typechecker.typecheck(node: ast) 51 | } catch { 52 | let error = error as! CompilationError 53 | let errorView = NSTextField() 54 | errorView.translatesAutoresizingMaskIntoConstraints = false 55 | errorView.stringValue = "Compilation error:\n\(error)" 56 | errorView.isBordered = false 57 | 58 | self.addArrangedSubview(errorView) 59 | return 60 | } 61 | let unoptimisedIR = IRGen.generateIR(forAST: ast) 62 | let optimisedIR = optimiser.optimise(ir: unoptimisedIR) 63 | 64 | 65 | func createIRRegion(forIR ir: IR, withTitle title: String) -> NSView { 66 | let irView = NSTextField() 67 | irView.backgroundColor = NSColor.white 68 | irView.drawsBackground = true 69 | irView.isBordered = false 70 | irView.attributedStringValue = ir.debugDescription.monospacedString 71 | irView.isEditable = false 72 | irView.translatesAutoresizingMaskIntoConstraints = false 73 | 74 | let headerView = NSTextField(labelWithString: title) 75 | headerView.font = .systemFont(ofSize: 33, weight: .semibold) 76 | headerView.translatesAutoresizingMaskIntoConstraints = false 77 | 78 | let irScrollView = NSScrollView() 79 | irScrollView.documentView = irView 80 | irScrollView.translatesAutoresizingMaskIntoConstraints = false 81 | irScrollView.addConstraint(NSLayoutConstraint(item: irView, attribute: .width, relatedBy: .greaterThanOrEqual, toItem: irScrollView, attribute: .width, multiplier: 1, constant: 0)) 82 | 83 | let stackView = NSStackView() 84 | stackView.orientation = .vertical 85 | stackView.addFullWidthView(headerView) 86 | stackView.addFullWidthView(irScrollView) 87 | stackView.translatesAutoresizingMaskIntoConstraints = false 88 | 89 | return stackView 90 | } 91 | 92 | let unoptimisedIRView = createIRRegion(forIR: unoptimisedIR, withTitle: "Unoptimised") 93 | let optimisedIRView = createIRRegion(forIR: optimisedIR, withTitle: "Optimised") 94 | 95 | irView.isVertical = true 96 | irView.dividerStyle = .thick 97 | irView.translatesAutoresizingMaskIntoConstraints = false 98 | irView.addArrangedSubview(unoptimisedIRView) 99 | irView.addArrangedSubview(optimisedIRView) 100 | 101 | self.addArrangedSubview(irView) 102 | } 103 | 104 | required public init?(coder: NSCoder) { 105 | fatalError("init(coder:) has not been implemented") 106 | } 107 | 108 | public override func viewDidMoveToWindow() { 109 | super.viewDidMoveToWindow() 110 | self.setPosition(self.frame.size.height * 0.3, ofDividerAt: 0) 111 | irView.setPosition(irView.frame.size.width * 0.5, ofDividerAt: 0) 112 | } 113 | } 114 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Optimisation.xcplaygroundpage/timeline.xctimeline: -------------------------------------------------------------------------------- 1 | 2 | 4 | 5 | 10 | 11 | 16 | 17 | 22 | 23 | 24 | 25 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Parser.xcplaygroundpage/Contents.swift: -------------------------------------------------------------------------------- 1 | /*: 2 | # Parser 3 | 4 | After lexing the source code into tokens, the *parser* parses the stream of tokens into an *abstract syntax tree (AST)*. This is a tree representation of your source code, which models the nesting that you probably see from just looking at it. 5 | */ 6 | let sourceFile: SwiftFile = #fileLiteral(resourceName: "Simple program.swift") 7 | /*: 8 | * callout(Discover): 9 | Explore the AST of the different sample programs in the live view. 10 | 11 | 12 | * note: 13 | Since the `else` part has not been implemented yet below, “Program with else” will fail to compile. 14 | 15 | In the following you see the source code for parsing `if` statements in a simplified Swift compiler: 16 | */ 17 | class MyParser: Parser { 18 | override func parseIfStatement() throws -> Statement { 19 | // Check that the next token is indeed `if`, otherwise emit an error 20 | guard nextToken == .if else { 21 | throw CompilationError(sourceRange: nextToken.sourceRange, 22 | errorMessage: "Expected 'if' but saw \(nextToken!)") 23 | } 24 | // Save the source range in which the `if` keyword occurred 25 | // This will potentially be used later for error messages 26 | let ifRange = nextToken.sourceRange 27 | // Consume the token so that we can have a look at the next one 28 | try consumeToken() 29 | // Parse the if statement's condition 30 | let condition = try parseExpression() 31 | // Parse the body of the if statment 32 | let body = try parseBraceStatement() 33 | 34 | let elseBody: BraceStatement? = nil 35 | let elseRange: SourceRange? = nil 36 | 37 | // #-----------------------------------# 38 | // # Code to parse else statement here # 39 | // #-----------------------------------# 40 | 41 | // Construct an if statement and return it 42 | return IfStatement(condition: condition, 43 | body: body, 44 | elseBody: elseBody, 45 | ifRange: ifRange, 46 | elseRange: elseRange, 47 | sourceRange: range(startingAt: ifRange.start)) 48 | } 49 | } 50 | /*: 51 | Note that the code to parse the `else` part of the `if` statement is still missing. 52 | 53 | * callout(Experiment): 54 | Can you implement parsing of the `else` part similar to how the body of the `if` part gets parsed? \ 55 | Pick the example source file “Program with else” to verify that your implementation works. 56 | 57 | [View a possible solution](ParserSolution) 58 | 59 | After the source code has been transformed into an AST, the AST is *type checked*. This verifies that there is no type system violation in the code like trying to add an integer with a string `5 + "abc"` or invoking a function with the wrong type of argument. If this passes, the compiler can continue to the next phase. 60 | 61 | [❮ Back to the lexer](Lexer) 62 | 63 | [❯ Continue with IR Generation](IRGeneration) 64 | 65 | --- 66 | */ 67 | // Setup for the live view 68 | import PlaygroundSupport 69 | PlaygroundPage.current.liveView = ASTExplorer(forSourceFile: sourceFile, withParser: MyParser()) 70 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Parser.xcplaygroundpage/Sources/ASTExplorer.swift: -------------------------------------------------------------------------------- 1 | import Cocoa 2 | 3 | /// View that let's the user explore the AST by showing a `TokensExplorer` on top 4 | /// and an expendable AST below it 5 | public class ASTExplorer: TokensExplorer { 6 | 7 | /// - Parameters: 8 | /// - sourceFile: The source file for which the AST shall be explored 9 | /// - parser: The parser used to parse the source code 10 | public override init(forSourceFile sourceFile: SwiftFile, withParser parser: Parser = Parser()) { 11 | super.init(forSourceFile: sourceFile, withParser: parser) 12 | 13 | // Parse the source code for the AST viewer 14 | var ast: ASTRoot? 15 | var compilationError: CompilationError? = nil 16 | do { 17 | ast = try parser.parse(sourceFile: sourceFile) 18 | } catch { 19 | compilationError = (error as! CompilationError) 20 | ast = nil 21 | } 22 | 23 | // Restrict the source viewer's height to 100 24 | let heightConstraint = NSLayoutConstraint(item: sourceViewer, 25 | attribute: .height, 26 | relatedBy: .equal, 27 | toItem: nil, 28 | attribute: .notAnAttribute, 29 | multiplier: 1, 30 | constant: 100) 31 | heightConstraint.priority = NSLayoutConstraint.Priority(rawValue: 900) 32 | sourceViewer.addConstraint(heightConstraint) 33 | 34 | // Create headers 35 | let astHeader = NSTextField(labelWithString: "AST") 36 | astHeader.font = .systemFont(ofSize: 33, weight: .semibold) 37 | 38 | // Create the ASTView 39 | let astView: NSView 40 | if let ast = ast { 41 | astView = ASTView(frame: self.frame, astRoot: ast, selectionCallback: self.highlightSourceCodeForNode) 42 | } else { 43 | astView = NSTextField(labelWithString: "Compilation error:\n\(compilationError!)") 44 | } 45 | 46 | // Put the ASTView into a scroll view 47 | let astScrollView = NSScrollView() 48 | astScrollView.documentView = astView 49 | astScrollView.hasVerticalScroller = true 50 | astScrollView.addConstraint(NSLayoutConstraint(item: astView, attribute: .width, relatedBy: .equal, toItem: astScrollView, attribute: .width, multiplier: 1, constant: 0)) 51 | 52 | let footerView = NSStackView() 53 | footerView.orientation = .vertical 54 | footerView.translatesAutoresizingMaskIntoConstraints = false 55 | footerView.addFullWidthView(astHeader) 56 | footerView.addFullWidthView(astScrollView) 57 | 58 | addArrangedSubview(footerView) 59 | } 60 | 61 | public required init?(coder: NSCoder) { 62 | fatalError("init(coder:) has not been implemented") 63 | } 64 | 65 | public override func viewDidMoveToWindow() { 66 | super.viewDidMoveToWindow() 67 | self.setPosition(self.frame.size.height / 3, ofDividerAt: 0) 68 | self.updateTrackingAreas() 69 | } 70 | 71 | private func highlightSourceCodeForNode(node: ASTNode?) { 72 | self.sourceViewer.highlight(range: node?.sourceRange) 73 | } 74 | } 75 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Parser.xcplaygroundpage/Sources/ASTView.swift: -------------------------------------------------------------------------------- 1 | import Cocoa 2 | 3 | fileprivate class TableViewCell: NSTableCellView { 4 | override var backgroundStyle: BackgroundStyle { 5 | didSet { 6 | switch backgroundStyle { 7 | case .light: 8 | self.textField?.textColor = NSColor.black 9 | case .dark: 10 | self.textField?.textColor = NSColor.white 11 | default: 12 | break 13 | } 14 | } 15 | } 16 | } 17 | 18 | fileprivate class ASTTreeViewItem { 19 | let node: ASTNode 20 | 21 | var children: [ASTTreeViewItem] { 22 | let childNodes: [ASTNode] 23 | if let casted = node as? IfStatement { 24 | childNodes = [casted.condition, casted.body, casted.elseBody].compactMap({ $0 }) 25 | } else if let casted = node as? BraceStatement { 26 | childNodes = casted.body 27 | } else if let casted = node as? ReturnStatement { 28 | childNodes = [casted.expression] 29 | } else if let casted = node as? FunctionDeclaration { 30 | childNodes = casted.parameters + [casted.body] 31 | } else if let casted = node as? BinaryOperatorExpression { 32 | childNodes = [casted.lhs, casted.rhs] 33 | } else if let casted = node as? FunctionCallExpression { 34 | childNodes = casted.arguments.compactMap({ $0 }) 35 | } else if let casted = node as? ASTRoot { 36 | childNodes = casted.statements 37 | } else { 38 | childNodes = [] 39 | } 40 | return childNodes.map(ASTTreeViewItem.init) 41 | } 42 | 43 | public var label: NSAttributedString { 44 | let monospaceFontAttributes = [ 45 | NSAttributedString.Key.font: NSFont(name: "Menlo", size: NSFont.systemFontSize)! 46 | ] 47 | if node is IfStatement { 48 | let str = NSMutableAttributedString(string: "If statement") 49 | str.setAttributes(monospaceFontAttributes, range: NSRange(location: 0, length: 2)) 50 | return str 51 | } else if node is BraceStatement { 52 | return NSAttributedString(string: "Brace statement") 53 | } else if node is ReturnStatement { 54 | return NSAttributedString(string: "Return statement") 55 | } else if let casted = node as? BinaryOperatorExpression { 56 | let str = NSMutableAttributedString(string: "Binary operator \(casted.operator.sourceCodeName)") 57 | str.setAttributes(monospaceFontAttributes, range: NSRange(location: 16, length: str.length - 16)) 58 | return str 59 | } else if let casted = node as? FunctionDeclaration { 60 | let str = NSMutableAttributedString(string: "Function \(casted.name)") 61 | str.setAttributes(monospaceFontAttributes, range: NSRange(location: 9, length: str.length - 9)) 62 | return str 63 | } else if let casted = node as? VariableDeclaration { 64 | let str = NSMutableAttributedString(string: "Variable \(casted.name)") 65 | str.setAttributes(monospaceFontAttributes, range: NSRange(location: 9, length: str.length - 9)) 66 | return str 67 | } else if let casted = node as? FunctionCallExpression { 68 | let str = NSMutableAttributedString(string: "Function call \(casted.functionName)") 69 | str.setAttributes(monospaceFontAttributes, range: NSRange(location: 14, length: str.length - 14)) 70 | return str 71 | } else if let casted = node as? IntegerLiteralExpression { 72 | let str = NSMutableAttributedString(string: "Integer literal \(casted.value)") 73 | str.setAttributes(monospaceFontAttributes, range: NSRange(location: 16, length: str.length - 16)) 74 | return str 75 | } else if let casted = node as? StringLiteralExpression { 76 | let str = NSMutableAttributedString(string: "String literal \(casted.value)") 77 | str.setAttributes(monospaceFontAttributes, range: NSRange(location: 15, length: str.length - 15)) 78 | return str 79 | } else if let casted = node as? IdentifierReferenceExpression { 80 | let str = NSMutableAttributedString(string: "Identifier reference \(casted.name)") 81 | str.setAttributes(monospaceFontAttributes, range: NSRange(location: 21, length: str.length - 21)) 82 | return str 83 | } else { 84 | return NSAttributedString(string: "\(type(of: node))") 85 | } 86 | } 87 | 88 | init(node: ASTNode) { 89 | self.node = node 90 | } 91 | } 92 | 93 | fileprivate class ASTViewDataSource: NSObject, NSOutlineViewDataSource, NSOutlineViewDelegate { 94 | 95 | let root: ASTTreeViewItem 96 | let selectionCallback: (ASTNode?) -> Void 97 | 98 | init(root: ASTRoot, selectionCallback: @escaping (ASTNode?) -> Void) { 99 | self.root = ASTTreeViewItem(node: root) 100 | self.selectionCallback = selectionCallback 101 | } 102 | 103 | func outlineView(_ outlineView: NSOutlineView, numberOfChildrenOfItem item: Any?) -> Int { 104 | if let item = item as? ASTTreeViewItem { 105 | return item.children.count 106 | } else { 107 | return root.children.count 108 | } 109 | } 110 | 111 | func outlineView(_ outlineView: NSOutlineView, child index: Int, ofItem item: Any?) -> Any { 112 | if let item = item as? ASTTreeViewItem { 113 | return item.children[index] 114 | } 115 | 116 | return root.children[index] 117 | } 118 | 119 | func outlineView(_ outlineView: NSOutlineView, isItemExpandable item: Any) -> Bool { 120 | return self.outlineView(outlineView, numberOfChildrenOfItem: item) > 0 121 | } 122 | 123 | func outlineView(_ outlineView: NSOutlineView, viewFor tableColumn: NSTableColumn?, item: Any) -> NSView? { 124 | let item = item as! ASTTreeViewItem 125 | let textView = NSTextField(labelWithAttributedString: item.label) 126 | let cell = TableViewCell() 127 | cell.textField = textView 128 | cell.addSubview(textView) 129 | return cell 130 | } 131 | 132 | fileprivate func outlineViewSelectionDidChange(_ notification: Notification) { 133 | let outlineView = notification.object as! ASTView 134 | let selectedItem = outlineView.item(atRow: outlineView.selectedRow) as! ASTTreeViewItem? 135 | selectionCallback(selectedItem?.node) 136 | } 137 | } 138 | 139 | public class ASTView: NSOutlineView { 140 | 141 | private let astViewDataSource: ASTViewDataSource 142 | 143 | public init(frame: CGRect, astRoot: ASTRoot, selectionCallback: @escaping (ASTNode?) -> Void) { 144 | self.astViewDataSource = ASTViewDataSource(root: astRoot, selectionCallback: selectionCallback) 145 | 146 | super.init(frame: frame) 147 | 148 | self.dataSource = self.astViewDataSource 149 | self.delegate = self.astViewDataSource 150 | 151 | let column = NSTableColumn(identifier: .theColumn) 152 | column.tableView = self 153 | self.addTableColumn(column) 154 | self.outlineTableColumn = column 155 | 156 | self.headerView = nil 157 | 158 | self.expandItem(nil, expandChildren: true) 159 | } 160 | 161 | required public init?(coder: NSCoder) { 162 | fatalError("init(coder:) has not been implemented") 163 | } 164 | 165 | } 166 | 167 | private extension NSUserInterfaceItemIdentifier { 168 | static let theColumn = NSUserInterfaceItemIdentifier("TheColumn") 169 | } 170 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/Parser.xcplaygroundpage/timeline.xctimeline: -------------------------------------------------------------------------------- 1 | 2 | 4 | 5 | 9 | 10 | 15 | 16 | 21 | 22 | 27 | 28 | 33 | 34 | 35 | 36 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Pages/ParserSolution.xcplaygroundpage/Contents.swift: -------------------------------------------------------------------------------- 1 | /*: 2 | # Parser Experiment Solution 3 | 4 | A possible solution to successfully implement the parsing of the else part is to change the variables `elseBody` and `elseRange` from `let` to `var` and perform the following instead of the placeholder: 5 | 6 | - Check if the next token is `else` (`nextToken == .else`) 7 | - If yes, retrieve the `else` token's range (`elseRange = nextToken.sourceRange`) 8 | - Consume the `else` token (`try consumeToken()`) 9 | - Parse the else body and store it (`elseBody = try parseBraceStatement()`) 10 | */ 11 | 12 | class MyParser: Parser { 13 | override func parseIfStatement() throws -> Statement { 14 | // Check that the next token is indeed `if`, otherwise emit an error 15 | guard nextToken == .if else { 16 | throw CompilationError(sourceRange: nextToken.sourceRange, 17 | errorMessage: "Expected 'if' but saw \(nextToken!)") 18 | } 19 | // Save the source range in which the `if` keyword occurred 20 | // This will potentially be used later for error messages 21 | let ifRange = nextToken.sourceRange 22 | // Consume the so that we can have a look at the next token 23 | try consumeToken() 24 | // Parse the if statement's condition 25 | let condition = try parseExpression() 26 | // Parse the body of the if statment 27 | let body = try parseBraceStatement() 28 | 29 | // ================================== // 30 | 31 | var elseBody: BraceStatement? = nil 32 | var elseRange: SourceRange? = nil 33 | 34 | // Check if the next token is `else` 35 | if nextToken == .else { 36 | // Retrieve the else token's source range 37 | elseRange = nextToken.sourceRange 38 | // Consume the `else` token since we have handled it 39 | try consumeToken() 40 | // Parse the else body 41 | elseBody = try parseBraceStatement() 42 | } 43 | 44 | // ================================== // 45 | 46 | // Construct an if statement and return it 47 | return IfStatement(condition: condition, 48 | body: body, 49 | elseBody: elseBody, 50 | ifRange: ifRange, 51 | elseRange: elseRange, 52 | sourceRange: range(startingAt: ifRange.start)) 53 | } 54 | } 55 | 56 | /*: 57 | [❮ Back to the parser](Parser) 58 | 59 | */ 60 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Resources/Fibonacci.swift: -------------------------------------------------------------------------------- 1 | func fibonacci(_ a: Int) -> Int { 2 | if a <= 0 { 3 | return 1 4 | } 5 | return fibonacci(a - 1) + fibonacci(a - 2) 6 | } 7 | print(fibonacci(2)) 8 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Resources/Program with else.swift: -------------------------------------------------------------------------------- 1 | if 1 + 2 == 3 { 2 | print("Math works") 3 | } else { 4 | print("Math broken") 5 | } 6 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Resources/Program with syntax error.swift: -------------------------------------------------------------------------------- 1 | if 4 + 4 == 8 2 | print("abc") 3 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Resources/Simple program.swift: -------------------------------------------------------------------------------- 1 | if 3 + 5 == 8 { 2 | print("3 + 5 = 8") 3 | } 4 | print("Program finished") 5 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/AST.swift: -------------------------------------------------------------------------------- 1 | /// A node in the source codes abstract syntax tree (AST) 2 | public class ASTNode: CustomDebugStringConvertible, CustomPlaygroundDisplayConvertible { 3 | /// The range in the source code which this node represents 4 | public let sourceRange: SourceRange 5 | 6 | public var debugDescription: String { 7 | let printer = ASTPrinter() 8 | printer.print(self) 9 | return printer.output 10 | } 11 | 12 | public var playgroundDescription: Any { 13 | return debugDescription.monospacedString.withPlaygroundQuickLookBackgroundColor 14 | } 15 | 16 | init(sourceRange: SourceRange) { 17 | self.sourceRange = sourceRange 18 | } 19 | } 20 | 21 | /// Represents a source file with its statements 22 | public class ASTRoot: ASTNode { 23 | public let statements: [Statement] 24 | 25 | init(statements: [Statement], sourceRange: SourceRange) { 26 | self.statements = statements 27 | super.init(sourceRange: sourceRange) 28 | } 29 | } 30 | 31 | /// Abstract base class for statements in the AST 32 | /// 33 | /// Statements do not return values whereas expresions do 34 | public class Statement: ASTNode { 35 | } 36 | 37 | /// An if statement in the AST 38 | public class IfStatement: Statement { 39 | /// The condition to be evaluated to decide if the statement's body shall 40 | /// be executed 41 | public let condition: Expression 42 | /// The body to be executed only if the condition evaluates to true 43 | public let body: BraceStatement 44 | /// The body of the else-clause if it existed 45 | public let elseBody: BraceStatement? 46 | /// The source range of the `if` keyword 47 | public let ifRange: SourceRange 48 | /// The source range of the `else` keyword if it existed 49 | public let elseRange: SourceRange? 50 | 51 | /// Create a node in the AST representing an `if` statement 52 | /// 53 | /// - Parameters: 54 | /// - condition: The condition to evaluate in order to determine if the if body shall be executed 55 | /// - body: The body of the `if` statement 56 | /// - elseBody: If the else statment has an `else` part, its body, otherwise `nil` 57 | /// - ifRange: The source range of the `if` keyword 58 | /// - elseRange: The source range of the `else` keyword, if present 59 | /// - sourceRange: The source range of the entire statement 60 | public init(condition: Expression, body: BraceStatement, elseBody: BraceStatement?, ifRange: SourceRange, elseRange: SourceRange?, sourceRange: SourceRange) { 61 | self.condition = condition 62 | self.body = body 63 | self.elseBody = elseBody 64 | self.ifRange = ifRange 65 | self.elseRange = elseRange 66 | super.init(sourceRange: sourceRange) 67 | } 68 | } 69 | 70 | /// A brace statement is a block of other statements combined using '{' and '}' 71 | public class BraceStatement: Statement { 72 | /// The statements this brace statement contains 73 | public let body: [Statement] 74 | 75 | init(body: [Statement], sourceRange: SourceRange) { 76 | self.body = body 77 | super.init(sourceRange: sourceRange) 78 | } 79 | } 80 | 81 | /// A return statement starting with `return` 82 | public class ReturnStatement: Statement { 83 | /// The expression whose value shall be returned 84 | public let expression: Expression 85 | 86 | init(expression: Expression, sourceRange: SourceRange) { 87 | self.expression = expression 88 | super.init(sourceRange: sourceRange) 89 | } 90 | } 91 | 92 | /// Abstract base class for expresions in the AST 93 | /// 94 | /// In contrast to statements, expressions calculate values 95 | public class Expression: Statement { 96 | } 97 | 98 | /// A binary operator expression combines the value two expressions using an infix 99 | /// operator like '+' or '==' 100 | public class BinaryOperatorExpression: Expression { 101 | 102 | /// Enumeration of all the binary operators supported by the BinaryOperatorExpression 103 | public enum Operator { 104 | case add 105 | case sub 106 | case equal 107 | case lessOrEqual 108 | 109 | /// The name with which this operator is spellec out in the source code 110 | public var sourceCodeName: String { 111 | switch self { 112 | case .add: 113 | return "+" 114 | case .sub: 115 | return "-" 116 | case .equal: 117 | return "==" 118 | case .lessOrEqual: 119 | return "<=" 120 | } 121 | } 122 | 123 | /// The precedence of the operator, e.g. '*' has higher precedence than '+'. 124 | /// 125 | /// A higher precedence value means that the value should bind stronger than 126 | /// values with lower precedence 127 | var precedence: Int { 128 | switch self { 129 | case .add: 130 | return 2 131 | case .sub: 132 | return 2 133 | case .equal: 134 | return 1 135 | case .lessOrEqual: 136 | return 1 137 | } 138 | } 139 | } 140 | 141 | /// The left-hand-side of the operator 142 | public let lhs: Expression 143 | /// The right-hand-side of the operator 144 | public let rhs: Expression 145 | /// The operator to combine the two expressions 146 | public let `operator`: Operator 147 | 148 | init(lhs: Expression, rhs: Expression, operator: Operator) { 149 | self.lhs = lhs 150 | self.rhs = rhs 151 | self.operator = `operator` 152 | let sourceRange = SourceRange(start: self.lhs.sourceRange.start, 153 | end: self.rhs.sourceRange.end) 154 | super.init(sourceRange: sourceRange) 155 | } 156 | } 157 | 158 | /// A constant integer spelled out in the source code like '42' 159 | public class IntegerLiteralExpression: Expression { 160 | /// The value of the literal 161 | public let value: Int 162 | 163 | init(value: Int, sourceRange: SourceRange) { 164 | self.value = value 165 | super.init(sourceRange: sourceRange) 166 | } 167 | } 168 | 169 | /// A constant string spelled out in the source code like 'hello world' 170 | public class StringLiteralExpression: Expression { 171 | /// The value of the literal 172 | public let value: String 173 | 174 | init(value: String, sourceRange: SourceRange) { 175 | self.value = value 176 | super.init(sourceRange: sourceRange) 177 | } 178 | } 179 | 180 | public class IdentifierReferenceExpression: Expression { 181 | /// The name of the referenced identifier 182 | public let name: String 183 | public var referencedDeclaration: Declaration? 184 | 185 | init(name: String, sourceRange: SourceRange) { 186 | self.name = name 187 | super.init(sourceRange: sourceRange) 188 | } 189 | } 190 | 191 | /// Call of a function with a single argument 192 | /// 193 | /// This is currently only used to model the 'print' function 194 | public class FunctionCallExpression: Expression { 195 | /// The name of the function to call 196 | public let functionName: String 197 | /// The arguments to pass on the function call 198 | public let arguments: [Expression] 199 | /// The source range of the function name 200 | public let functionNameRange: SourceRange 201 | 202 | init(functionName: String, arguments: [Expression], functionNameRange: SourceRange, sourceRange: SourceRange) { 203 | self.functionName = functionName 204 | self.arguments = arguments 205 | self.functionNameRange = functionNameRange 206 | super.init(sourceRange: sourceRange) 207 | } 208 | } 209 | 210 | 211 | /// Abstract base class for declarations like functions or function arguments 212 | public class Declaration: Statement { 213 | /// The type of the declaration 214 | let type: Type 215 | 216 | init(type: Type, sourceRange: SourceRange) { 217 | self.type = type 218 | super.init(sourceRange: sourceRange) 219 | } 220 | } 221 | 222 | /// Declaration of a function's argument 223 | public class VariableDeclaration: Declaration, Hashable { 224 | /// The name of the variable 225 | public let name: String 226 | 227 | 228 | public func hash(into hasher: inout Hasher) { 229 | hasher.combine(name) 230 | hasher.combine(type) 231 | } 232 | 233 | init(name: String, type: Type, sourceRange: SourceRange) { 234 | self.name = name 235 | super.init(type: type, sourceRange: sourceRange) 236 | } 237 | 238 | public static func ==(lhs: VariableDeclaration, rhs: VariableDeclaration) -> Bool { 239 | return lhs.name == rhs.name && lhs.type == rhs.type 240 | } 241 | } 242 | 243 | /// A function declaration 244 | public class FunctionDeclaration: Declaration { 245 | /// The function's name 246 | public let name: String 247 | /// The parameters the function takes 248 | public let parameters: [VariableDeclaration] 249 | /// The name of the function's return type 250 | public let returnType: Type 251 | /// The function's body 252 | public let body: BraceStatement 253 | 254 | public init(name: String, parameters: [VariableDeclaration], returnType: Type, body: BraceStatement, sourceRange: SourceRange) { 255 | self.name = name 256 | self.parameters = parameters 257 | self.returnType = returnType 258 | self.body = body 259 | super.init(type: .function, sourceRange: sourceRange) 260 | } 261 | } 262 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/ASTPrinter.swift: -------------------------------------------------------------------------------- 1 | /// Class that prints an AST into a string that can later be retrieved as the output property 2 | class ASTPrinter: ASTWalker { 3 | typealias Result = Void 4 | 5 | /// The output of the printer after `print` has been called 6 | public var output: String = "" 7 | 8 | /// Keep a stack of variables that indicate whether a body has been printed for a node 9 | /// and if the closing paranthesis shall thus be indented or not 10 | private var bodyPrinted: [Bool] = [] 11 | private var indentation = 0 12 | 13 | public func print(_ node: ASTNode) { 14 | walk(node) 15 | } 16 | 17 | private func setBodyPrinted() { 18 | bodyPrinted[bodyPrinted.count - 1] = true 19 | } 20 | 21 | private func print(_ string: String, withIndentation: Bool = true, withNewline: Bool = true) { 22 | if withIndentation { 23 | for _ in 0.. Void { 36 | setBodyPrinted() 37 | print("", withIndentation: false) 38 | walk(ifStatement.condition) 39 | walk(ifStatement.body) 40 | if let elseBody = ifStatement.elseBody { 41 | walk(elseBody) 42 | } 43 | } 44 | 45 | func visit(braceStatement: BraceStatement) -> Void { 46 | if braceStatement.body.count > 0 { 47 | print("", withIndentation: false) 48 | setBodyPrinted() 49 | } 50 | for statement in braceStatement.body { 51 | walk(statement) 52 | } 53 | } 54 | 55 | func visit(returnStatement: ReturnStatement) -> Void { 56 | setBodyPrinted() 57 | print("", withIndentation: false) 58 | walk(returnStatement.expression) 59 | } 60 | 61 | func visit(binaryOperatorExpression: BinaryOperatorExpression) -> Void { 62 | setBodyPrinted() 63 | print(" operator=\(binaryOperatorExpression.operator)", withIndentation: false) 64 | walk(binaryOperatorExpression.lhs) 65 | walk(binaryOperatorExpression.rhs) 66 | } 67 | 68 | func visit(integerLiteralExpression: IntegerLiteralExpression) -> Void { 69 | print(" value=\(integerLiteralExpression.value)", withIndentation: false, withNewline: false) 70 | } 71 | 72 | func visit(stringLiteralExpression: StringLiteralExpression) -> Void { 73 | print(" value=\(stringLiteralExpression.value)", withIndentation: false, withNewline: false) 74 | } 75 | 76 | func visit(identifierReferenceExpression: IdentifierReferenceExpression) -> Void { 77 | print(" name=\(identifierReferenceExpression.name)", withIndentation: false, withNewline: false) 78 | } 79 | 80 | func visit(functionCallExpression: FunctionCallExpression) -> Void { 81 | print(" name=\(functionCallExpression.functionName)", withIndentation: false, withNewline: false) 82 | if !functionCallExpression.arguments.isEmpty { 83 | setBodyPrinted() 84 | print("", withIndentation: false) 85 | } 86 | for argument in functionCallExpression.arguments { 87 | walk(argument) 88 | } 89 | } 90 | 91 | func visit(variableDeclaration: VariableDeclaration) -> Void { 92 | print(" name=\(variableDeclaration.name) type=\(variableDeclaration.type)", withIndentation: false, withNewline: false) 93 | } 94 | 95 | func visit(functionDeclaration: FunctionDeclaration) -> Void { 96 | print(" name=\(functionDeclaration.name) returnType=\(functionDeclaration.returnType)", withIndentation: false) 97 | setBodyPrinted() 98 | for parameter in functionDeclaration.parameters { 99 | walk(parameter) 100 | } 101 | walk(functionDeclaration.body) 102 | } 103 | 104 | 105 | func visit(astRoot: ASTRoot) -> Void { 106 | if astRoot.statements.count > 0 { 107 | print("", withIndentation: false) 108 | setBodyPrinted() 109 | } 110 | 111 | for statement in astRoot.statements { 112 | walk(statement) 113 | } 114 | } 115 | 116 | func preVisit(node: ASTNode) { 117 | print("(\(type(of: node))", withNewline: false) 118 | indentation += 1 119 | bodyPrinted.append(false) 120 | } 121 | 122 | func postVisit(node: ASTNode) { 123 | indentation -= 1 124 | if bodyPrinted[bodyPrinted.count - 1] { 125 | print(")") 126 | } else { 127 | print(")", withIndentation: false) 128 | } 129 | bodyPrinted.removeLast() 130 | } 131 | } 132 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/ASTWalker.swift: -------------------------------------------------------------------------------- 1 | protocol ASTWalker { 2 | associatedtype Result 3 | 4 | @discardableResult 5 | func walk(_ node: ASTNode) -> Result 6 | 7 | func visit(ifStatement: IfStatement) -> Result 8 | func visit(braceStatement: BraceStatement) -> Result 9 | func visit(returnStatement: ReturnStatement) -> Result 10 | func visit(binaryOperatorExpression: BinaryOperatorExpression) -> Result 11 | func visit(integerLiteralExpression: IntegerLiteralExpression) -> Result 12 | func visit(stringLiteralExpression: StringLiteralExpression) -> Result 13 | func visit(identifierReferenceExpression: IdentifierReferenceExpression) -> Result 14 | func visit(functionCallExpression: FunctionCallExpression) -> Result 15 | func visit(functionDeclaration: FunctionDeclaration) -> Result 16 | func visit(variableDeclaration: VariableDeclaration) -> Result 17 | func visit(astRoot: ASTRoot) -> Result 18 | 19 | func preVisit(node: ASTNode) 20 | func postVisit(node: ASTNode) 21 | } 22 | 23 | extension ASTWalker { 24 | 25 | func walk(_ node: ASTNode) -> Result { 26 | defer { 27 | postVisit(node: node) 28 | } 29 | preVisit(node: node) 30 | if let casted = node as? IfStatement { 31 | return visit(ifStatement: casted) 32 | } else if let casted = node as? BraceStatement { 33 | return visit(braceStatement: casted) 34 | } else if let casted = node as? ReturnStatement { 35 | return visit(returnStatement: casted) 36 | } else if let casted = node as? BinaryOperatorExpression { 37 | return visit(binaryOperatorExpression: casted) 38 | } else if let casted = node as? IntegerLiteralExpression { 39 | return visit(integerLiteralExpression: casted) 40 | } else if let casted = node as? StringLiteralExpression { 41 | return visit(stringLiteralExpression: casted) 42 | } else if let casted = node as? IdentifierReferenceExpression { 43 | return visit(identifierReferenceExpression: casted) 44 | } else if let casted = node as? FunctionCallExpression { 45 | return visit(functionCallExpression: casted) 46 | } else if let casted = node as? VariableDeclaration { 47 | return visit(variableDeclaration: casted) 48 | } else if let casted = node as? FunctionDeclaration { 49 | return visit(functionDeclaration: casted) 50 | } else if let casted = node as? ASTRoot { 51 | return visit(astRoot: casted) 52 | } else { 53 | fatalError("Unknown AST node \(type(of: node))") 54 | } 55 | } 56 | 57 | func preVisit(node: ASTNode) { 58 | } 59 | 60 | func postVisit(node: ASTNode) { 61 | } 62 | } 63 | 64 | protocol ThrowingASTWalker { 65 | associatedtype Result 66 | 67 | func visit(ifStatement: IfStatement) throws -> Result 68 | func visit(braceStatement: BraceStatement) throws -> Result 69 | func visit(returnStatement: ReturnStatement) throws -> Result 70 | func visit(binaryOperatorExpression: BinaryOperatorExpression) throws -> Result 71 | func visit(integerLiteralExpression: IntegerLiteralExpression) throws -> Result 72 | func visit(stringLiteralExpression: StringLiteralExpression) throws -> Result 73 | func visit(identifierReferenceExpression: IdentifierReferenceExpression) throws -> Result 74 | func visit(functionCallExpression: FunctionCallExpression) throws -> Result 75 | func visit(functionDeclaration: FunctionDeclaration) throws -> Result 76 | func visit(variableDeclaration: VariableDeclaration) throws -> Result 77 | func visit(astRoot: ASTRoot) throws -> Result 78 | 79 | func preVisit(node: ASTNode) 80 | func postVisit(node: ASTNode) 81 | } 82 | 83 | extension ThrowingASTWalker { 84 | 85 | @discardableResult 86 | func walk(_ node: ASTNode) throws -> Result { 87 | defer { 88 | postVisit(node: node) 89 | } 90 | preVisit(node: node) 91 | if let casted = node as? IfStatement { 92 | return try visit(ifStatement: casted) 93 | } else if let casted = node as? BraceStatement { 94 | return try visit(braceStatement: casted) 95 | } else if let casted = node as? ReturnStatement { 96 | return try visit(returnStatement: casted) 97 | } else if let casted = node as? BinaryOperatorExpression { 98 | return try visit(binaryOperatorExpression: casted) 99 | } else if let casted = node as? IntegerLiteralExpression { 100 | return try visit(integerLiteralExpression: casted) 101 | } else if let casted = node as? StringLiteralExpression { 102 | return try visit(stringLiteralExpression: casted) 103 | } else if let casted = node as? IdentifierReferenceExpression { 104 | return try visit(identifierReferenceExpression: casted) 105 | } else if let casted = node as? FunctionCallExpression { 106 | return try visit(functionCallExpression: casted) 107 | } else if let casted = node as? VariableDeclaration { 108 | return try visit(variableDeclaration: casted) 109 | } else if let casted = node as? FunctionDeclaration { 110 | return try visit(functionDeclaration: casted) 111 | } else if let casted = node as? ASTRoot { 112 | return try visit(astRoot: casted) 113 | } else { 114 | fatalError("Unknown AST node \(type(of: node))") 115 | } 116 | } 117 | 118 | func preVisit(node: ASTNode) { 119 | } 120 | 121 | func postVisit(node: ASTNode) { 122 | } 123 | } 124 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/Common.swift: -------------------------------------------------------------------------------- 1 | import Cocoa 2 | 3 | /// A location in the source code 4 | public struct SourceLoc: CustomDebugStringConvertible { 5 | /// The line in the source code (starting at 1) 6 | public internal(set) var line: Int 7 | /// The column in the source code (starting at 1) 8 | public internal(set) var column: Int 9 | /// The toal offset of the location by just counting the characters in the source string 10 | /// (starting at 0) 11 | public internal(set) var offset: Int 12 | 13 | public static let empty = SourceLoc(line: 0, column: 0, offset: 0) 14 | 15 | public var debugDescription: String { 16 | return "\(line):\(column)" 17 | } 18 | } 19 | 20 | /// A range in the source code 21 | public struct SourceRange: CustomDebugStringConvertible { 22 | /// The start of the range (inclusive) 23 | public let start: SourceLoc 24 | /// The end of the range (exclusive) 25 | public let end: SourceLoc 26 | 27 | public static let empty = SourceRange(start: SourceLoc.empty, end: SourceLoc.empty) 28 | 29 | public var debugDescription: String { 30 | return "\(start.debugDescription)-\(end.debugDescription)" 31 | } 32 | } 33 | 34 | /// Error thrown when a compilation error occurs 35 | public struct CompilationError: Error, CustomDebugStringConvertible, CustomPlaygroundDisplayConvertible { 36 | /// The location at which this error occurred 37 | public let location: SourceLoc 38 | /// The error message to be displayed to the user 39 | public let errorMessage: String 40 | 41 | public var debugDescription: String { 42 | return "\(location): \(errorMessage)" 43 | } 44 | 45 | public var playgroundDescription: Any { 46 | return debugDescription 47 | } 48 | 49 | public init(sourceRange: SourceRange, errorMessage: String) { 50 | self.location = sourceRange.start 51 | self.errorMessage = errorMessage 52 | } 53 | 54 | public init(location: SourceLoc, errorMessage: String) { 55 | self.location = location 56 | self.errorMessage = errorMessage 57 | } 58 | } 59 | 60 | public extension String { 61 | var monospacedString: NSAttributedString { 62 | let attrs = [ 63 | NSAttributedString.Key.font: NSFont(name: "Menlo", size: 11)! 64 | ] 65 | return NSAttributedString(string: self, attributes: attrs) 66 | } 67 | } 68 | 69 | public extension NSAttributedString { 70 | var withPlaygroundQuickLookBackgroundColor: NSAttributedString { 71 | let s = NSMutableAttributedString(attributedString: self) 72 | s.addAttribute(.backgroundColor, value: NSColor(white: 247.0/255, alpha: 1), range: NSRange(location: 0, length: s.length)) 73 | return s 74 | } 75 | } 76 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/Compiler.swift: -------------------------------------------------------------------------------- 1 | /// Driver for the compiler phases implemented in this playground that invokes 2 | /// the different compiler phases described on the different playground pages 3 | public class Compiler { 4 | /// Compile the given Swift file to Intermediate Representation without optimising it 5 | /// 6 | /// - Parameter file: The Swift file to compile 7 | /// - Returns: The compiled IR 8 | /// - Throws: A compilation error if compilation failed 9 | public static func compile(swiftFile file: SwiftFile) throws -> IR { 10 | let parser = Parser() 11 | let ast = try parser.parse(sourceFile: file) 12 | 13 | let typechecker = Typechecker() 14 | try typechecker.typecheck(node: ast) 15 | 16 | return IRGen.generateIR(forAST: ast) 17 | } 18 | } 19 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/IR.swift: -------------------------------------------------------------------------------- 1 | import Cocoa 2 | 3 | /// A register whose value can be loaded and to whcih values can be stored in the IR 4 | public struct Register: Hashable, CustomStringConvertible { 5 | public let name: Int 6 | 7 | init(name: Int) { 8 | self.name = name 9 | } 10 | 11 | public static func ==(lhs: Register, rhs: Register) -> Bool { 12 | return lhs.name == rhs.name 13 | } 14 | 15 | public func hash(into hasher: inout Hasher) { 16 | hasher.combine(name) 17 | } 18 | 19 | public var description: String { 20 | return "%\(name)" 21 | } 22 | } 23 | 24 | /// The name uniquely identifying a basic block in the IR. This name will be used to jump to a block 25 | /// using `jump` or `branch` 26 | public struct BlockName: Hashable, CustomStringConvertible { 27 | public let name: Int 28 | 29 | public init(name: Int) { 30 | self.name = name 31 | } 32 | 33 | public static func ==(lhs: BlockName, rhs: BlockName) -> Bool { 34 | return lhs.name == rhs.name 35 | } 36 | 37 | public func hash(into hasher: inout Hasher) { 38 | hasher.combine(name) 39 | } 40 | 41 | public var description: String { 42 | return "b\(name)" 43 | } 44 | } 45 | 46 | /// The different kinds of values, arguments in the IR can take 47 | /// 48 | /// - register: Retrieve the value from a specified register 49 | /// - integer: A constant register argument 50 | /// - boolean: A constant boolean argument 51 | /// - string: A constant string argument 52 | public enum IRValue: CustomStringConvertible { 53 | case register(Register) 54 | case integer(Int) 55 | case boolean(Bool) 56 | case string(String) 57 | 58 | public var description: String { 59 | switch self { 60 | case .register(let register): 61 | return register.description 62 | case .integer(let value): 63 | return "\(value)" 64 | case .boolean(let value): 65 | return "\(value)" 66 | case .string(let value): 67 | return "\"\(value)\"" 68 | } 69 | } 70 | } 71 | 72 | /// Instructions in the IR 73 | public enum IRInstruction: CustomDebugStringConvertible { 74 | /// Add `lhs` and `rhs` and store the value to `destination` 75 | case add(lhs: IRValue, rhs: IRValue, destination: Register) 76 | /// Subtract `rhs` from `lhs` and store the value to `destination` 77 | case sub(lhs: IRValue, rhs: IRValue, destination: Register) 78 | /// Set `destination` to `true` if `lhs == rhs` and to `false` otherwise 79 | case equal(lhs: IRValue, rhs: IRValue, destination: Register) 80 | /// Set `destination` to `true` if `lhs <= rhs` and to `false` otherwise 81 | case lessOrEqual(lhs: IRValue, rhs: IRValue, destination: Register) 82 | /// Jump to `trueBlock` if `check` is `true` and to `falseBlock` otherwise 83 | case branch(check: IRValue, trueBlock: BlockName, falseBlock: BlockName) 84 | /// Jump to `toBlock` 85 | case jump(toBlock: BlockName) 86 | /// Call function with name `functionName` with `argument` and store the result in `destination` 87 | case call(functionName: String, arguments: [IRValue], destination: Register) 88 | /// Load the constant `value` into `destination` 89 | case load(value: IRValue, destination: Register) 90 | /// Finish execution of the program 91 | case `return`(returnValue: IRValue) 92 | 93 | public var debugDescription: String { 94 | switch self { 95 | case let .add(lhs, rhs, destination): 96 | return "add \(lhs), \(rhs) -> \(destination)" 97 | case let .sub(lhs, rhs, destination): 98 | return "sub \(lhs), \(rhs) -> \(destination)" 99 | case let .equal(lhs, rhs, destination): 100 | return "equal \(lhs), \(rhs) -> \(destination)" 101 | case let .lessOrEqual(lhs, rhs, destination): 102 | return "lessOrEqual \(lhs), \(rhs) -> \(destination)" 103 | case let .branch(check, trueBlock, falseBlock): 104 | return "branch \(check), true: \(trueBlock), false: \(falseBlock)" 105 | case let .jump(toBlock): 106 | return "jump \(toBlock)" 107 | case let .call(text, arguments, destination): 108 | let args = arguments.map({ $0.description }).joined(separator: ", ") 109 | return "call \(text)(\(args)) -> \(destination)" 110 | case let .load(value, destination): 111 | return "load \(value) -> \(destination)" 112 | case .return(let value): 113 | return "return \(value)" 114 | } 115 | } 116 | } 117 | 118 | /// A program in the compiler's IR, consisting of the IR's basic blocks and the block in which to 119 | /// start execution 120 | public struct IRFunction: CustomDebugStringConvertible, CustomPlaygroundDisplayConvertible { 121 | public let startBlock: BlockName 122 | public let blocks: [BlockName: [IRInstruction]] 123 | public let argumentRegisters: [Register] 124 | 125 | public var debugDescription: String { 126 | var result = "Start block: \(startBlock)\n\n" 127 | 128 | for blockName in blocks.keys.sorted(by: { $0.name < $1.name }) { 129 | let instructions = blocks[blockName]! 130 | result += "\(blockName):\n" 131 | for instruction in instructions { 132 | result += " " + instruction.debugDescription + "\n" 133 | } 134 | } 135 | return result 136 | } 137 | 138 | public var playgroundDescription: Any { 139 | return self.debugDescription.monospacedString.withPlaygroundQuickLookBackgroundColor 140 | } 141 | } 142 | 143 | public struct IR: CustomDebugStringConvertible, CustomPlaygroundDisplayConvertible { 144 | public let functions: [String: IRFunction] 145 | 146 | public var debugDescription: String { 147 | var result = "" 148 | for functionName in functions.keys { 149 | let function = functions[functionName]! 150 | result += functionName + "(" 151 | result += function.argumentRegisters.map({ $0.description }).joined(separator: ", ") 152 | result += "): \n" 153 | result += function.debugDescription.components(separatedBy: "\n").map({ " " + $0 }).joined(separator: "\n") 154 | result += "\n" 155 | } 156 | return result 157 | } 158 | 159 | public var playgroundDescription: Any { 160 | return self.debugDescription.monospacedString.withPlaygroundQuickLookBackgroundColor 161 | } 162 | } 163 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/IRDebugger.swift: -------------------------------------------------------------------------------- 1 | public struct DebuggerState { 2 | public let callStack: [StackFrame] 3 | public let currentFunctionName: String 4 | public let currentBlock: BlockName 5 | public let currentInstructionIndex: Int 6 | public let output: String 7 | } 8 | 9 | public class IRDebugger: IRExecutor { 10 | 11 | /// The output the program has produced so far 12 | private var output = "" 13 | 14 | public var debuggerState: DebuggerState? { 15 | if callStack.isEmpty { 16 | return nil 17 | } else { 18 | var adjustedCallStack: [StackFrame] = [] 19 | adjustedCallStack.append(callStack[0]) 20 | for i in 1.. 1 { 103 | callStack[1].registers[callStack[1].functionCallDestinationRegister!] = .integer(evaluate(irValue: returnValue)) 104 | } 105 | callStack.remove(at: 0) 106 | } 107 | } 108 | 109 | /// Evaluate the given IR value as an integer. This looks up values in registers and 110 | /// short-circuits literal values. 111 | /// 112 | /// Crashes if the type is not convertible to an integer. 113 | /// 114 | /// - Parameter irValue: The value 115 | /// - Returns: The `Int` value of the given `IRValue` 116 | private func evaluate(irValue: IRValue) -> Int { 117 | switch irValue { 118 | case .register(let register): 119 | if let registerValue = callStack[0].registers[register] { 120 | return evaluate(irValue: registerValue) 121 | } else { 122 | return 0 123 | } 124 | case .integer(let value): 125 | return value 126 | case .boolean(let value): 127 | return value ? 1 : 0 128 | default: 129 | fatalError("Cannot convert \(irValue) to int") 130 | } 131 | } 132 | 133 | /// Evaluate the given IR value as an string. This looks up values in registers and 134 | /// short-circuits literal values. 135 | /// 136 | /// - Parameter irValue: The value 137 | /// - Returns: The `String` value of the given `IRValue` 138 | private func evaluate(irValue: IRValue) -> String { 139 | switch irValue { 140 | case .register(let register): 141 | if let registerValue = callStack[0].registers[register] { 142 | return evaluate(irValue: registerValue) 143 | } else { 144 | return "" 145 | } 146 | case .integer(let value): 147 | return "\(value)" 148 | case .boolean(let value): 149 | return value ? "true" : "false" 150 | case .string(let string): 151 | return string 152 | } 153 | } 154 | 155 | private func jump(toBlock: BlockName) { 156 | callStack[0].block = toBlock 157 | callStack[0].instructionIndex = 0 158 | } 159 | 160 | private func increaseProgramCounter() { 161 | callStack[0].instructionIndex += 1 162 | } 163 | } 164 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/IRFunctionsGen.swift: -------------------------------------------------------------------------------- 1 | public class IRFunctionGen: ASTWalker { 2 | typealias Result = IRValue? 3 | 4 | /// The lowest id so that no register with this ID has been created yet 5 | private var _nextFreeRegisterId = 1 6 | 7 | /// Returns a register that hasn't been used yet, incrementing the `_nextFreeRegisterId` counter 8 | /// 9 | /// - Returns: A register that hasn't been used yet 10 | private func nextFreeRegister() -> Register { 11 | defer { _nextFreeRegisterId += 1 } 12 | return Register(name: _nextFreeRegisterId) 13 | } 14 | 15 | /// The name of the block that is currently beeing generated 16 | private var currentBlockName = BlockName(name: 0) 17 | 18 | /// The lowest name so that no block with this name has been created yet 19 | private var _nextFreeBlockName = 1 20 | 21 | /// Returns a block name that hasn't been used yet, incrementing the `_nextFreeBlockName` counter 22 | /// 23 | /// - Returns: A block that hasn't been used yet 24 | private func nextFreeBlock() -> BlockName { 25 | defer { _nextFreeBlockName += 1 } 26 | return BlockName(name: _nextFreeBlockName) 27 | } 28 | 29 | /// The block that is currently being created 30 | private var currentBlock: [IRInstruction] = [] 31 | 32 | /// A collection of blocks that have been finished generating 33 | private var finishedBlocks: [BlockName: [IRInstruction]] = [:] 34 | 35 | private var variables: [VariableDeclaration: Register] = [:] 36 | 37 | private var argumentRegisters: [Register] = [] 38 | 39 | // MARK: - Public Methods 40 | 41 | public init() { 42 | } 43 | 44 | /// Generate the IR for the given AST 45 | /// 46 | /// - Parameter astRoot: The AST for which to create IR code 47 | /// - Returns: The generated IR 48 | public static func generateIR(forAST astNode: ASTNode) -> IRFunction { 49 | return IRFunctionGen().generateIR(forAST: astNode) 50 | } 51 | 52 | /// Generate the IR for the given AST 53 | /// 54 | /// - Parameter astRoot: The AST for which to create IR code 55 | /// - Returns: The generated IR 56 | public func generateIR(forAST astNode: ASTNode) -> IRFunction { 57 | let mainBlock = currentBlockName 58 | _ = walk(astNode) 59 | 60 | finishedBlocks[currentBlockName] = currentBlock 61 | 62 | return IRFunction(startBlock: mainBlock, blocks: finishedBlocks, argumentRegisters: argumentRegisters) 63 | } 64 | 65 | // MARK: Private helper methods 66 | 67 | /// Add the given instruction to the current block 68 | /// 69 | /// - Parameter instruction: The instruction to add to the current block 70 | private func emit(instruction: IRInstruction) { 71 | currentBlock.append(instruction) 72 | } 73 | 74 | /// Add the current block to the list of finished blocks by assigning it the given name and 75 | /// clear the `currentBlock` buffer. 76 | /// 77 | /// - Parameter withName: The name to assign the current block 78 | private func finishCurrentBlock(withName: BlockName) { 79 | finishedBlocks[withName] = currentBlock 80 | currentBlock = [] 81 | } 82 | 83 | // MARK: ASTWalker 84 | 85 | func visit(ifStatement: IfStatement) -> Result { 86 | let trueBlock = nextFreeBlock() 87 | let falseBlock = nextFreeBlock() 88 | // Short circuit the falsBlock to the restBlock if the if statement has no else body 89 | let restBlock = ifStatement.elseBody != nil ? nextFreeBlock() : falseBlock 90 | 91 | let conditionValue = walk(ifStatement.condition) 92 | emit(instruction: .branch(check: conditionValue!, 93 | trueBlock: trueBlock, 94 | falseBlock: falseBlock)) 95 | finishCurrentBlock(withName: currentBlockName) 96 | 97 | _ = walk(ifStatement.body) 98 | emit(instruction: .jump(toBlock: restBlock)) 99 | finishCurrentBlock(withName: trueBlock) 100 | 101 | if let elseBody = ifStatement.elseBody { 102 | _ = walk(elseBody) 103 | emit(instruction: .jump(toBlock: restBlock)) 104 | finishCurrentBlock(withName: falseBlock) 105 | } 106 | 107 | currentBlockName = restBlock 108 | 109 | return nil 110 | } 111 | 112 | func visit(braceStatement: BraceStatement) -> Result { 113 | for statement in braceStatement.body { 114 | _ = walk(statement) 115 | } 116 | return nil 117 | } 118 | 119 | func visit(returnStatement: ReturnStatement) -> Result { 120 | let returnValue = walk(returnStatement.expression)! 121 | currentBlock.append(.return(returnValue: returnValue)) 122 | return nil 123 | } 124 | 125 | func visit(binaryOperatorExpression: BinaryOperatorExpression) -> Result { 126 | let lhsValue = walk(binaryOperatorExpression.lhs) 127 | let rhsValue = walk(binaryOperatorExpression.rhs) 128 | let destination = nextFreeRegister() 129 | 130 | switch binaryOperatorExpression.operator { 131 | case .add: 132 | emit(instruction: .add(lhs: lhsValue!, rhs: rhsValue!, destination: destination)) 133 | return IRValue.register(destination) 134 | case .sub: 135 | emit(instruction: .sub(lhs: lhsValue!, rhs: rhsValue!, destination: destination)) 136 | return IRValue.register(destination) 137 | case .equal: 138 | emit(instruction: .equal(lhs: lhsValue!, rhs: rhsValue!, destination: destination)) 139 | return IRValue.register(destination) 140 | case .lessOrEqual: 141 | emit(instruction: .lessOrEqual(lhs: lhsValue!, rhs: rhsValue!, destination: destination)) 142 | return IRValue.register(destination) 143 | } 144 | } 145 | 146 | func visit(integerLiteralExpression: IntegerLiteralExpression) -> Result { 147 | return .integer(integerLiteralExpression.value) 148 | } 149 | 150 | func visit(stringLiteralExpression: StringLiteralExpression) -> Result { 151 | let destination = nextFreeRegister() 152 | emit(instruction: .load(value: .string(stringLiteralExpression.value), destination: destination)) 153 | return .register(destination) 154 | } 155 | 156 | func visit(identifierReferenceExpression: IdentifierReferenceExpression) -> Result { 157 | let referencedVariable = identifierReferenceExpression.referencedDeclaration! as! VariableDeclaration 158 | return .register(variables[referencedVariable]!) 159 | } 160 | 161 | func visit(functionCallExpression: FunctionCallExpression) -> Result { 162 | let destination = nextFreeRegister() 163 | let arguments = functionCallExpression.arguments.map({ walk($0)! }) 164 | emit(instruction: .call(functionName: functionCallExpression.functionName, arguments: arguments, destination: destination)) 165 | return .register(destination) 166 | } 167 | 168 | func visit(variableDeclaration: VariableDeclaration) -> Result { 169 | let register = nextFreeRegister() 170 | variables[variableDeclaration] = register 171 | return .register(register) 172 | } 173 | 174 | func visit(functionDeclaration: FunctionDeclaration) -> Result { 175 | for parameter in functionDeclaration.parameters { 176 | guard case .register(let argumentRegister) = walk(parameter)! else { 177 | fatalError() 178 | } 179 | argumentRegisters.append(argumentRegister) 180 | } 181 | _ = walk(functionDeclaration.body) 182 | return nil 183 | } 184 | 185 | 186 | func visit(astRoot: ASTRoot) -> Result { 187 | for statement in astRoot.statements { 188 | if !(statement is FunctionDeclaration) { 189 | _ = walk(statement) 190 | } 191 | } 192 | currentBlock.append(.return(returnValue: .boolean(true))) 193 | return nil 194 | } 195 | 196 | } 197 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/IRGen.swift: -------------------------------------------------------------------------------- 1 | public class IRGen: ASTWalker { 2 | private var irFunctions: [String: IRFunction] = [:] 3 | 4 | typealias Result = Void 5 | 6 | /// Generate the IR for the given AST 7 | /// 8 | /// - Parameter astRoot: The AST for which to create IR code 9 | /// - Returns: The generated IR 10 | public static func generateIR(forAST astNode: ASTNode) -> IR { 11 | return IRGen().generateIR(forAST: astNode) 12 | } 13 | 14 | /// Generate the IR for the given AST 15 | /// 16 | /// - Parameter astRoot: The AST for which to create IR code 17 | /// - Returns: The generated IR 18 | public func generateIR(forAST astNode: ASTNode) -> IR { 19 | walk(astNode) 20 | return IR(functions: irFunctions) 21 | } 22 | 23 | func visit(ifStatement: IfStatement) -> Void { 24 | } 25 | 26 | func visit(braceStatement: BraceStatement) -> Void { 27 | for statement in braceStatement.body { 28 | walk(statement) 29 | } 30 | } 31 | 32 | func visit(returnStatement: ReturnStatement) -> Void { 33 | } 34 | 35 | func visit(binaryOperatorExpression: BinaryOperatorExpression) -> Void { 36 | } 37 | 38 | func visit(integerLiteralExpression: IntegerLiteralExpression) -> Void { 39 | } 40 | 41 | func visit(stringLiteralExpression: StringLiteralExpression) -> Void { 42 | } 43 | 44 | func visit(identifierReferenceExpression: IdentifierReferenceExpression) -> Void { 45 | } 46 | 47 | func visit(functionCallExpression: FunctionCallExpression) -> Void { 48 | } 49 | 50 | func visit(functionDeclaration: FunctionDeclaration) -> Void { 51 | irFunctions[functionDeclaration.name] = IRFunctionGen.generateIR(forAST: functionDeclaration) 52 | walk(functionDeclaration.body) 53 | } 54 | 55 | func visit(variableDeclaration: VariableDeclaration) -> Void { 56 | } 57 | 58 | func visit(astRoot: ASTRoot) -> Void { 59 | irFunctions["main"] = IRFunctionGen.generateIR(forAST: astRoot) 60 | for statement in astRoot.statements { 61 | walk(statement) 62 | } 63 | } 64 | } 65 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/Lexer.swift: -------------------------------------------------------------------------------- 1 | // MARK: - UnicodeScalar extensions 2 | 3 | // UnicodeScalar extension to classify characters 4 | fileprivate extension UnicodeScalar { 5 | var isWhitespace: Bool { 6 | return self == " " || self == "\t" || self == "\n" || self == "\r" 7 | } 8 | 9 | var isAlpha: Bool { 10 | let alphaChars = Set("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ".unicodeScalars) 11 | return alphaChars.contains(self) 12 | } 13 | 14 | var isNumeric: Bool { 15 | let numChars = Set("0123456789".unicodeScalars) 16 | return numChars.contains(self) 17 | } 18 | 19 | var isAlnum: Bool { 20 | return isAlpha || isNumeric 21 | } 22 | 23 | /// Valid identifier characters are alphanumeric or '_' 24 | var isIdentifier: Bool { 25 | return isAlnum || self == "_" 26 | } 27 | 28 | /// If the character represents a character that can occur in operators 29 | /// Currently includes '+', '-', '*', '/', '=' 30 | var isOperator: Bool { 31 | let operatorChars = Set("+-*/=<>".unicodeScalars) 32 | return operatorChars.contains(self) 33 | } 34 | } 35 | 36 | // MARK: - Token 37 | 38 | /// The diffent kinds a token generated by the lexer can have 39 | /// - integer: 40 | /// - identifier: An identifier 41 | /// - `operator`: A operator 42 | /// - leftBrace: '{' 43 | /// - rightBrace: '}' 44 | /// - leftParen: '(' 45 | /// - rightParen: ')' 46 | /// - stringLiteral: A string literal 47 | /// - endOfFile: Artificial token to represent the end of a file 48 | public enum TokenKind: CustomStringConvertible { 49 | /// The 'if' keyword 50 | case `if` 51 | /// The 'else' keyword 52 | case `else` 53 | /// The 'func' keyword 54 | case `func` 55 | /// The 'return' keyword 56 | case `return` 57 | /// An integer literal 58 | case integer(value: Int) 59 | /// An identifier 60 | case identifier(name: String) 61 | /// An operator 62 | case `operator`(name: String) 63 | /// `{` 64 | case leftBrace 65 | /// `}` 66 | case rightBrace 67 | /// `(` 68 | case leftParen 69 | /// `)` 70 | case rightParen 71 | /// `:` 72 | case colon 73 | /// `->` 74 | case arrow 75 | /// ',' 76 | case comma 77 | /// '_' 78 | case underscore 79 | /// A string literal 80 | case stringLiteral(value: String) 81 | /// End of file marker 82 | case endOfFile 83 | 84 | public var description: String { 85 | switch self { 86 | case .if: 87 | return "Keyword if" 88 | case .else: 89 | return "Keyword else" 90 | case .func: 91 | return "Keyword func" 92 | case .return: 93 | return "Keyword return" 94 | case .integer(let value): 95 | return "Integer \(value)" 96 | case .identifier(let name): 97 | return "Identifier \(name)" 98 | case .operator(let name): 99 | return "Operator \(name)" 100 | case .leftBrace: 101 | return "Left brace '{'" 102 | case .rightBrace: 103 | return "Right brace '}'" 104 | case .leftParen: 105 | return "Left paranthesis '('" 106 | case .rightParen: 107 | return "Right paranthesis ')'" 108 | case .colon: 109 | return "Colon ':'" 110 | case .arrow: 111 | return "Arrow '->'" 112 | case .comma: 113 | return "Comma ','" 114 | case .underscore: 115 | return "Underscore '_'" 116 | case .stringLiteral(let value): 117 | return "String literal \"\(value)\"" 118 | case .endOfFile: 119 | return "End of file" 120 | } 121 | } 122 | 123 | public var sourceCodeRepresentation: String { 124 | switch self { 125 | case .if: 126 | return "if" 127 | case .else: 128 | return "else" 129 | case .func: 130 | return "func" 131 | case .return: 132 | return "return" 133 | case .integer(let value): 134 | return "\(value)" 135 | case .identifier(let name): 136 | return "\(name)" 137 | case .operator(let name): 138 | return "\(name)" 139 | case .leftBrace: 140 | return "{" 141 | case .rightBrace: 142 | return "}" 143 | case .leftParen: 144 | return "(" 145 | case .rightParen: 146 | return ")" 147 | case .colon: 148 | return ":" 149 | case .arrow: 150 | return "->" 151 | case .comma: 152 | return "," 153 | case .underscore: 154 | return "_" 155 | case .stringLiteral(let value): 156 | return "\"\(value)\"" 157 | case .endOfFile: 158 | return "End of file" 159 | } 160 | } 161 | } 162 | 163 | extension TokenKind: Equatable { 164 | public static func ==(lhs: TokenKind, rhs: TokenKind) -> Bool { 165 | switch (lhs, rhs) { 166 | case (.if, .if): 167 | return true 168 | case (.else, .else): 169 | return true 170 | case (.func, .func): 171 | return true 172 | case (.return, .return): 173 | return true 174 | case (.integer(let lhsValue), .integer(let rhsValue)) where lhsValue == rhsValue: 175 | return true 176 | case (.identifier(let lhsName), .identifier(let rhsName)) where lhsName == rhsName: 177 | return true 178 | case (.operator(let lhsName), .operator(let rhsName)) where lhsName == rhsName: 179 | return true 180 | case (.leftBrace, .leftBrace): 181 | return true 182 | case (.rightBrace, .rightBrace): 183 | return true 184 | case (.leftParen, .leftParen): 185 | return true 186 | case (.rightParen, .rightParen): 187 | return true 188 | case (.colon, .colon): 189 | return true 190 | case (.arrow, .arrow): 191 | return true 192 | case (.comma, .comma): 193 | return true 194 | case (.underscore, .underscore): 195 | return true 196 | case (.stringLiteral(let lhsValue), .stringLiteral(let rhsValue)) where lhsValue == rhsValue: 197 | return true 198 | case (.endOfFile, .endOfFile): 199 | return true 200 | default: 201 | return false 202 | } 203 | } 204 | } 205 | 206 | public func ==(token: Token, tokenKind: TokenKind) -> Bool { 207 | return token.payload == tokenKind 208 | } 209 | 210 | public func !=(token: Token, tokenKind: TokenKind) -> Bool { 211 | return !(token == tokenKind) 212 | } 213 | 214 | /// A token generated by the lexer 215 | public struct Token { 216 | /// The actual payload of the token 217 | public let payload: TokenKind 218 | 219 | /// The source range where this token appeared 220 | public let sourceRange: SourceRange 221 | 222 | init(_ payload: TokenKind, sourceRange: SourceRange) { 223 | self.payload = payload 224 | self.sourceRange = sourceRange 225 | } 226 | } 227 | 228 | // MARK: - Lexer 229 | 230 | public class Lexer { 231 | /// The scanner is responsible to return characters in the source code one by one and maintain 232 | /// the source location of these characters 233 | private let scanner: Scanner 234 | 235 | /// Create a new lexer to lex the given source code 236 | /// 237 | /// - Parameter sourceCode: The source code to lex 238 | public init(sourceCode: String) { 239 | self.scanner = Scanner(sourceCode: sourceCode) 240 | } 241 | 242 | /// Lex the next token in the source code and return it 243 | /// 244 | /// - Returns: The next token in the source code 245 | /// - Throws: A CompilationError if the next token could not be lexed 246 | public func nextToken() throws -> Token { 247 | while let char = scanner.currentChar, char.isWhitespace { 248 | scanner.consumeChar() 249 | } 250 | 251 | let directCharacterMapping: [UnicodeScalar: TokenKind] = [ 252 | "{": .leftBrace, 253 | "}": .rightBrace, 254 | "(": .leftParen, 255 | ")": .rightParen, 256 | ":": .colon, 257 | ",": .comma, 258 | "_": .underscore, 259 | ] 260 | 261 | switch scanner.currentChar { 262 | case let .some(char) where char.isAlpha: 263 | return lexIdentifier() 264 | case let .some(char) where char.isNumeric: 265 | return lexIntegerLiteral() 266 | case let .some(char) where char.isOperator: 267 | return lexOperator() 268 | case let .some(char) where directCharacterMapping.keys.contains(char): 269 | let startLoc = scanner.sourceLoc 270 | scanner.consumeChar() 271 | return Token(directCharacterMapping[char]!, sourceRange: range(startingAt: startLoc)) 272 | case .some("\""): 273 | return try lexStringLiteral() 274 | case nil: // End of file 275 | return Token(.endOfFile, sourceRange: range(startingAt: scanner.sourceLoc)) 276 | default: 277 | defer { 278 | scanner.consumeChar() 279 | } 280 | throw CompilationError(location: scanner.sourceLoc, 281 | errorMessage: "Invalid character: '\(scanner.currentChar!)'") 282 | } 283 | } 284 | 285 | /// Helper method to create a source range starting at the given location and ending at the 286 | /// next character to be parsed 287 | /// 288 | /// - Parameter startingAt: The lcoation where the source range shall tart 289 | /// - Returns: A source range from the given location to the current scanner position 290 | private func range(startingAt: SourceLoc) -> SourceRange { 291 | return SourceRange(start: startingAt, end: scanner.sourceLoc) 292 | } 293 | 294 | /// Keep consuming characters while they satisfy the given condition and return the string made 295 | /// up from these characters 296 | /// 297 | /// - Parameter condition: Gather characters that satisfy this condition 298 | /// - Returns: The string made up from the characters that satisfy the given condition 299 | private func gatherWhile(_ condition: (UnicodeScalar) -> Bool) -> String { 300 | var buildupString = "" 301 | while let char = scanner.currentChar, condition(char) { 302 | buildupString.append(String(char)) 303 | scanner.consumeChar() 304 | } 305 | return buildupString 306 | } 307 | 308 | private func lexIdentifier() -> Token { 309 | let startLoc = scanner.sourceLoc 310 | let name = gatherWhile({ $0.isIdentifier }) 311 | let tokenKind: TokenKind 312 | switch name { 313 | case "if": 314 | tokenKind = .if 315 | case "else": 316 | tokenKind = .else 317 | case "func": 318 | tokenKind = .func 319 | case "return": 320 | tokenKind = .return 321 | default: 322 | tokenKind = .identifier(name: name) 323 | } 324 | return Token(tokenKind, sourceRange: range(startingAt: startLoc)) 325 | } 326 | 327 | private func lexIntegerLiteral() -> Token { 328 | let startLoc = scanner.sourceLoc 329 | let value = gatherWhile({ $0.isNumeric }) 330 | return Token(.integer(value: Int(value)!), sourceRange: range(startingAt: startLoc)) 331 | } 332 | 333 | private func lexOperator() -> Token { 334 | let startLoc = scanner.sourceLoc 335 | let name = gatherWhile({ $0.isOperator }) 336 | let tokenKind: TokenKind 337 | switch name { 338 | case "->": 339 | tokenKind = .arrow 340 | default: 341 | tokenKind = .operator(name: name) 342 | } 343 | return Token(tokenKind, sourceRange: range(startingAt: startLoc)) 344 | } 345 | 346 | private func lexStringLiteral() throws -> Token { 347 | let startLoc = scanner.sourceLoc 348 | 349 | precondition(scanner.currentChar == "\"") 350 | scanner.consumeChar() 351 | var escapedMode = false 352 | 353 | var buildupString = "" 354 | 355 | let escapedCharacters: [UnicodeScalar: String] = [ 356 | "t": "\t", 357 | "r": "\r", 358 | "n": "\n", 359 | "\\": "\\" 360 | ] 361 | 362 | while (true) { 363 | let sourceLoc = scanner.sourceLoc 364 | let nextChar = scanner.currentChar 365 | scanner.consumeChar() 366 | 367 | switch nextChar { 368 | case .some("\\") where !escapedMode: 369 | escapedMode = true 370 | case .some("\"") where escapedMode: 371 | buildupString += "\"" 372 | escapedMode = false 373 | case .some("\"") where !escapedMode: 374 | return Token(.stringLiteral(value: buildupString), sourceRange: range(startingAt: startLoc)) 375 | case let .some(char) where escapedMode: 376 | guard let unescaped = escapedCharacters[char] else { 377 | throw CompilationError(location: sourceLoc, errorMessage: "Unknown escape sequence: '\\\(char)'") 378 | } 379 | buildupString += unescaped 380 | escapedMode = false 381 | case let .some(char) where !escapedMode: 382 | buildupString += String(char) 383 | case nil: 384 | throw CompilationError(location: sourceLoc, errorMessage: "Found end of file while scanning for string literal") 385 | default: 386 | fatalError() 387 | } 388 | } 389 | } 390 | } 391 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/Optimiser.swift: -------------------------------------------------------------------------------- 1 | /// Enum containing all the possible optimisation options that can be applied 2 | /// by the optimiser 3 | public struct OptimisationOptions: OptionSet { 4 | public let rawValue: Int 5 | 6 | public init(rawValue: Int) { 7 | self.rawValue = rawValue 8 | } 9 | 10 | /// Evaluate constant expressions. 11 | /// For example 12 | /// ``` 13 | /// add 1, 2 -> %1 14 | /// ``` 15 | /// becomes 16 | /// ``` 17 | /// load 3 -> %1 18 | /// ``` 19 | public static let constantExprssionEvaluation = OptimisationOptions(rawValue: 1 << 0) 20 | /// Propagate constants into instructions. For example replace 21 | /// 22 | /// ``` 23 | /// load 1 -> %1 24 | /// add %1, 2 -> %2 25 | /// ``` 26 | /// by 27 | /// ``` 28 | /// load 1 -> %1 29 | /// add 1, 2 -> %2 30 | /// ``` 31 | public static let constantPropagation = OptimisationOptions(rawValue: 1 << 1) 32 | /// Remove instructions that store to registers that are never read 33 | public static let deadStoreElimination = OptimisationOptions(rawValue: 1 << 2) 34 | /// Remove instructions after `jump`, `return`, or `branch` instructions 35 | public static let deadCodeElimination = OptimisationOptions(rawValue: 1 << 3) 36 | /// Remove block that only contain a `jump` instruction by redirecting jumps or branches to that 37 | /// block to the `jump`'s target. 38 | /// 39 | /// For example: 40 | /// ``` 41 | /// Start block: b0 42 | /// b0: 43 | /// jump b1 44 | /// b1: 45 | /// ... 46 | /// ``` 47 | /// gets replace by 48 | /// ``` 49 | /// Start block: b1 50 | /// b1: 51 | /// ... 52 | /// ``` 53 | public static let emptyBlockElimination = OptimisationOptions(rawValue: 1 << 4) 54 | /// Replace `jump` instructions with the block the `jump` jumps to. 55 | /// 56 | /// For example: 57 | /// ``` 58 | /// b0: 59 | /// ... 60 | /// jump b1 61 | /// b1: 62 | /// load "abc" -> %1 63 | /// call print(%1) -> %2 64 | /// return true 65 | /// ``` 66 | /// gets replace by 67 | /// ``` 68 | /// b0: 69 | /// ... 70 | /// load "abc" -> %1 71 | /// call print(%1) -> %2 72 | /// return true 73 | /// b1: 74 | /// load "abc" -> %1 75 | /// call print(%1) -> %2 76 | /// return true 77 | /// ``` 78 | public static let inlineJumpTargets = OptimisationOptions(rawValue: 1 << 5) 79 | /// Remove blocks that are never jumped or branched to 80 | public static let deadBlockElimination = OptimisationOptions(rawValue: 1 << 6) 81 | 82 | public static let all: OptimisationOptions = [.constantExprssionEvaluation, 83 | .constantPropagation, 84 | .deadStoreElimination, 85 | .deadCodeElimination, 86 | .emptyBlockElimination, 87 | .inlineJumpTargets, 88 | .deadBlockElimination] 89 | } 90 | 91 | /// The optimiser translates IR into a different IR with the same semantics by 92 | /// applying transformations that will speed up execution. 93 | public class Optimiser { 94 | 95 | private let options: OptimisationOptions 96 | 97 | // MARK: - Public interface 98 | 99 | /// - Parameter options: The optimisation options to apply 100 | public init(options: OptimisationOptions) { 101 | self.options = options 102 | } 103 | 104 | public static func optimise(irFunction ir: IRFunction, withOptions options: OptimisationOptions) -> IRFunction { 105 | let optimiser = Optimiser(options: options) 106 | return optimiser.optimise(irFunction: ir) 107 | } 108 | 109 | public func optimise(ir: IR) -> IR { 110 | var optimisedFunctions: [String: IRFunction] = [:] 111 | for (name, function) in ir.functions { 112 | optimisedFunctions[name] = self.optimise(irFunction: function) 113 | } 114 | return IR(functions: optimisedFunctions) 115 | } 116 | 117 | /// Optimise the given IR function with the optimisation options this optimiser was 118 | /// initialised with 119 | /// 120 | /// - Parameter ir: The ir function to optimise 121 | /// - Returns: The optimised compilation result 122 | public func optimise(irFunction ir: IRFunction) -> IRFunction { 123 | var optimisedBlocks: [BlockName: [IRInstruction]] = [:] 124 | for (blockName, instructions) in ir.blocks { 125 | optimisedBlocks[blockName] = peepholeOptimise(block: instructions) 126 | } 127 | var optimised = IRFunction(startBlock: ir.startBlock, blocks: optimisedBlocks, argumentRegisters: ir.argumentRegisters) 128 | 129 | if options.contains(.deadCodeElimination) { 130 | optimised = eliminateDeadCode(in: optimised) 131 | } 132 | if options.contains(.deadStoreElimination) { 133 | optimised = eliminateDeadStores(in: optimised) 134 | } 135 | if options.contains(.emptyBlockElimination) { 136 | optimised = eliminateEmptyBlocks(in: optimised) 137 | } 138 | if options.contains(.inlineJumpTargets) { 139 | optimised = inlineJumpTargets(in: optimised) 140 | } 141 | if options.contains(.deadBlockElimination) { 142 | optimised = eliminateDeadBlocks(in: optimised) 143 | } 144 | return optimised 145 | } 146 | 147 | // MARK: - Private 148 | 149 | // MARK: Peephole optimisation 150 | 151 | /// Run peephole optimisation on the given instruction block. This evaluated constant 152 | /// expressions and propagates constants if the corresponding options are enabled 153 | /// 154 | /// - Parameter block: The block to optimise 155 | /// - Returns: The optimised block 156 | private func peepholeOptimise(block: [IRInstruction]) -> [IRInstruction] { 157 | if block.isEmpty { 158 | return block 159 | } 160 | var optimisedBlock: [IRInstruction] = block 161 | 162 | var currentInstructionIndex = 0 163 | while currentInstructionIndex < optimisedBlock.count { 164 | while true { 165 | let previousInstruction: IRInstruction? 166 | if currentInstructionIndex > 0 { 167 | previousInstruction = optimisedBlock[currentInstructionIndex - 1] 168 | } else { 169 | previousInstruction = nil 170 | } 171 | 172 | let currentInstruction = optimisedBlock[currentInstructionIndex] 173 | 174 | guard let optimisedInstruction = peepholeOptimise(previousInstruction: previousInstruction, currentInstruction: currentInstruction) else { 175 | break 176 | } 177 | optimisedBlock[currentInstructionIndex] = optimisedInstruction 178 | } 179 | currentInstructionIndex += 1 180 | } 181 | 182 | return optimisedBlock 183 | } 184 | 185 | /// Run a sing peephole optimisation step 186 | /// 187 | /// - Parameters: 188 | /// - previousInstruction: The previous instruction if the current instruction is not the 189 | /// first in the block 190 | /// - currentInstruction: The current instruction 191 | /// - Returns: A new instruction that replaces the current instruction if the current 192 | /// instruction could be optimised or `nil` if no optimisation was performed 193 | private func peepholeOptimise(previousInstruction: IRInstruction?, 194 | currentInstruction: IRInstruction) -> IRInstruction? { 195 | 196 | // Optimisation only operating on the currentInstruction 197 | 198 | if options.contains(.constantExprssionEvaluation) { 199 | // Addition of two constants 200 | if case .add(.integer(let lhs), .integer(let rhs), let destination) = currentInstruction { 201 | return .load(value: .integer(lhs + rhs), destination: destination) 202 | } 203 | 204 | // Eliminate constant compares 205 | if case .equal(.integer(let lhs), .integer(let rhs), let destination) = currentInstruction { 206 | return .load(value: .boolean(lhs == rhs), destination: destination) 207 | } 208 | 209 | // Eliminate constant branches 210 | if case .branch(.boolean(let check), let trueBlock, let falseBlock) = currentInstruction { 211 | return .jump(toBlock: check ? trueBlock : falseBlock) 212 | } 213 | } 214 | 215 | // Optimisation taking into account the last instruction 216 | 217 | guard let lastInstruction = previousInstruction else { 218 | return nil 219 | } 220 | 221 | if options.contains(.constantPropagation) { 222 | // Propagate constant into add on lhs 223 | if case .add(.register(let lhs), let rhs, let destination) = currentInstruction, 224 | case .load(value: .integer(let lhsValue), destination: lhs) = lastInstruction { 225 | return .add(lhs: .integer(lhsValue), rhs: rhs, destination: destination) 226 | } 227 | // Propagate constant into add on rhs 228 | if case .add(let lhs, .register(let rhs), let destination) = currentInstruction, 229 | case .load(value: .integer(let rhsValue), destination: rhs) = lastInstruction { 230 | return .add(lhs: lhs, rhs: .integer(rhsValue), destination: destination) 231 | } 232 | 233 | // Propagate constant into compare on lhs 234 | if case .equal(.register(let lhs), let rhs, let destination) = currentInstruction, 235 | case .load(value: .integer(let lhsValue), destination: lhs) = lastInstruction { 236 | return .equal(lhs: .integer(lhsValue), rhs: rhs, destination: destination) 237 | } 238 | // Propagate constant into compare on rhs 239 | if case .equal(let lhs, .register(let rhs), let destination) = currentInstruction, 240 | case .load(value: .integer(let rhsValue), destination: rhs) = lastInstruction { 241 | return .equal(lhs: lhs, rhs: .integer(rhsValue), destination: destination) 242 | } 243 | 244 | // Propagate constant into branch 245 | if case .branch(.register(let check), let trueBlock, let falseBlock) = currentInstruction, 246 | case .load(value: .boolean(let value), destination: check) = lastInstruction { 247 | return .branch(check: .boolean(value), trueBlock: trueBlock, falseBlock: falseBlock) 248 | } 249 | } 250 | 251 | return nil 252 | } 253 | 254 | // MARK: Empty block elimination 255 | 256 | /// Eliminate block only containting a `jump` by redirecting `jump`s or `branch`es to this block 257 | /// to that block's jump target 258 | /// 259 | /// - Parameter ir: The IR function to optimise 260 | /// - Returns: The optimised IR function without block just containing `jump`s 261 | private func eliminateEmptyBlocks(in ir: IRFunction) -> IRFunction { 262 | var result = ir 263 | for (blockName, instructions) in result.blocks { 264 | var replacement: (BlockName, BlockName)? = nil 265 | if instructions.count == 1 { 266 | if case .jump(let toBlock) = instructions.first! { 267 | replacement = (blockName, toBlock) 268 | } 269 | } 270 | if let (replaceBlock, replaceBy) = replacement { 271 | result = redirectJumps(from: replaceBlock, to: replaceBy, in: result) 272 | } 273 | } 274 | return result 275 | } 276 | 277 | /// Redirect all jumps from `source` to `destination` in the given compilation result 278 | /// 279 | /// - Parameters: 280 | /// - source: `jump`s to this block shall be redirected 281 | /// - destination: The new destination where the `jump`s should point 282 | /// - ir: The IR function to optimise 283 | /// - Returns: The optimised IR function 284 | private func redirectJumps(from source: BlockName, to destination: BlockName, in ir: IRFunction) -> IRFunction { 285 | func performReplacement(_ block: BlockName) -> BlockName { 286 | if block == source { 287 | return destination 288 | } else { 289 | return block 290 | } 291 | } 292 | 293 | var resultBlocks: [BlockName: [IRInstruction]] = [:] 294 | 295 | for (blockName, instructions) in ir.blocks { 296 | var resultIntructions: [IRInstruction] = [] 297 | for instruction in instructions { 298 | switch instruction { 299 | case .branch(let check, let trueBlock, let falseBlock): 300 | resultIntructions.append(.branch(check: check, 301 | trueBlock: performReplacement(trueBlock), 302 | falseBlock: performReplacement(falseBlock))) 303 | case .jump(let toBlock): 304 | resultIntructions.append(.jump(toBlock: performReplacement(toBlock))) 305 | default: 306 | resultIntructions.append(instruction) 307 | } 308 | } 309 | resultBlocks[blockName] = resultIntructions 310 | } 311 | 312 | return IRFunction(startBlock: performReplacement(ir.startBlock), blocks: resultBlocks, argumentRegisters: ir.argumentRegisters) 313 | } 314 | 315 | // MARK: Dead code elimination 316 | 317 | /// Eliminate instructions after `branch`, `jump`, or `return` 318 | /// 319 | /// - Parameter ir: The IR function to optimise 320 | /// - Returns: The optimised IR function 321 | private func eliminateDeadCode(in ir: IRFunction) -> IRFunction { 322 | var result: [BlockName: [IRInstruction]] = [:] 323 | for (blockName, instructions) in ir.blocks { 324 | var resultInstructions: [IRInstruction] = [] 325 | 326 | instructionsLoop: for instruction in instructions { 327 | switch instruction { 328 | case .jump(_), .branch(_), .return(_): 329 | resultInstructions.append(instruction) 330 | break instructionsLoop 331 | default: 332 | resultInstructions.append(instruction) 333 | } 334 | } 335 | 336 | result[blockName] = resultInstructions 337 | } 338 | 339 | return IRFunction(startBlock: ir.startBlock, blocks: result, argumentRegisters: ir.argumentRegisters) 340 | } 341 | 342 | // MARK: Dead store elimination 343 | 344 | /// Eliminate instructions that write to registers that are never read 345 | /// 346 | /// - Parameter ir: The IR function to optimise 347 | /// - Returns: The optimised IR function 348 | private func eliminateDeadStores(in ir: IRFunction) -> IRFunction { 349 | var blocks = ir.blocks 350 | while true { 351 | let usedRegisters = determineUsedRegisters(inBlocks: blocks) 352 | let (changed, optimisedBlocks) = removeInstructions(assigningRegistersNotIn: usedRegisters, 353 | inBlocks: blocks) 354 | if !changed { 355 | return IRFunction(startBlock: ir.startBlock, blocks: blocks, argumentRegisters: ir.argumentRegisters) 356 | } else { 357 | blocks = optimisedBlocks 358 | } 359 | } 360 | } 361 | 362 | /// Determine the registers whose values are ever read 363 | /// 364 | /// - Parameter blocks: The blocks to analyse 365 | /// - Returns: The registers whose value is read in any of the blocks 366 | private func determineUsedRegisters(inBlocks blocks: [BlockName: [IRInstruction]]) -> [Register] { 367 | var usedRegisters: [Register] = [] 368 | 369 | func use(irValue: IRValue) { 370 | if case .register(let register) = irValue { 371 | usedRegisters.append(register) 372 | } 373 | } 374 | 375 | for (_, instructions) in blocks { 376 | for instruction in instructions { 377 | switch instruction { 378 | case .add(let lhs, let rhs, _): 379 | use(irValue: lhs) 380 | use(irValue: rhs) 381 | case .equal(let lhs, let rhs, _): 382 | use(irValue: lhs) 383 | use(irValue: rhs) 384 | case .branch(let check, _, _): 385 | use(irValue: check) 386 | case .call(_, let arguments, _): 387 | for argument in arguments { 388 | use(irValue: argument) 389 | } 390 | case .return(returnValue: let returnValue): 391 | use(irValue: returnValue) 392 | default: 393 | break 394 | } 395 | } 396 | } 397 | 398 | return usedRegisters 399 | } 400 | 401 | /// Remove all instructions in the given blocks that assign registers not in 402 | /// `assigningRegistersNotIn` 403 | /// 404 | /// - Parameters: 405 | /// - assigningRegistersNotIn: Remove all instructions assigning registers that are not in 406 | /// this list 407 | /// - blocks: The blocks in which instructions shall be removed 408 | /// - Returns: A tuple `(changed, blocks)`. `changed` is true if any instruction was removed 409 | /// `blocks` contains the blocks in which instructions were removed as described above 410 | private func removeInstructions(assigningRegistersNotIn: [Register], 411 | inBlocks blocks: [BlockName: [IRInstruction]]) -> (Bool, [BlockName: [IRInstruction]]) { 412 | var result: [BlockName: [IRInstruction]] = [:] 413 | var changed = false 414 | for (blockName, instructions) in blocks { 415 | var resultInstructions: [IRInstruction] = [] 416 | 417 | for instruction in instructions { 418 | switch instruction { 419 | case .add(_, _, let destination) where !assigningRegistersNotIn.contains(destination): 420 | changed = true 421 | break 422 | case .equal(_, _, let destination) where !assigningRegistersNotIn.contains(destination): 423 | changed = true 424 | break 425 | case .load(_, let destination) where !assigningRegistersNotIn.contains(destination): 426 | changed = true 427 | break 428 | default: 429 | resultInstructions.append(instruction) 430 | } 431 | } 432 | 433 | result[blockName] = resultInstructions 434 | } 435 | 436 | return (changed, result) 437 | } 438 | 439 | // MARK: Dead block elimination 440 | 441 | /// Remove all blocks that are not the start block and that are never jumped to 442 | /// 443 | /// - Parameter ir: The IR function in which dead blocks shall be removed 444 | /// - Returns: The optimised IR function 445 | private func eliminateDeadBlocks(in ir: IRFunction) -> IRFunction { 446 | var optimisedBlocks = ir.blocks 447 | 448 | while true { 449 | var usedBlocks: Set = [ir.startBlock] 450 | for (_, instructions) in optimisedBlocks { 451 | for instruction in instructions { 452 | switch instruction { 453 | case .branch(_, let trueBlock, let falseBlock): 454 | usedBlocks.insert(trueBlock) 455 | usedBlocks.insert(falseBlock) 456 | case .jump(let toBlock): 457 | usedBlocks.insert(toBlock) 458 | default: 459 | break 460 | } 461 | } 462 | } 463 | 464 | let blocksToRemove = Set(optimisedBlocks.keys).subtracting(usedBlocks) 465 | if blocksToRemove.isEmpty { 466 | break 467 | } else { 468 | for toRemove in blocksToRemove { 469 | optimisedBlocks[toRemove] = nil 470 | } 471 | } 472 | } 473 | 474 | return IRFunction(startBlock: ir.startBlock, blocks: optimisedBlocks, argumentRegisters: ir.argumentRegisters) 475 | } 476 | 477 | // MARK: Jump target inlining 478 | 479 | /// Eliminate `jump`s by replacing the `jump` instruction with the block the `jump` jumps to. 480 | /// 481 | /// This asssumes that there are no loops in the programs. Otherwise this method may not 482 | /// terminate 483 | /// 484 | /// - Parameter ir: The IR function to optimise 485 | /// - Returns: The optimised compilation result 486 | private func inlineJumpTargets(in ir: IRFunction) -> IRFunction { 487 | var resultBlocks: [BlockName: [IRInstruction]] = [:] 488 | for (blockName, _) in ir.blocks { 489 | resultBlocks[blockName] = inlineJumpTargets(in: blockName, ir: ir) 490 | } 491 | return IRFunction(startBlock: ir.startBlock, blocks: resultBlocks, argumentRegisters: ir.argumentRegisters) 492 | } 493 | 494 | /// Eliminate `jump`s in this block by replacing the `jump` instruction with the block the 495 | /// `jump` jumps to. 496 | /// 497 | /// This asssumes that there are no loops in the programs. Otherwise this method may not 498 | /// terminate 499 | /// 500 | /// - Parameters: 501 | /// - block: The block in which `jump` shall be eliminated 502 | /// - ir: The IR function that specifies the other blocks 503 | /// - Returns: A block semantically equivalent to `block` but without `jump`s 504 | private func inlineJumpTargets(in block: BlockName, ir: IRFunction) -> [IRInstruction] { 505 | var result: [IRInstruction] = [] 506 | for instruction in ir.blocks[block]! { 507 | switch instruction { 508 | case .jump(let toBlock): 509 | result += inlineJumpTargets(in: toBlock, ir: ir) 510 | default: 511 | result.append(instruction) 512 | } 513 | } 514 | return result 515 | } 516 | } 517 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/Parser.swift: -------------------------------------------------------------------------------- 1 | /// The parser converts a stream of tokens generated by a lexer into an 2 | /// abstract syntax tree that will be used by later stages of the 3 | /// compilation process. 4 | /// 5 | /// It generates and operates the lexer internally 6 | open class Parser { 7 | 8 | /// The lexer that converts the source code into tokens 9 | private var lexer: Lexer! 10 | 11 | /// The token that shall be parsed next 12 | public var nextToken: Token! 13 | 14 | /// The last token that was parsed 15 | private var lastToken: Token? 16 | 17 | public init() { 18 | } 19 | 20 | /// Parse the sourceCode of this parser into an abstract syntax tree (AST) 21 | /// 22 | /// - Parameter sourceFile: The source file to parse 23 | /// - Returns: The parsed abstract syntax tree 24 | /// - Throws: A CompilationError if compilation failed 25 | public static func parse(sourceFile: SwiftFile) throws -> ASTRoot { 26 | let parser = Parser() 27 | return try parser.parse(sourceFile: sourceFile) 28 | } 29 | 30 | /// Parse the sourceCode of this parser into an abstract syntax tree (AST) 31 | /// 32 | /// - Parameter sourceFile: The source file to parse 33 | /// - Returns: The parsed abstract syntax tree 34 | /// - Throws: A CompilationError if compilation failed 35 | public func parse(sourceFile: SwiftFile) throws -> ASTRoot { 36 | self.lexer = Lexer(sourceCode: sourceFile.sourceCode) 37 | 38 | self.nextToken = try lexer.nextToken() 39 | let startLoc = self.nextToken.sourceRange.start 40 | 41 | var statements: [Statement] = [] 42 | while nextToken.payload != .endOfFile { 43 | statements.append(try parseStatement()) 44 | } 45 | 46 | let endLoc = self.nextToken.sourceRange.start 47 | 48 | return ASTRoot(statements: statements, 49 | sourceRange: SourceRange(start: startLoc, end: endLoc)) 50 | } 51 | 52 | /// Create a source range starting at the given location and ending at the curren 53 | /// position 54 | /// 55 | /// - Parameter startingAt: The source location where the range shall start 56 | /// - Returns: A range starting at the given location and ending at the current token 57 | public func range(startingAt: SourceLoc) -> SourceRange { 58 | if let lastToken = lastToken { 59 | return SourceRange(start: startingAt, end: lastToken.sourceRange.end) 60 | } else { 61 | return SourceRange(start: startingAt, end: nextToken.sourceRange.start) 62 | } 63 | } 64 | 65 | /// Consume the next token and fill the nextToken variable with 66 | /// the upcoming token from the lexer 67 | /// 68 | /// - Returns: The token that has just been consumed 69 | /// - Throws: A CompilationError if the lexer failed to return the next token 70 | @discardableResult 71 | public func consumeToken() throws -> Token? { 72 | lastToken = nextToken 73 | nextToken = try self.lexer.nextToken() 74 | return lastToken 75 | } 76 | 77 | /// Consume the current token if it is of the given TokenKind 78 | /// 79 | /// - Parameter token: The type of the token that shall be consume if possible 80 | /// - Returns: Whether or not a token has been consumed 81 | /// - Throws: A CompilationError if the lexer failed to return the next token 82 | @discardableResult 83 | func consumeIf(_ token: TokenKind) throws -> Bool { 84 | if nextToken.payload == token { 85 | try consumeToken() 86 | return true 87 | } else { 88 | return false 89 | } 90 | } 91 | 92 | 93 | /// Parse the base of an expression, i.e. expressions that cannot contain nested expressions 94 | /// This currently only includes literals and identifiers (for variables) 95 | /// 96 | /// - Returns: The parsed expression 97 | /// - Throws: A CompilationError if compilation failed 98 | private func parseBaseExpression() throws -> Expression { 99 | switch nextToken.payload { 100 | case .integer(let value): 101 | let sourceRange = nextToken.sourceRange 102 | try consumeToken() 103 | return IntegerLiteralExpression(value: value, sourceRange: sourceRange) 104 | case .stringLiteral(let value): 105 | let sourceRange = nextToken.sourceRange 106 | try consumeToken() 107 | return StringLiteralExpression(value: value, sourceRange: sourceRange) 108 | case .identifier(let name): 109 | let identifierRange = nextToken.sourceRange 110 | try consumeToken() 111 | if try consumeIf(.leftParen) { 112 | var arguments: [Expression] = [] 113 | while true { 114 | if try consumeIf(.rightParen) { 115 | break 116 | } 117 | arguments.append(try parseExpression()) 118 | if !(try consumeIf(.comma)) { 119 | if !(try consumeIf(.rightParen)) { 120 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Expected ')' to terminate function call") 121 | } 122 | break 123 | } 124 | } 125 | 126 | return FunctionCallExpression(functionName: name, 127 | arguments: arguments, 128 | functionNameRange: identifierRange, 129 | sourceRange: range(startingAt: identifierRange.start)) 130 | } else { 131 | return IdentifierReferenceExpression(name: name, sourceRange: range(startingAt: identifierRange.start)) 132 | } 133 | default: 134 | throw CompilationError(sourceRange: nextToken.sourceRange, 135 | errorMessage: "Expected expression but found '\(nextToken.payload.sourceCodeRepresentation)'") 136 | } 137 | } 138 | 139 | /// Parse an expression including nested expressions. Takes an optional argument to specify 140 | /// that restrict the operators in order to parse to parse expressions with binary operators 141 | /// of different precedences correctly 142 | /// 143 | /// - Parameter precedenceGreaterOrEqualThan: Only parse binary operator expressions if their 144 | /// operator's precedence is greater or equal to the value 145 | /// - Returns: The parsed expression 146 | /// - Throws: A CompilationError if compilation failed 147 | public func parseExpression(precedenceGreaterOrEqualThan: Int = 0) throws -> Expression { 148 | var workingExpression = try parseBaseExpression() 149 | 150 | while case .operator(let name) = nextToken.payload { 151 | let binOpRange = nextToken.sourceRange 152 | let binOperator: BinaryOperatorExpression.Operator 153 | switch name { 154 | case "+": 155 | binOperator = .add 156 | case "-": 157 | binOperator = .sub 158 | case "==": 159 | binOperator = .equal 160 | case "<=": 161 | binOperator = .lessOrEqual 162 | default: 163 | try consumeToken() 164 | throw CompilationError(sourceRange: binOpRange, 165 | errorMessage: "Unknown operator '\(name)'") 166 | } 167 | if binOperator.precedence < precedenceGreaterOrEqualThan { 168 | break 169 | } 170 | 171 | try consumeToken() 172 | 173 | let rhs = try parseExpression(precedenceGreaterOrEqualThan: binOperator.precedence) 174 | workingExpression = BinaryOperatorExpression(lhs: workingExpression, 175 | rhs: rhs, 176 | operator: binOperator) 177 | } 178 | return workingExpression 179 | } 180 | 181 | 182 | /// Parses a single statement 183 | /// 184 | /// - Returns: The parsed expression 185 | /// - Throws: A CompilationError if compilation failed 186 | private func parseStatement() throws -> Statement { 187 | switch nextToken.payload { 188 | case .if: 189 | return try parseIfStatement() 190 | case .func: 191 | return try parseFunctionDeclaration() 192 | case .return: 193 | let returnRange = nextToken.sourceRange 194 | try consumeToken() 195 | let expr = try parseExpression() 196 | return ReturnStatement(expression: expr, sourceRange: range(startingAt: returnRange.start)) 197 | default: 198 | return try parseExpression() 199 | } 200 | } 201 | 202 | private func parseFunctionDeclaration() throws -> FunctionDeclaration { 203 | guard nextToken.payload == .func else { 204 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Expected 'func' but saw \(nextToken!)") 205 | } 206 | let funcKeywordRange = nextToken.sourceRange 207 | try consumeToken() 208 | guard case .identifier(let funcName) = nextToken.payload else { 209 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Expected identifier after 'func' keyword but saw \(nextToken!)") 210 | } 211 | try consumeToken() 212 | guard try consumeIf(.leftParen) else { 213 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Expeced '(' to start the functions parameter list but saw \(nextToken!)") 214 | } 215 | 216 | var parameters: [VariableDeclaration] = [] 217 | 218 | while true { 219 | if try consumeIf(.rightParen) { 220 | break 221 | } 222 | guard try consumeIf(.underscore) else { 223 | throw CompilationError(sourceRange: nextToken.sourceRange, 224 | errorMessage: "Named parameters are not supported yet") 225 | } 226 | parameters.append(try parseVariableDeclaration()) 227 | if !(try consumeIf(.comma)) { 228 | if !(try consumeIf(.rightParen)) { 229 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Expeced ')' to end the functions parameter list but saw \(nextToken!)") 230 | } 231 | break 232 | } 233 | } 234 | 235 | var returnTypeName = "Void" 236 | if try consumeIf(.arrow) { 237 | guard case .identifier(let returnTypeName2) = nextToken.payload else { 238 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Expeced the function's return type but saw \(nextToken!)") 239 | } 240 | try consumeToken() 241 | returnTypeName = returnTypeName2 242 | } 243 | 244 | let body = try parseBraceStatement() 245 | 246 | guard let returnType = Type.fromString(returnTypeName) else { 247 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Unknown type '\(returnTypeName)'") 248 | } 249 | 250 | return FunctionDeclaration(name: funcName, parameters: parameters, returnType: returnType, body: body, sourceRange: range(startingAt: funcKeywordRange.start)) 251 | } 252 | 253 | private func parseVariableDeclaration() throws -> VariableDeclaration { 254 | let startLoc = nextToken.sourceRange.start 255 | guard case .identifier(let paramName) = nextToken.payload else { 256 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Expected identifier to specify the parameter's name but saw \(nextToken!)") 257 | } 258 | try consumeToken() 259 | guard try consumeIf(.colon) else { 260 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Expected ':' to seperate the parameter's name and type but saw \(nextToken!)") 261 | } 262 | guard case .identifier(let paramTypeName) = nextToken.payload else { 263 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Expected identifier to specify the parameter's type but saw \(nextToken!)") 264 | } 265 | guard let paramType = Type.fromString(paramTypeName) else { 266 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Unknown type '\(paramTypeName)'") 267 | } 268 | try consumeToken() 269 | return VariableDeclaration(name: paramName, type: paramType, sourceRange: range(startingAt: startLoc)) 270 | } 271 | 272 | open func parseIfStatement() throws -> Statement { 273 | guard nextToken == .if else { 274 | throw CompilationError(sourceRange: nextToken.sourceRange, errorMessage: "Expected 'if' but saw \(nextToken!)") 275 | } 276 | let ifRange = nextToken.sourceRange 277 | try consumeToken() 278 | let condition = try parseExpression() 279 | let body = try parseBraceStatement() 280 | var elseBody: BraceStatement? = nil 281 | var elseRange: SourceRange? = nil 282 | if nextToken == .else { 283 | elseRange = nextToken.sourceRange 284 | try consumeToken() 285 | elseBody = try parseBraceStatement() 286 | } 287 | return IfStatement(condition: condition, 288 | body: body, 289 | elseBody: elseBody, 290 | ifRange: ifRange, 291 | elseRange: elseRange, 292 | sourceRange: range(startingAt: ifRange.start)) 293 | } 294 | 295 | 296 | /// Parse a brace statement including its braces '{' and '}' 297 | /// 298 | /// - Returns: The parsed expression 299 | /// - Throws: A CompilationError if compilation failed 300 | public func parseBraceStatement() throws -> BraceStatement { 301 | let startLoc = nextToken.sourceRange.start 302 | if !(try consumeIf(.leftBrace)) { 303 | throw CompilationError(location: nextToken.sourceRange.start, 304 | errorMessage: "Missing '{' to start body of brace statement") 305 | } 306 | var body: [Statement] = [] 307 | while self.nextToken.payload != .rightBrace { 308 | let stmt = try parseStatement() 309 | body.append(stmt) 310 | } 311 | assert(nextToken.payload == .rightBrace) 312 | try consumeToken() 313 | 314 | return BraceStatement(body: body, sourceRange: range(startingAt: startLoc)) 315 | } 316 | } 317 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/Scanner.swift: -------------------------------------------------------------------------------- 1 | /// The scanner reads the source code character by character and maintains the character's position 2 | class Scanner { 3 | 4 | /// The source code as a list of `UnicodeScalar`s that can be picked one by one 5 | private let sourceCode: [UnicodeScalar] 6 | 7 | /// The index in `souceCode` that contains the character currently being scanned 8 | private var parsePosition = 0 9 | 10 | /// The location of the current character in the source code 11 | private(set) var sourceLoc = SourceLoc(line: 1, column: 1, offset: 0) 12 | 13 | init(sourceCode: String) { 14 | self.sourceCode = Array(sourceCode.unicodeScalars) 15 | self.parsePosition = self.sourceCode.startIndex 16 | } 17 | 18 | /// Peek at the current character without consuming it. Is `nil` if the end of the file has been 19 | /// reached 20 | var currentChar: UnicodeScalar? { 21 | guard parsePosition < sourceCode.count else { 22 | return nil 23 | } 24 | return sourceCode[parsePosition] 25 | } 26 | 27 | /// Consume the current character and move scanning one character ahead 28 | func consumeChar() { 29 | if currentChar == "\n" { 30 | self.sourceLoc.line += 1 31 | self.sourceLoc.column = 1 32 | } else { 33 | self.sourceLoc.column += 1 34 | } 35 | self.sourceLoc.offset += 1 36 | 37 | parsePosition += 1 38 | } 39 | } 40 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/SwiftFile.swift: -------------------------------------------------------------------------------- 1 | import Foundation 2 | 3 | /// Represents the source code that was contained in a `.swift` file 4 | public struct SwiftFile: _ExpressibleByFileReferenceLiteral, CustomStringConvertible, CustomPlaygroundDisplayConvertible { 5 | public let sourceCode: String 6 | 7 | public init(fileReferenceLiteralResourceName path: String) { 8 | let url = Bundle.main.url(forResource: path, withExtension: nil)! 9 | sourceCode = try! String(contentsOf: url) 10 | } 11 | 12 | /// Create a `SwiftFile` with manually obtained source code 13 | /// 14 | /// - Parameter sourceCode: The source code of the file 15 | public init(fromSourceCode sourceCode: String) { 16 | self.sourceCode = sourceCode 17 | } 18 | 19 | public var description: String { 20 | return sourceCode 21 | } 22 | 23 | /// The syntax-highlighted source code of the file 24 | public var highlightedString: NSAttributedString { 25 | return SyntaxHighlighter.highlight(sourceFile: self) 26 | } 27 | 28 | public var playgroundDescription: Any { 29 | return self.highlightedString.withPlaygroundQuickLookBackgroundColor 30 | } 31 | } 32 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/SyntaxHighlighter.swift: -------------------------------------------------------------------------------- 1 | import Cocoa 2 | 3 | class SyntaxHighlighter { 4 | typealias Result = Void 5 | 6 | private static let numberColor: NSColor = #colorLiteral(red: 0.1098039216, green: 0, blue: 0.8117647059, alpha: 1) 7 | private static let stringColor: NSColor = #colorLiteral(red: 0.768627451, green: 0.1019607843, blue: 0.0862745098, alpha: 1) 8 | private static let keywordColor: NSColor = #colorLiteral(red: 0.6666666667, green: 0.05098039216, blue: 0.568627451, alpha: 1) 9 | private static let identifierColor: NSColor = #colorLiteral(red: 0.1490196078, green: 0.2784313725, blue: 0.2941176471, alpha: 1) 10 | 11 | /// Create a monospaced attributed string where characters have been coloured according to 12 | /// Xcode's default syntax highlighting scheme 13 | /// 14 | /// - Parameter sourceFile: The source file to highlight 15 | /// - Returns: The syntax-highlighted source code 16 | static func highlight(sourceFile: SwiftFile) -> NSAttributedString { 17 | var workingString = NSMutableAttributedString(attributedString: sourceFile.sourceCode.monospacedString) 18 | let lexer = Lexer(sourceCode: sourceFile.sourceCode) 19 | do { 20 | var token = try lexer.nextToken() 21 | while token != .endOfFile { 22 | let color: NSColor? 23 | switch token.payload { 24 | case .if, .else, .func, .return: 25 | color = keywordColor 26 | case .integer(_): 27 | color = numberColor 28 | case .identifier(_): 29 | color = identifierColor 30 | case .stringLiteral(_): 31 | color = stringColor 32 | default: 33 | color = nil 34 | } 35 | if let color = color { 36 | self.color(range: token.sourceRange, in: &workingString, withColor: color) 37 | } 38 | 39 | token = try lexer.nextToken() 40 | } 41 | } catch {} 42 | 43 | return workingString 44 | } 45 | 46 | /// Colour part of a `NSMutableAttributedString` in a colour 47 | /// 48 | /// - Parameters: 49 | /// - range: The source range to colour 50 | /// - string: The string to colour 51 | /// - color: The colour that the given range shall be assigned in the string 52 | private static func color(range: SourceRange, 53 | in string: inout NSMutableAttributedString, 54 | withColor color: NSColor) { 55 | let textRange = NSRange(location: range.start.offset, 56 | length: range.end.offset - range.start.offset) 57 | 58 | string.addAttributes([.foregroundColor: color], range: textRange) 59 | } 60 | } 61 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/Compiler/Typechecker.swift: -------------------------------------------------------------------------------- 1 | /// Represents the different types an expression can return 2 | public enum Type { 3 | case integer 4 | case boolean 5 | case string 6 | case function 7 | case none 8 | 9 | public static func fromString(_ string: String) -> Type? { 10 | switch string { 11 | case "Int": 12 | return .integer 13 | case "Bool": 14 | return .boolean 15 | case "String": 16 | return .string 17 | case "Void": 18 | return .none 19 | default: 20 | return nil 21 | } 22 | } 23 | } 24 | 25 | 26 | fileprivate class LookupScope { 27 | let previousScope: LookupScope? 28 | var lookupTable: [String: Declaration] = [:] 29 | 30 | init(previousScope: LookupScope?) { 31 | self.previousScope = previousScope 32 | } 33 | 34 | func add(_ variable: VariableDeclaration) { 35 | lookupTable[variable.name] = variable 36 | } 37 | 38 | func add(_ variable: FunctionDeclaration) { 39 | lookupTable[variable.name] = variable 40 | } 41 | 42 | func lookup(_ name: String) -> Declaration? { 43 | if let decl = lookupTable[name] { 44 | return decl 45 | } 46 | return previousScope?.lookup(name) 47 | } 48 | } 49 | 50 | /// Checks that there are no type-system violations in a given AST 51 | public class Typechecker: ThrowingASTWalker { 52 | typealias Result = Type 53 | 54 | private var lookupScope = LookupScope(previousScope: nil) 55 | private var functions: [String: FunctionDeclaration] = [:] 56 | 57 | public init() { 58 | } 59 | 60 | /// Check that there are no type-system violation in the given AST. Returns if there are no 61 | /// violations and throws an error if a violation was found 62 | /// 63 | /// - Parameter node: The AST to check 64 | /// - Throws: A `CompilationError` if a type system violation was found 65 | public static func typecheck(node: ASTNode) throws { 66 | let typechecker = Typechecker() 67 | try typechecker.typecheck(node: node) 68 | } 69 | 70 | public func typecheck(node: ASTNode) throws { 71 | _ = try self.walk(node) 72 | } 73 | 74 | // MARK: - ASTWalker 75 | 76 | func visit(ifStatement: IfStatement) throws -> Type { 77 | _ = try walk(ifStatement.condition) 78 | _ = try walk(ifStatement.body) 79 | if let elseBody = ifStatement.elseBody { 80 | _ = try walk(elseBody) 81 | } 82 | return .none 83 | } 84 | 85 | func visit(braceStatement: BraceStatement) throws -> Type { 86 | for statement in braceStatement.body { 87 | _ = try walk(statement) 88 | } 89 | return .none 90 | } 91 | 92 | func visit(returnStatement: ReturnStatement) throws -> Type { 93 | try walk(returnStatement.expression) 94 | return .none 95 | } 96 | 97 | func visit(binaryOperatorExpression: BinaryOperatorExpression) throws -> Type { 98 | let lhsType = try walk(binaryOperatorExpression.lhs) 99 | let rhsType = try walk(binaryOperatorExpression.rhs) 100 | switch (binaryOperatorExpression.operator) { 101 | case .add, .sub: 102 | if lhsType == .integer && rhsType == .integer { 103 | return .integer 104 | } else { 105 | throw CompilationError(sourceRange: binaryOperatorExpression.sourceRange, 106 | errorMessage: "The left-hand-side and right-hand side of '\(binaryOperatorExpression.operator.sourceCodeName)' need to be integers") 107 | } 108 | case .equal, .lessOrEqual: 109 | if lhsType == .integer && rhsType == .integer { 110 | return .boolean 111 | } else { 112 | throw CompilationError(sourceRange: binaryOperatorExpression.sourceRange, 113 | errorMessage: "The left-hand-side and right-hand side of '==' need to be integers") 114 | } 115 | } 116 | } 117 | 118 | func visit(integerLiteralExpression: IntegerLiteralExpression) throws -> Type { 119 | return .integer 120 | } 121 | 122 | func visit(stringLiteralExpression: StringLiteralExpression) throws -> Type { 123 | return .string 124 | } 125 | 126 | func visit(identifierReferenceExpression: IdentifierReferenceExpression) throws -> Type { 127 | guard let referencedDeclaration = lookupScope.lookup(identifierReferenceExpression.name) else { 128 | throw CompilationError(sourceRange: identifierReferenceExpression.sourceRange, 129 | errorMessage: "Referenced undefined variable \(identifierReferenceExpression.name)") 130 | } 131 | identifierReferenceExpression.referencedDeclaration = referencedDeclaration 132 | 133 | return referencedDeclaration.type 134 | } 135 | 136 | func visit(functionCallExpression: FunctionCallExpression) throws -> Type { 137 | for argument in functionCallExpression.arguments { 138 | _ = try walk(argument) 139 | } 140 | if functionCallExpression.functionName == "print" { 141 | return .none 142 | } else { 143 | let lookupResult = lookupScope.lookup(functionCallExpression.functionName) 144 | guard let functionDeclaration = lookupResult as? FunctionDeclaration else { 145 | throw CompilationError(sourceRange: functionCallExpression.sourceRange, 146 | errorMessage: "Only functions can be called") 147 | } 148 | return functionDeclaration.returnType 149 | } 150 | } 151 | 152 | func visit(variableDeclaration: VariableDeclaration) throws -> Type { 153 | lookupScope.add(variableDeclaration) 154 | return .none 155 | } 156 | 157 | func visit(functionDeclaration: FunctionDeclaration) throws -> Type { 158 | lookupScope.add(functionDeclaration) 159 | lookupScope = LookupScope(previousScope: lookupScope) 160 | for parameter in functionDeclaration.parameters { 161 | try walk(parameter) 162 | } 163 | try walk(functionDeclaration.body) 164 | lookupScope = lookupScope.previousScope! 165 | 166 | return .none 167 | } 168 | 169 | func visit(astRoot: ASTRoot) throws -> Type { 170 | for statement in astRoot.statements { 171 | _ = try walk(statement) 172 | } 173 | return .none 174 | } 175 | } 176 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/GUI/TokenHoverView.swift: -------------------------------------------------------------------------------- 1 | import Cocoa 2 | 3 | public class TokenHoverView: NSTextView { 4 | private let tokenHoverView = NSTextField() 5 | 6 | // These variables need to be IUO since they would otherwise be overwritten 7 | // by init(frame:textcontainer) which is called out of line rdar://31317871 8 | private var tokens: [Token]! = nil 9 | private var sourceFile: SwiftFile! = nil 10 | 11 | init(frame: NSRect, sourceFile: SwiftFile) { 12 | super.init(frame: frame) 13 | 14 | self.sourceFile = sourceFile 15 | 16 | // Lex the source code for lexer tooltip 17 | do { 18 | let lexer = Lexer(sourceCode: sourceFile.sourceCode) 19 | var tokens: [Token] = [] 20 | var token: Token = try lexer.nextToken() 21 | while token.payload != TokenKind.endOfFile { 22 | tokens.append(token) 23 | token = try lexer.nextToken() 24 | } 25 | self.tokens = tokens 26 | } catch { 27 | self.tokens = [] 28 | } 29 | 30 | // Set self layout 31 | self.backgroundColor = NSColor.white 32 | self.drawsBackground = true 33 | self.textStorage?.setAttributedString(self.sourceFile.highlightedString) 34 | self.isEditable = false 35 | self.textContainerInset = NSSize(width: -5, height: 0) 36 | 37 | // Create the hover view 38 | self.tokenHoverView.backgroundColor = NSColor(white: 0.95, alpha: 1) 39 | self.tokenHoverView.font = .systemFont(ofSize: 13) 40 | self.tokenHoverView.drawsBackground = true 41 | self.tokenHoverView.wantsLayer = true 42 | self.tokenHoverView.layer?.borderColor = NSColor.lightGray.cgColor 43 | self.tokenHoverView.layer?.borderWidth = 1 44 | self.tokenHoverView.isEditable = false 45 | } 46 | 47 | // Workaround for rdar://31317871 48 | public override init(frame frameRect: NSRect, textContainer container: NSTextContainer?) { 49 | super.init(frame: frameRect, textContainer: container) 50 | } 51 | 52 | required public init?(coder: NSCoder) { 53 | fatalError("init(coder:) has not been implemented") 54 | } 55 | 56 | @objc override public func mouseMoved(with event: NSEvent) { 57 | // Determine the hover point in this frame 58 | let layoutManager = self.layoutManager! 59 | let textContainer = self.textContainer! 60 | let pointInTextView = self.convert(event.locationInWindow, from: nil) 61 | 62 | if pointInTextView.x < 0 || pointInTextView.x > self.frame.size.width || 63 | pointInTextView.y < 0 || pointInTextView.y > self.frame.size.height { 64 | self.mouseExited(with: event) 65 | return 66 | } 67 | 68 | var pointInTextContainer = pointInTextView 69 | pointInTextContainer.x -= self.textContainerOrigin.x 70 | pointInTextContainer.y -= self.textContainerOrigin.y 71 | 72 | // Determine the hovered character 73 | let glyphIndex = layoutManager.glyphIndex(for: pointInTextContainer, 74 | in: textContainer) 75 | 76 | // Get the token at that position 77 | let token = self.tokens.filter({ $0.sourceRange.start.offset <= glyphIndex && $0.sourceRange.end.offset > glyphIndex }).first 78 | 79 | // Display token 80 | if let token = token { 81 | tokenHoverView.stringValue = token.payload.description 82 | tokenHoverView.sizeToFit() 83 | let origin = CGPoint(x: pointInTextView.x, 84 | y: pointInTextView.y + 10) 85 | if self.tokenHoverView.superview == nil { 86 | self.window!.contentView!.addSubview(self.tokenHoverView) 87 | } 88 | tokenHoverView.frame.origin = self.convert(origin, to: self.window!.contentView!) 89 | tokenHoverView.frame.origin.y -= tokenHoverView.frame.size.height 90 | } else { 91 | self.tokenHoverView.removeFromSuperview() 92 | } 93 | 94 | // Highlight the source range or remove the highlighting 95 | self.highlight(range: token?.sourceRange) 96 | } 97 | 98 | @objc public override func mouseExited(with event: NSEvent) { 99 | self.tokenHoverView.removeFromSuperview() 100 | } 101 | 102 | public func highlight(range: SourceRange?) { 103 | let highlightedString = NSMutableAttributedString(attributedString: sourceFile.highlightedString) 104 | if let range = range { 105 | highlightedString.addAttributes([ 106 | .backgroundColor: NSColor.selectedControlColor 107 | ], range: NSRange(location: range.start.offset, length: range.end.offset - range.start.offset)) 108 | } 109 | textStorage?.setAttributedString(highlightedString) 110 | } 111 | } 112 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/GUI/TokensExplorer.swift: -------------------------------------------------------------------------------- 1 | import Cocoa 2 | 3 | /// Shows the syntax-highlighted source code of a `SwiftFile` and shows tooltips 4 | /// with the token's description when hovering over the source code 5 | open class TokensExplorer: NSSplitView { 6 | 7 | private let sourceFile: SwiftFile 8 | public let sourceViewer: TokenHoverView 9 | private let sourceViewerScrollView = NSScrollView() 10 | private let mainArea = NSStackView() 11 | 12 | public init(forSourceFile sourceFile: SwiftFile, withParser parser: Parser = Parser()) { 13 | self.sourceFile = sourceFile 14 | 15 | self.sourceViewer = TokenHoverView(frame: CGRect(x: 0, y: 0, width: 50, height: 600), 16 | sourceFile: sourceFile) 17 | self.sourceViewer.translatesAutoresizingMaskIntoConstraints = false 18 | 19 | super.init(frame: CGRect(x: 0, y: 0, width: 500, height: 600)) 20 | 21 | self.wantsLayer = true 22 | self.layer!.backgroundColor = NSColor(white: 247/255, alpha: 1).cgColor 23 | 24 | // Create header 25 | let sourceCodeHeader = NSTextField(labelWithString: "Source Code") 26 | sourceCodeHeader.font = .systemFont(ofSize: 33, weight: .semibold) 27 | 28 | sourceViewerScrollView.documentView = self.sourceViewer 29 | sourceViewerScrollView.translatesAutoresizingMaskIntoConstraints = false 30 | sourceViewerScrollView.hasVerticalScroller = true 31 | sourceViewerScrollView.addConstraint(NSLayoutConstraint(item: sourceViewer, attribute: .width, relatedBy: .equal, toItem: sourceViewerScrollView, attribute: .width, multiplier: 1, constant: 0)) 32 | 33 | mainArea.orientation = .vertical 34 | mainArea.translatesAutoresizingMaskIntoConstraints = false 35 | mainArea.addFullWidthView(sourceCodeHeader) 36 | mainArea.addFullWidthView(sourceViewerScrollView) 37 | 38 | self.addArrangedSubview(mainArea) 39 | } 40 | 41 | public required init?(coder: NSCoder) { 42 | fatalError("init(coder:) has not been implemented") 43 | } 44 | 45 | open override func updateTrackingAreas() { 46 | super.updateTrackingAreas() 47 | for trackingArea in self.trackingAreas { 48 | self.removeTrackingArea(trackingArea) 49 | } 50 | let trackingArea = NSTrackingArea(rect: self.mainArea.frame, options: [.activeAlways, .mouseMoved, .mouseEnteredAndExited], owner: self.sourceViewer, userInfo: nil) 51 | self.addTrackingArea(trackingArea) 52 | } 53 | } 54 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/Sources/GUI/ViewExtensions.swift: -------------------------------------------------------------------------------- 1 | import Cocoa 2 | 3 | public extension NSView { 4 | func addHeightConstraint(forHeight height: CGFloat, withPriority priority: Float = 1000) { 5 | let heightConstraint = NSLayoutConstraint(item: self, 6 | attribute: .height, 7 | relatedBy: .equal, 8 | toItem: nil, 9 | attribute: .notAnAttribute, 10 | multiplier: 1, 11 | constant: 100) 12 | heightConstraint.priority = NSLayoutConstraint.Priority(priority) 13 | self.addConstraint(heightConstraint) 14 | } 15 | 16 | func addWidthConstraint(forWidth width: CGFloat, withPriority priority: Float = 1000) { 17 | let widthConstraint = NSLayoutConstraint(item: self, 18 | attribute: .width, 19 | relatedBy: .equal, 20 | toItem: nil, 21 | attribute: .notAnAttribute, 22 | multiplier: 1, 23 | constant: 100) 24 | widthConstraint.priority = NSLayoutConstraint.Priority(priority) 25 | self.addConstraint(widthConstraint) 26 | } 27 | } 28 | 29 | public extension NSStackView { 30 | func addFullWidthView(_ view: NSView) { 31 | view.translatesAutoresizingMaskIntoConstraints = false 32 | self.addView(view, in: .top) 33 | self.addConstraints(NSLayoutConstraint.constraints(withVisualFormat: "|[view]|", 34 | options: [], 35 | metrics: nil, 36 | views: ["view": view])) 37 | } 38 | } 39 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/contents.xcplayground: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/playground.xcworkspace/contents.xcworkspacedata: -------------------------------------------------------------------------------- 1 | 2 | 4 | 6 | 7 | 8 | -------------------------------------------------------------------------------- /Introduction to compilers.playground/playground.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | IDEDidComputeMac32BitWarning 6 | 7 | 8 | 9 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Alex Hoppen 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Introduction to Compilers 2 | 3 | A macOS Playground I have created for the WWDC 2017 Schlarship application that shall provide a high-level overview of how modern compilers operate. 4 | 5 | ![Introductory page of the playground](Screenshot1.png) 6 | ![Parser page in the playground](Screenshot2.png) 7 | ![IR Generation page in the playground](Screenshot3.png) 8 | ![Optimisation page in the playground](Screenshot4.png) -------------------------------------------------------------------------------- /Screenshot1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahoppen/introduction-to-compilers/d5dbd3ae62032074652c26f5290cf1b40c669dcf/Screenshot1.png -------------------------------------------------------------------------------- /Screenshot2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahoppen/introduction-to-compilers/d5dbd3ae62032074652c26f5290cf1b40c669dcf/Screenshot2.png -------------------------------------------------------------------------------- /Screenshot3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahoppen/introduction-to-compilers/d5dbd3ae62032074652c26f5290cf1b40c669dcf/Screenshot3.png -------------------------------------------------------------------------------- /Screenshot4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahoppen/introduction-to-compilers/d5dbd3ae62032074652c26f5290cf1b40c669dcf/Screenshot4.png --------------------------------------------------------------------------------