├── CONTROL-FLOW.md ├── LICENSE ├── README.md ├── abbreviations ├── parser.md └── scanner.md /CONTROL-FLOW.md: -------------------------------------------------------------------------------- 1 | # Sections 2 | The main https://github.com/Microsoft/TypeScript/tree/master/src/compiler 3 | 4 | scanner -> parser -> (binder,checker) -> emitter 5 | 6 | `parser.ts` / `createProgram` (and then the program) seems to be the main orchestrator in the compiler. 7 | 8 | # CURRENT 9 | ## Parser 10 | https://github.com/basarat/typescript-compiler-docs/blob/master/parser.md 11 | 12 | ## Scanner 13 | We have a single scanner `const scanner = createScanner(ScriptTarget.Latest, /*skipTrivia*/ true);` in the parser that is shared (for performance) and muated using `setText` and `setScriptTarget`. 14 | 15 | 16 | # LEGACY 17 | 18 | # Call Order 19 | 20 | `tc.ts` `executeCommandLine` (also inside this function `parseCommandLine` from `commandLineParser.ts`) 21 | 22 | will call `createProgram` in parser. 23 | 24 | ## createProgram (parser) 25 | `createProgram` in `parser.ts` The filenames from the command-line are called `rootNames` here. 26 | 27 | The parser initiates the scanner on each of the files in `rootName` + `lib.d.ts`, using `processRootFile` (> `processSourceFile` > `findSourceFile` > `host.getSourceFile`(which is inside `ts.ts` based on `host` and used to simply read the content) > calls back into `createSourceFile` inside the parser. 28 | 29 | 30 | ## createSourceFile (parser) 31 | `createSourceFile` in the parser will create the scanner on the found text using `createScanner`. Also it has a bunch of utility (nested) functions e.g. `processReferenceComments` 32 | 33 | ## createScanner (scanner) 34 | Creates the scanner. A new one is created for each file that needs to be scanned (including `lib.d.ts`) 35 | 36 | ## emitFiles (emitter) 37 | The main function for emiting the output. Called from the `checker` (which itself is a part of the `program`). 38 | See `tc.ts` : `program.getTypeChecker().emitFiles()` (an alias to `invokeEmitter` in `checker.ts` which simply calls `emitFiles` in `emitter.ts`) 39 | 40 | 41 | # Type Checker 42 | ## createTypeChecker 43 | Called from `parser` specifically `createProgram`, and stored on the `program` (`getTypeChecker`). 44 | 45 | Does its work on a call to `checker.getDiagnostics` (called from `tc.ts`) 46 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 Basarat Ali Syed 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | typescript-compiler-docs 2 | ======================== 3 | 4 | *UPDATE* : we have offical docs now! : https://github.com/Microsoft/TypeScript/wiki/Using-the-Compiler-API 5 | 6 | A work in progress documentation of how the TypeScript compiler works. 7 | 8 | # Running tests 9 | Best way I have found is to : 10 | ``` 11 | ./node_modules/mocha/bin/mocha -R dot -g nodeModules --colors -t 20000 built/local/run.js 12 | ``` 13 | The official greping `jake runtests test=nodeModules` didn't work (though `test=project` does work). 14 | 15 | Also see : https://github.com/Microsoft/TypeScript/blob/master/CONTRIBUTING.md 16 | 17 | **Baselines** 18 | And use `jake diff` with beyond compare. 19 | `set DIFF=C:\Program Files (x86)\Beyond Compare 3\BComp.exe` 20 | Its just a folder compare between baselines/local and baselines/reference 21 | 22 | # Useful PRs with Insight 23 | 24 | ## Missing try block with a catch / finally 25 | https://github.com/Microsoft/TypeScript/pull/262/files 26 | **Parser** : Gives information about the `isStatement` function. If that function returns then `parseStatement` will get called which must return *some* type of `Statement`. In this case we are going to return a `TryStatement` from a utility `parseCatchOrFinallyBlocksMissingTryStatement` function. These utility (`parse-`) functions apparently prepare the AST for the emitter by creating (`createNode` function) and finishing Nodes (`finishNode` function). 27 | 28 | ## Parsing logic for Object Literals 29 | https://github.com/Microsoft/TypeScript/pull/273/files 30 | **Parser** : `parseObjectLiteral` is used to parse object literals. `parseDelimitedList` is used to parse the properties of object literals. 31 | 32 | ## Reference tag lookup 33 | https://github.com/Microsoft/TypeScript/pull/365/files Changed to load extensionless reference tags (`.d.ts` and `.ts`) 34 | 35 | ## Fast compilation on watch 36 | https://github.com/Microsoft/TypeScript/pull/324 37 | Basically create a copy of the `program` and update it based on changes. Mostly inside `tc.ts` the `recompile` function. 38 | 39 | 40 | ## Language Service 41 | 42 | ### Initial port of the language service 43 | https://github.com/Microsoft/TypeScript/pull/303 44 | 45 | ### Pull model for contextual types 46 | https://github.com/Microsoft/TypeScript/pull/330 47 | `getContextualType` function by anders 48 | 49 | # Emitter 50 | ## The class emit 51 | https://github.com/Microsoft/TypeScript/pull/331/files 52 | change the order of `this` emit in class body. `emitClassDeclaration` 53 | 54 | ## Emitter / checker for `.d.ts` files 55 | https://github.com/Microsoft/TypeScript/pull/414 56 | Declaration file needs to always emit typeof function/static function instead of emitting signature 57 | 58 | ## File extension for emit different from '.js' 59 | Ability to modify the output filename extension https://github.com/Microsoft/TypeScript/pull/425 60 | 61 | # Checker 62 | ## Change the order in which the functions are checked in overloads 63 | https://github.com/Microsoft/TypeScript/pull/378 64 | ```ts 65 | interface A { (x: string): void } 66 | interface B { (x: 'foo'): string } 67 | var b: B; 68 | b('foo') // <- here overloads should be processed as [(x:'foo'): string, (x: string): void] 69 | ``` 70 | 71 | # Tutorials 72 | **hacking it to add a documentation emitter** http://www.codeproject.com/Articles/635280/TypeScript-Compiler-Documentation-Output 73 | 74 | 75 | ## More language service 76 | An overview of a file in terms of lexical structure : https://github.com/Microsoft/TypeScript/issues/534 77 | 78 | # Various Classifer 79 | https://github.com/Microsoft/TypeScript/issues/1477#issuecomment-66904683 80 | 81 | # Walking the syntax Tree 82 | 83 | Uses the `ts.forEachChild` exported from the `parser`. Helpful information from [this comment](https://github.com/Microsoft/TypeScript/issues/254#issuecomment-68940929). Note how the `Node` is augmented by `services` [here](https://github.com/Microsoft/TypeScript/blob/6f6c46a99f446144702bb324f6b50d94a000a690/src/services/services.ts#L14) 84 | 85 | Demo : http://blog.ctaggart.com/2015/01/typescript-14-ast-from-nodejs.html 86 | 87 | # Things on nodes only get written once 88 | https://github.com/Microsoft/TypeScript/issues/1514#issuecomment-69105512 89 | 90 | # Some Terminal nodes are not in the AST 91 | They can be "rehydrated" on demand using the sourcetext by calling `getChildren()` https://github.com/Microsoft/TypeScript/issues/1514#issuecomment-69118975 92 | 93 | # Formatting your documents 94 | In the language service https://github.com/Microsoft/TypeScript/issues/1651#issuecomment-69684066 95 | * `getFormattingEditsForRange` 96 | * `getFormattingEditsAfterKeystroke` 97 | * `getFormattingEditsForDocument` 98 | for `getFormattingEditsForDocument` once you've gotten the appropriate edits ranges, you can easily apply them in reverse and fix up the original source text. 99 | 100 | Once you have them you can apply them in reverse : https://github.com/Microsoft/TypeScript/issues/1651#issuecomment-69877863 101 | ```ts 102 | function formatCode(orig: string, changes: TextChange[]): string { 103 | var result = orig; 104 | 105 | for (var i = changes.length - 1; i >= 0; i--) { 106 | var change = changes[i]; 107 | var head = result.slice(0, change.span.start); 108 | var tail = result.slice(change.span.start + change.span.length) 109 | result = head + change.newText + tail; 110 | } 111 | 112 | return result; 113 | } 114 | ``` 115 | 116 | Demo : http://blog.ctaggart.com/2015/01/format-typescript-with-v14-language.html 117 | There is also smart indentation alone: https://github.com/Microsoft/TypeScript/issues/1754 118 | 119 | # Support JSX 120 | You can convert JSX to JavaScript using the API from : https://www.npmjs.com/package/react-tools 121 | JSX Specification : http://facebook.github.io/jsx/ 122 | How to augment: https://github.com/facebook/react/issues/759#issuecomment-50529015 123 | 124 | # Skipped tokens by the new foreachchild AST traversal 125 | https://github.com/Microsoft/TypeScript/issues/1728#issuecomment-70592200 126 | 127 | # Program vs. Language Service 128 | `languageService` and `program` are very similar in principal. Which you use depends on your scenario. 129 | 130 | `program` is great if you just want emit / linting, the os related features can be provided with a `compilerHost` generated with a simple call to `createCompilerHost`. Examples : https://github.com/Microsoft/TypeScript/wiki/Using-the-Compiler-API#a-minimal-compiler and https://github.com/Microsoft/TypeScript/wiki/Using-the-Compiler-API#traversing-the-ast-with-a-little-linter 131 | 132 | `languageService` is great if want something like a full blown IDE support. 133 | 134 | More on the difference: 135 | 136 | https://github.com/Microsoft/TypeScript/issues/1786#issuecomment-71567975 137 | 138 | > a singe LanguageService object does not map to a single Program object, it actually maps to a set of them. A Program is an immutable object, the LanguageService on the other hand handles changes. under the covers, the LanguageService, on every change, creates a new Program instance to map to the current context, but uses the previous Program instance's unchanged SourceFiles to save computation. If the LanguageService was just an extension over Program, you would need to create a new LanguageService object on every change. 139 | 140 | 141 | # Expand the list of files knows by language service 142 | `LanguageService.getProgram().getSourceFiles()` More https://github.com/Microsoft/TypeScript/commit/9628191a1476bc0dbdb28bfd30b840656ffc26a3#commitcomment-9490325 143 | 144 | # Langauge service is just a wrapper on `program` and a thin one : 145 | 146 | e.g. [`getQuickInfoAtPosition`](https://github.com/Microsoft/TypeScript/blob/master/src/services/services.ts#L3195) and [`getCompletionsAtPosition`](https://github.com/Microsoft/TypeScript/blob/master/src/services/services.ts#L2311) https://github.com/Microsoft/TypeScript/issues/2105#issuecomment-75417304 147 | 148 | 149 | # Useful blog post on using `program` 150 | http://www.scottlogic.com/blog/2015/01/20/typescript-compiler-api.html 151 | 152 | 153 | # Why the compielr AST is different from Mozilla AST 154 | https://github.com/Microsoft/TypeScript/issues/1690#issuecomment-77713066 155 | 156 | # Formatting JSX 157 | https://github.com/Microsoft/TypeScript/pull/4330/files `smartformatter.ts` + test 158 | 159 | # Incremental parsing call stack insight 160 | https://github.com/Microsoft/TypeScript/issues/7310 161 | about `currentNode` and `canReuseNode` https://github.com/Microsoft/TypeScript/blob/release-1.8/src/compiler/parser.ts#L1463 162 | 163 | # How to tell a const / let declaration 164 | https://github.com/Microsoft/TypeScript/issues/22681#issuecomment-374002621 165 | -------------------------------------------------------------------------------- /abbreviations: -------------------------------------------------------------------------------- 1 | # ASI 2 | Automatic semicolon insertion 3 | -------------------------------------------------------------------------------- /parser.md: -------------------------------------------------------------------------------- 1 | # Parser 2 | 3 | > Starting from a set of sources, and following the productions of the language grammar, to generate an Abstract Syntax Tree (AST). 4 | 5 | We have a single parser (as a module `ts.Parser`) as a namespace module. This gets reinitilized using `initializeState` function which also resets the internal *single `const`* `scanner`. 6 | 7 | ## Entry 8 | Main entry point into the parser is `createSourceFile` (called from `program`) 9 | 10 | This in turn calls `parseSourceFile` which in turn just initializes the scanner and works through (local function `parseSourceFileWorker`) the file contents. 11 | 12 | ## Managing the scanner 13 | `initializeState` resets the scanner (and does other cleanup). 14 | 15 | 16 | ## The heart of the parser 17 | `parseSourceFileWorker` has the following line (notice `parseStatement`): 18 | 19 | ```ts 20 | sourceFile.statements = parseList(ParsingContext.SourceElements, parseStatement); 21 | ``` 22 | 23 | This starts the main recursive decent into the syntax tree. 24 | 25 | There are a bunch of `parse*` functions (e.g. `parseSwitchStatement`) that do the heavy lifting. They all lead up to `parseStatement` at some point. Many of these call `parseStatement` themselves. 26 | 27 | 28 | # DEMO 29 | ## PR for JSX / `as` 30 | https://github.com/Microsoft/TypeScript/pull/3564 31 | Specifically : https://github.com/RyanCavanaugh/TypeScript/commit/556cb70c1d0de4d450baadf48279b7f0ca3d954e and https://github.com/RyanCavanaugh/TypeScript/commit/a4045e539b4f115f854d3cb3351089e62a59f3f0 32 | -------------------------------------------------------------------------------- /scanner.md: -------------------------------------------------------------------------------- 1 | Main entry point : the `createScanner` function 2 | --------------------------------------------------------------------------------