└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # ECMAScriptParser 2 | 3 | **Champions**: Finding one... 4 | 5 | **Author**: Jack Works 6 | 7 | **Stage**: N/A 8 | 9 | This proposal describes adding an ECMAScriptParser to JavaScript. Just like [DOMParser](http://mdn.io/DOMParser) in HTML and [Houdini's parser API in CSS](https://github.com/WICG/CSS-Parser-API/blob/master/README.md). 10 | 11 | ## The problem and opportunity 12 | 13 | ### Usage A: Check if a piece of code is syntax-correct 14 | 15 | Some "feature detection" (See: https://github.com/Tokimon/es-feature-detection/blob/master/syntax/es2019.json) use `eval` to check if some syntax-level feature is available. Like 16 | 17 | ```js 18 | try { 19 | eval('async function x() {}') 20 | asyncFunctionSupported = true 21 | } catch {} 22 | ``` 23 | 24 | Or in some other scenarios, we just want to check if a piece of code is syntax-correct instead of `eval` it. 25 | 26 | ## Usage B: Sandbox 27 | 28 | Some sandbox proposal/libraries do need an AST transformer to transform or reject the dangerous JS code. 29 | 30 | A sandbox proposal, [realms](https://github.com/tc39/proposal-realms/), it's [polyfill](https://github.com/Agoric/realms-shim) is using RegExp to [reject "HTML style comments"](https://github.com/Agoric/realms-shim/blob/025d975da12c8033022c271e5d99a3810066dfa4/src/sourceParser.js#L19) and [reject ESModules](https://github.com/Agoric/realms-shim/blob/025d975da12c8033022c271e5d99a3810066dfa4/src/sourceParser.js#L55). Using RegExp is very likely to make false positives on valid codes. 31 | 32 | But if they want to reduce false positives, they have to load a babel, typescript or whatever what parser into the library and run it to transform the code into the safe pattern. 33 | 34 | ### Due to the lack of built-in parser, they imported ... 35 | 36 | - Babel as a parser in realms-shim (https://github.com/Agoric/realms-shim/pull/15) 37 | - TypeScript Compiler as a parser in (https://github.com/Agoric/realms-shim/issues/47) 38 | - TypeScript as a transformer to transform ESModule into SystemJS (https://github.com/Jack-Works/loader-with-esmodule-example/) 39 | 40 | ### False positive rejection examples: 41 | 42 | - [#34: rejectSomeDirectEvalExpressions is too aggressive](https://github.com/Agoric/realms-shim/issues/34) 43 | - [#37: rejectHtmlComments throws on bignumber.js comment](https://github.com/Agoric/realms-shim/issues/37) 44 | - [#39: Any code with import expression inside string literal or comment throws error](https://github.com/Agoric/realms-shim/issues/39) 45 | - ... and so on 46 | 47 | ## Usage C: Join the parser step 48 | 49 | This part may be hard to accomplish. Need to discuss if it is needed or even possible. 50 | 51 | Just like CSS Houdini's parser can parse CSS in a programmable way, this ECMAScript parser may let developer handle the step of parsing the source code in a programmable way. 52 | 53 | Don't know what the API should be like, just for example: 54 | 55 | ```js 56 | const JSXElement = new ECMAScriptParser.Grammar('JSXElement', [ 57 | JSXSelfClosingElement, 58 | new ECMAScriptParser.Grammar( 59 | JSXOpeningElement, 60 | { 61 | type: JSXChildren, 62 | optional: true 63 | }, 64 | JSXClosingElement 65 | ) 66 | ]) 67 | const jsxParser = new ECMAScriptParser() 68 | const primaryExpression = parser.getGrammar('PrimaryExpression') 69 | primaryExpression.extends(JSXElement) 70 | jsxParser.parse(`const expr = `) 71 | ``` 72 | 73 | # Goal 74 | 75 | - Generate AST from source code 76 | - Generate source code from an AST 77 | - [Maybe] a built-in AST walker to replace AST Nodes on demand 78 | - (If AST is not plain object,) provide a way to construct new AST Nodes. 79 | - [Maybe] a set of API that extends ECMAScript Parser (like we can create a special parser for [JSX](https://facebook.github.io/jsx/)). 80 | 81 | # API design 82 | 83 | ```ts 84 | class ECMAScriptParser { 85 | parse(source: string): ECMAScriptAST 86 | 87 | compile(ast: ECMAScriptAST): string 88 | 89 | static visitChildNodes(mapFunction: (beforeTransform: ECMAScriptAST) => ECMAScriptAST): ECMAScriptAST 90 | 91 | // [Maybe] 92 | addGrammar(gr: Grammar): this 93 | getGrammar(grName: string): Grammar | null 94 | replaceGrammar(gr: Grammar): this 95 | deleteGrammar(gr: Grammar): this 96 | removeGrammar(gr: Grammar): this 97 | static Grammar = class Grammar { 98 | // [wait for discussion] 99 | } 100 | } 101 | ``` 102 | 103 | ## What shape is type ECMAScriptAST? 104 | 105 | wait for discussion 106 | 107 | see also: 108 | 109 | - https://github.com/tc39/proposal-binary-ast 110 | - https://github.com/rricard/proposal-const-value-types 111 | 112 | ## Example usage 113 | 114 | ### Used to check if new Syntax is supported 115 | 116 | ```js 117 | try { 118 | new ECMAScriptParser().parse(` 119 | const res = await fetch(jsonService) 120 | case (res) { 121 | when {status: 200, headers: {'Content-Length': s}} -> 122 | console.log(\`size is \${s}\`), 123 | when {status: 404} -> 124 | console.log('JSON not found'), 125 | when {status} if (status >= 400) -> { 126 | throw new RequestError(res) 127 | }, 128 | }`) 129 | } catch { 130 | // https://github.com/tc39/proposal-pattern-matching 131 | console.log('Your browser does not support Pattern Matching') 132 | } 133 | ``` 134 | 135 | ### Use in Realms API 136 | 137 | ```js 138 | const parser = new ECMAScriptParser() 139 | const secure = new Realms({ 140 | transforms: [ 141 | { 142 | rewrite: context => { 143 | const ast = parser.parse(context.src) 144 | ECMAScriptParser.visitChildNodes(node => { 145 | if (node.kind === ECMAScriptParser.SyntaxKind.WithStatement) { 146 | throw new SyntaxError('with statement is not supported') 147 | } else if (node.kind === ECMAScriptParser.SyntaxKind.ImportDeclaration) { 148 | return ECMAScriptParser.createImportDeclaration( 149 | node.decorators, 150 | node.modifiers, 151 | node.importClause, 152 | ECMAScriptParser.createStringLiteral( 153 | '/safe-import-transformer.js?src=' + node.moduleSpecifier.toString() 154 | ) 155 | ) 156 | } 157 | return node 158 | }) 159 | return context 160 | } 161 | } 162 | ] 163 | }) 164 | 165 | secure.evaluate(`with (window) {}`) 166 | // SyntaxError: with statement is not supported 167 | source.evaluate(`import x from './y.js'`) 168 | // will evaluate: `import x from '/safe-import-transformer.js?src=./y.js'` 169 | ``` 170 | 171 | ### [Maybe] Using in enhance ECMAScript's syntax 172 | 173 | ```js 174 | import { enhanceParser, jsxCompiler } from 'react-experimental-jsx-parser-in-browser' 175 | /* 176 | JSX extends the PrimaryExpression in the ECMAScript 6th Edition (ECMA-262) grammar: 177 | PrimaryExpression : 178 | JSXElement 179 | JSXFragment 180 | */ 181 | const jsxParser = enhanceParser(new ECMAScriptParser()) 182 | const ast = jsxParser.parse(`const link = `) 183 | const code = jsxParser.compile( 184 | jsxCompiler({ 185 | jsxFactory: 'React' 186 | }) 187 | ) 188 | console.log(code) 189 | // const link = React.createElement('a', {}) 190 | ``` 191 | --------------------------------------------------------------------------------