├── .gitignore ├── README.md ├── grammar.txt ├── interpreter.js ├── lexer.js ├── package.json ├── parser.js ├── run.js ├── sample.txt ├── test ├── lexerTests.js └── parserTests.js └── util.js /.gitignore: -------------------------------------------------------------------------------- 1 | node_modules/ 2 | 3 | sample_tree.json -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Javascript DSL Example 2 | 3 | An example of an external DSL (Domain specific language) written in Javascript. This is a pretty small DSL, which let's me write the lexer and the parser by hand. If the language was any larger, writing it manually could start to get out of hand. 4 | 5 | This DSL allows a user to write test cases around a JSON API. It will call an endpoint and run assertions over types and data. This does not support any type of authentication at the moment, so probably fairly limited in it's use. 6 | 7 | ### Running the example 8 | 9 | First, clone the repo to your machine. Then install NPM packages. 10 | 11 | npm install 12 | 13 | Then you can run the sample code. 14 | 15 | node run.js 16 | 17 | ## The Language 18 | 19 | The language is extemely simple, but has a strict structure to keep the parser simple. Here is some sample code. 20 | 21 | test "https://api.github.com/users/davesters" { 22 | should be object 23 | size should equal 30 24 | 25 | fields { 26 | "login" should be string 27 | id should equal 439674 28 | name should equal "David Corona" 29 | site_admin should not equal true 30 | } 31 | } 32 | 33 | Most whitespace does not matter one bit, as long as there are spaces between words. You could actually put this entire thing on one line and it would still work (minifiable FTW!). 34 | 35 | Note: Strings must be enclosed with double quotes ("). 36 | 37 | ### Test block 38 | 39 | Everything is run inside of a `test` block. It takes a string argument, which is the URL of the API endpoint to call. Test blocks contain any number of `should` statements, `fields` blocks, or `each` blocks. A program may contain more than one `test` block, but they cannot be nested. 40 | 41 | ### Scopes 42 | 43 | A quick side note about scopes. This DSL has a concept of scopes, or maybe you can call them contexts. A scope would be the current block that assertions are being run in. 44 | 45 | For example. In the main body of a `test` block, the current scope is the root of the JSON object or array that is returned by the API call. `fields` or `each` blocks can change the scope to another node that is inside the current scope. We will see these blocks shortly. 46 | 47 | ### Should statement 48 | 49 | A should statement is an assertion upon a JSON object, array or field. It has the format of: 50 | 51 | [field name] should [not] (be [type] | equal [value]) 52 | 53 | * `[field name]` is optional. If it exists, it denotes which field in the current scope is being tested. If it does not exist, it applies to the object or array of the current scope. 54 | * `not` is an optional operator which tests for anything not true. 55 | * `be [type]` is a test that a field is a specific data type. (i.e. number, string, boolean, object, array, true or false) 56 | * `equal [value]` is a test that a field's value matches the specified value. 57 | 58 | Some sample `should` statements: 59 | 60 | should be object // Tests if the current item in scope is an object 61 | size should equal 30 // Tests if the size of current item in scope is 30 62 | "login" should be string // Tests if the login field is a string type 63 | id should equal 439674 // Tests if the id field equals the number 439674 64 | name should equal "David Corona" // Tests the name field equals "David Corona" 65 | site_admin should not equal true // Tests the site_admin field is not true 66 | 67 | ### Fields block 68 | 69 | A fields block lets us step into an object in our current scope, and make that object our new scope. Any `should` statements inside of a `fields` block will apply to this new scope. Field blocks can also contain other nested `field` or `each` blocks. It has the format of: 70 | 71 | fields [field name] { } 72 | 73 | Note that the field name of a field block is optional. If it is not specified, the `should` statements inside the `field` block will apply to fields in the object that is currently in scope. Note that it must be an object that is in scope. Using a `field` block without a field name is not that useful when you are inside of an object. You could just put the `should` statements inside the current block. 74 | 75 | ### Each block 76 | 77 | An each block lets us loop over objects inside an array and run a set of tests against those objects. Each iteration of the loop will run within the scope of the current object in the array. Each blocks can contain other nested `field` or `each` blocks. It has a format of: 78 | 79 | each [field name] { } 80 | 81 | Note that the field name of an each block is optional. If it is not specified, it will loop over the objects in the array currently in scope. Note that it must be an array that is in scope. 82 | 83 | ### Size keyword 84 | 85 | There is a special `size` keyword that can be used instead of the field name in a `should` statement. This let's you assert over the size of the current item in scope. If the scope is an object, the number of fields. If the scope is an array, the number of items in the array. 86 | 87 | Example: 88 | 89 | size should equal 30 90 | 91 | ## The Grammar 92 | 93 | This has a pretty simple grammar with a small amount of productions. 94 | 95 | Z => S 96 | S => test t { T } 97 | T => fields F { T } 98 | T => each F { T } 99 | T => A should BCE 100 | T => e 101 | A => size | i | t | e 102 | B => not | e 103 | C => be | equal 104 | E => object | array | string | number | boolean | true | false | t 105 | F => i | t | e 106 | 107 | ### Keywords 108 | 109 | These are the keywords, or reserved words, in this DSL. If you want to use these words as field names, you should put quotes around them as if they were strings. 110 | 111 | test 112 | fields 113 | size 114 | each 115 | object 116 | array 117 | string 118 | number 119 | boolean 120 | true 121 | false 122 | should 123 | not 124 | be 125 | equal -------------------------------------------------------------------------------- /grammar.txt: -------------------------------------------------------------------------------- 1 | Keywords: 2 | 3 | test 4 | fields 5 | size 6 | each 7 | object 8 | array 9 | string 10 | number 11 | boolean 12 | true 13 | false 14 | should 15 | not 16 | be 17 | equal 18 | 19 | 20 | Legend: 21 | 22 | i = Identifier 23 | t = Literal 24 | e = null 25 | 26 | 27 | Productions: 28 | 29 | Z => S 30 | S => test t { T } 31 | T => fields F { T } 32 | T => each F { T } 33 | T => A should BCE 34 | T => e 35 | A => size | i | t | e 36 | B => not | e 37 | C => be | equal 38 | E => object | array | string | number | boolean | true | false | t 39 | F => i | t | e -------------------------------------------------------------------------------- /interpreter.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | let got = require('got'); 4 | let sprintf = require('util').format; 5 | let parser = require('./parser'); 6 | let colors = require('colors'); 7 | 8 | // Keep track of errors so they can be printed at the end 9 | let errors = []; 10 | 11 | // Keep track of number of tests to print at the end 12 | let testCount = 0; 13 | 14 | // Keep track of how many spaces to indent when printing assertions 15 | let indent = 2; 16 | 17 | function addException(message, line) { 18 | errors.push({ 19 | msg: message, 20 | line: line, 21 | }); 22 | } 23 | 24 | function printAssert(msg, not, failed) { 25 | let test = sprintf(msg.replace(/%%/g, '%'), (not) ? 'not ' : ''); 26 | 27 | if (!failed) { 28 | testCount++; 29 | console.log(' '.repeat(indent) + colors.green.bold(' ✔'), colors.white(test)); 30 | } else { 31 | console.log(' '.repeat(indent) + colors.red.bold(' x', test)); 32 | } 33 | } 34 | 35 | module.exports = (input) => { 36 | // Pass the input to the parser to get the parse tree. 37 | let tree = parser(input); 38 | 39 | // Lets iterate over each test block 40 | runTests(0, tree.children); 41 | 42 | return tree; 43 | }; 44 | 45 | function runTests(index, tests) { 46 | // If there are no more test blocks, then print final messages and exit. 47 | if (index > tests.length - 1) { 48 | printSummary(); 49 | process.exit(errors.length > 0 ? 1: 0); 50 | } 51 | 52 | let url = tests[index].left.literal; 53 | 54 | // Make the request to the test URL 55 | console.log('\n' + colors.cyan.bold(url)); 56 | got(url, { json: true, timeout: 5000 }) 57 | .then(response => { 58 | handleBlock(tests[index].right, response.body); 59 | runTests(index + 1, tests); 60 | }) 61 | .catch(error => { 62 | console.log(error); 63 | }); 64 | } 65 | 66 | // Generic handler for a block of statements 67 | function handleBlock(block, data) { 68 | block.children.forEach(el => { 69 | handleBlockChild(el, data); 70 | }); 71 | 72 | return true; 73 | } 74 | 75 | function handleBlockChild(child, data) { 76 | switch (child.type) { 77 | case 'shouldStmt': 78 | handleShould(child, data); 79 | break; 80 | case 'fieldsStmt': 81 | indent += 2; 82 | if (child.left === null) { 83 | console.log(colors.cyan(' '.repeat(indent) + 'fields')); 84 | handleBlock(child.right, data); 85 | } else { 86 | console.log(colors.cyan(' '.repeat(indent) + 'fields - ' + child.left.literal)); 87 | handleBlock(child.right, data[child.left.literal]); 88 | } 89 | indent -= 2; 90 | break; 91 | case 'eachStmt': 92 | let childData = data; 93 | if (child.left !== null) { 94 | console.log(colors.cyan(' '.repeat(indent) + 'each - ' + child.left.literal)); 95 | childData = data[child.left.literal]; 96 | } else { 97 | console.log(colors.cyan(' '.repeat(indent) + 'each')); 98 | } 99 | 100 | indent += 2; 101 | // Iterate over each child in the array and run the statements for each 102 | childData.forEach((c, index) => { 103 | console.log(colors.cyan(' '.repeat(indent) + 'child ' + (index + 1))); 104 | indent += 2; 105 | handleBlock(child.right, c); 106 | indent -= 2; 107 | }); 108 | indent -= 2; 109 | break; 110 | } 111 | } 112 | 113 | function handleShould(should, data) { 114 | let childNum = 0; 115 | let el = should.right.children[childNum]; 116 | let not = el.literal === 'not'; 117 | let actual = data; 118 | 119 | if (not) { 120 | childNum++; 121 | el = should.right.children[childNum]; 122 | } 123 | 124 | if (should.left !== null) { 125 | actual = data[should.left.literal]; 126 | } 127 | 128 | if (el.literal === 'be') { 129 | handleShouldBe(should.right.children[childNum + 1].literal, actual, not, should.line); 130 | } else if (el.literal === 'equal') { 131 | handleShouldEqual(should.right.children[childNum + 1].literal, should.left.literal, data, not, should.line); 132 | } 133 | } 134 | 135 | function handleShouldBe(expected, actual, not, line) { 136 | let failed = false; 137 | 138 | switch (expected) { 139 | case 'object': 140 | if (not && actual instanceof Object) { 141 | addException('Expected not to find Object', line); 142 | failed = true; 143 | } 144 | if (!not && !(actual instanceof Object)) { 145 | addException('Expected to find Object but found ' + Object.prototype.toString.call(actual), line); 146 | failed = true; 147 | } 148 | break; 149 | case 'array': 150 | if (not && actual instanceof Array) { 151 | addException('Expected not to find Array', line); 152 | failed = true; 153 | } 154 | if (!not && !(actual instanceof Array)) { 155 | addException('Expected to find Array but found ' + Object.prototype.toString.call(actual), line); 156 | failed = true; 157 | } 158 | break; 159 | default: 160 | if (not && typeof actual === expected) { 161 | addException('Expected not to find ' + expected, line); 162 | failed = true; 163 | } 164 | if (!not && typeof actual !== expected) { 165 | addException('Expected to find ' + expected + ' but found ' + Object.prototype.toString.call(actual), line); 166 | failed = true; 167 | } 168 | break; 169 | } 170 | 171 | printAssert(sprintf('%s should %%sbe %s', Object.prototype.toString.call(actual), expected), not, failed); 172 | } 173 | 174 | function handleShouldEqual(expected, property, data, not, line) { 175 | let failed = false; 176 | let test = ''; 177 | 178 | if (property === 'size') { 179 | var len = Object.keys(data).length; 180 | 181 | if (not && len === expected) { 182 | addException(sprintf('Expected size not to be %d', expected), line); 183 | failed = true; 184 | } 185 | if (!not && len !== expected) { 186 | addException(sprintf('Expected size to be %d, but found %d', expected, len), line); 187 | failed = true; 188 | } 189 | 190 | test = sprintf('size should %%sbe %s', expected); 191 | } else { 192 | let actualProperty = data[property]; 193 | if (typeof data[property] === 'boolean') { 194 | actualProperty = data[property].toString(); 195 | } 196 | 197 | if (not && actualProperty === expected) { 198 | addException(sprintf('Expected %s not to equal %s', property, expected), line); 199 | failed = true; 200 | } 201 | if (!not && actualProperty !== expected) { 202 | addException(sprintf('Expected %s to equal %s but found %s', property, expected, actualProperty), line); 203 | failed = true; 204 | } 205 | 206 | test = sprintf('%s should %%sequal %s', property, expected); 207 | } 208 | 209 | printAssert(test, not, failed); 210 | } 211 | 212 | function printSummary() { 213 | if (errors.length > 0) { 214 | printErrors(); 215 | } 216 | 217 | console.log(colors.yellow.bold(sprintf('\n%d of %d Test(s) passed with %d error(s)', testCount, testCount + errors.length, errors.length))); 218 | } 219 | 220 | function printErrors() { 221 | console.log('\n' + colors.red.bold(errors.length, 'error(s)')); 222 | errors.forEach((err, index) => { 223 | console.log(' ', colors.white.bold(index + 1) + '.', '\t', colors.red.bold(sprintf('%s (line %d)', err.msg, err.line))) 224 | }); 225 | } 226 | -------------------------------------------------------------------------------- /lexer.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | let _ = require('lodash'); 4 | let util = require('./util'); 5 | let os = require('os'); 6 | 7 | function LexerException(message) { 8 | this.name = 'LexerException'; 9 | this.message = message; 10 | } 11 | LexerException.prototype = Object.create(Error.prototype); 12 | LexerException.prototype.constructor = LexerException; 13 | 14 | module.exports = (input) => { 15 | if (!input || input.trim() === '') { 16 | throw new LexerException('No input provided'); 17 | } 18 | 19 | // List of valid found tokens to return 20 | let tokens = []; 21 | 22 | // Tracks the current line we are on. 23 | let line = 0; 24 | 25 | // Tracks the current character we are on for a line. 26 | let pointer = 0; 27 | 28 | // Used to build up full tokens before adding to list. 29 | let token = ''; 30 | 31 | // Current state in the lex table state machine. 32 | let state = 1; 33 | 34 | // List of lines in the input 35 | let lines = input.split(os.EOL).map((l) => l + os.EOL + ' '); 36 | 37 | // Function to add a token to the list of found valid tokens 38 | let addToken = (type, literal, backtrack) => { 39 | let charPos = pointer; 40 | 41 | // Need to adjust char position if token is a curly brace or a string. 42 | if (literal && literal !== '{' && literal != '}') { 43 | charPos -= literal.toString().length; 44 | if (type === 'str') { 45 | charPos--; 46 | } 47 | } 48 | 49 | // Add token to the list of found tokens 50 | tokens.push({ 51 | type: type, 52 | literal: literal, 53 | line: line + 1, 54 | char: charPos, 55 | }); 56 | token = ''; 57 | state = 1; 58 | 59 | // If this token requires backtracking, then decrement the pointer 60 | // so we can continue at the right place. 61 | if (backtrack) { 62 | pointer--; 63 | } 64 | }; 65 | 66 | // Loop over every character in the input 67 | do { 68 | let currentChar = lines[line][pointer]; 69 | 70 | // Get the matching column for this char from the lex table 71 | // Runs regex match over each column. Probably slow, but more concise. 72 | let column = _.findIndex(util.lexTable[0], (regex) => { 73 | return currentChar.match(regex); 74 | }); 75 | 76 | // We did not find any matching states, throw an exception. 77 | if (column < 0) { 78 | throw new LexerException('Unidentified character "' + currentChar + 79 | '" on line: ' + (line + 1) + ', char: ' + (pointer + 1)); 80 | } 81 | 82 | // Change to new state found in the lex table. 83 | state = util.lexTable[state][column]; 84 | 85 | pointer++; 86 | 87 | // Check if we have hit a finishing state and act accordingly. 88 | switch (state) { 89 | case 0: // This is an invalid finishing state. 90 | throw new LexerException('Unexpected character "' + currentChar + 91 | '" on line: ' + (line + 1) + ', char: ' + pointer); 92 | case 1: // Whitespace. Remain in state 1 93 | break; 94 | case 3: // End Identifier 95 | // Identifiers are either keywords or arbitrary identifiers. 96 | // Identifiers require backtrack so we can process the next char correctly. 97 | if (util.keywords.indexOf(token.toLowerCase()) != -1) { 98 | addToken('kwd', token.toLowerCase(), true); 99 | } else { 100 | addToken('id', token, true); 101 | } 102 | break; 103 | case 5: // End Number 104 | case 10: // End Decimal 105 | // Numbers require backtrack so we can process the next char correctly. 106 | addToken('num', parseFloat(token, 2), true); 107 | break; 108 | case 7: // End String 109 | // Remove the preceding quote before adding. 110 | // We don't need to keep the quotes in the stored token. 111 | token = token.substring(1); 112 | addToken('str', token); 113 | break; 114 | case 12: // Found start block 115 | addToken('op', '{'); 116 | break; 117 | case 13: // End start block 118 | addToken('cl', '}'); 119 | break; 120 | default: // Not a finishing state. Add to the token. 121 | token += currentChar; 122 | } 123 | 124 | // EOL reached. Advance line and reset pointer. 125 | if (pointer >= lines[line].length) { 126 | line++; 127 | pointer = 0; 128 | } 129 | 130 | // EOF reached. Exit the loop. 131 | if (line >= lines.length) { 132 | addToken('eof'); 133 | break; 134 | } 135 | } while (true) 136 | 137 | let tokenIndex = -1; 138 | 139 | // We return an object interface with a 'next' and 'backup' function. 140 | // This will act as a sort of iterator for the parser to consume tokens with. 141 | // 'next()' will return the next token. 142 | // 'backup()' will back up the cursor one position. 143 | return { 144 | 145 | next: () => { 146 | if (tokenIndex > tokens.length) { 147 | throw new LexerException('No more tokens'); 148 | } 149 | 150 | tokenIndex++; 151 | return { 152 | value: tokens[tokenIndex], 153 | done: tokens[tokenIndex].type === 'eof', 154 | }; 155 | }, 156 | 157 | backup: () => { 158 | tokenIndex--; 159 | }, 160 | 161 | }; 162 | }; -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "js-dsl", 3 | "version": "1.0.0", 4 | "repository": { 5 | "type": "git", 6 | "url": "https://github.com/davesters/js-dsl-example" 7 | }, 8 | "engines": { 9 | "node": ">=5.9.0" 10 | }, 11 | "devDependencies": { 12 | "mocha": "^2.2.1", 13 | "should": "^5.1.0" 14 | }, 15 | "dependencies": { 16 | "colors": "^1.1.2", 17 | "got": "^6.3.0", 18 | "lodash": "^3.5.0" 19 | }, 20 | "scripts": { 21 | "test": "mocha test" 22 | } 23 | } 24 | -------------------------------------------------------------------------------- /parser.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | let util = require('./util'); 4 | let sprintf = require('util').format; 5 | let lexer = require('./lexer'); 6 | let equalKeywords = ['object', 'array', 'string', 'number', 'boolean', 'true', 'false']; 7 | 8 | function ParserException(message) { 9 | this.name = 'ParserException'; 10 | this.message = message; 11 | } 12 | ParserException.prototype = Object.create(Error.prototype); 13 | ParserException.prototype.constructor = ParserException; 14 | 15 | module.exports = (input) => { 16 | // Pass the input to the lexer to get all the tokens. Lexer errors will get thrown from within. 17 | let tokens = lexer(input); 18 | 19 | // Call the starting production. The return value will be the final AST. 20 | // Productions sort of recursively call other productions and returning 21 | // their final trees which get added up to the final result. 22 | // We have to pass around the iterator interface tho. 23 | let ast = Z({ 24 | next: () => { 25 | return tokens.next().value; 26 | }, 27 | backup: tokens.backup, 28 | }); 29 | 30 | return ast; 31 | }; 32 | 33 | function throwException(token, message) { 34 | throw new ParserException(sprintf('%s (line: %d, char: %d)', message, token.line, token.char)); 35 | } 36 | 37 | // Z => S 38 | // Entry point to any program 39 | function Z(tokens) { 40 | let node = { 41 | type: 'program', 42 | children: [], 43 | }; 44 | 45 | // Loop over all valid S productions. 46 | do { 47 | node.children.push(S(tokens)); 48 | 49 | // Need to check for EOF here so we don't try to keep going. 50 | let token = tokens.next(); 51 | if (util.isEOF(token)) { 52 | break; 53 | } 54 | 55 | // If the last token was not the EOF, then we need to 56 | // backup because it is the start of another S production. 57 | tokens.backup(); 58 | } while (true); 59 | 60 | return node; 61 | } 62 | 63 | // S => test t { T } 64 | function S(tokens) { 65 | let token = tokens.next(); 66 | 67 | if (!util.isKeyword(token, 'test')) { 68 | throwException(token, 'Expected keyword `test`'); 69 | } 70 | 71 | token = tokens.next(); 72 | if (!util.isStr(token)) { 73 | throwException(token, 'Expected string'); 74 | } 75 | 76 | // Found the start of a test block. 77 | let node = { 78 | type: 'testStmt', 79 | left: { 80 | type: 'strlit', 81 | literal: token.literal, 82 | }, 83 | right: null, 84 | }; 85 | 86 | token = tokens.next(); 87 | if (!util.isOpeningBrace(token)) { 88 | throwException(token, 'Expected token `{`'); 89 | } 90 | 91 | node.right = { 92 | type: 'blockStmt', 93 | children: [], 94 | } 95 | 96 | // Loop over all valid T productions. 97 | do { 98 | node.right.children.push(T(tokens)); 99 | 100 | // Check if we have a closing brace to end the test block. 101 | if (util.isClosingBrace(tokens.next())) { 102 | break; 103 | } 104 | 105 | // If the last token was not a closing brace, then we need to 106 | // backup because it is the start of another T production. 107 | tokens.backup(); 108 | } while (true); 109 | 110 | return node; 111 | } 112 | 113 | // T => fields F { T } 114 | // T => each F { T } 115 | // T => A should BCE 116 | function T(tokens) { 117 | let token = tokens.next(); 118 | 119 | // Check if we are starting a 'fields' or 'each' block. 120 | // We have a separate function for these. 121 | if (util.isKeyword(token, 'fields')) { 122 | return TBlock('fields', tokens); 123 | } else if (util.isKeyword(token, 'each')) { 124 | return TBlock('each', tokens); 125 | } 126 | 127 | // Found the start of a should statement 128 | let node = { 129 | type: 'shouldStmt', 130 | left: null, 131 | right: null, 132 | }; 133 | 134 | // See if we have an optional identifier before our 'should' keyword. 135 | let a = A(token); 136 | if (a) { 137 | node.left = a; 138 | token = tokens.next(); 139 | } 140 | 141 | let shouldArgsNode = { 142 | type: 'args', 143 | children: [], 144 | }; 145 | if (!util.isKeyword(token, 'should')) { 146 | if (a) { 147 | throwException(token, 'Expected to find keyword `should` after a field name. If field name is more than one word, try enclosing in quotes.'); 148 | } 149 | throwException(token, 'Expected keyword `should` or closing `}`'); 150 | } 151 | 152 | // Check if we have an optional 'not' operator. 153 | let b = B(tokens); 154 | if (b) { 155 | shouldArgsNode.children.push(b); 156 | } 157 | 158 | // Get the last bits of the should statement. 159 | shouldArgsNode.children.push(C(tokens)); 160 | shouldArgsNode.children.push(E(tokens)); 161 | 162 | node.right = shouldArgsNode; 163 | node.line = token.line; 164 | return node; 165 | } 166 | 167 | // T => fields F { T } 168 | // T => each F { T } 169 | function TBlock(blockType, tokens) { 170 | let node = { 171 | type: blockType + 'Stmt', 172 | left: F(tokens), // Check if we have an optional identifier 173 | right: null, 174 | }; 175 | let token = tokens.next(); 176 | 177 | if (!util.isOpeningBrace(token)) { 178 | throwException(token, 'Expected token `{`'); 179 | } 180 | 181 | node.right = { 182 | type: 'blockStmt', 183 | children: [], 184 | } 185 | 186 | // Loop over all valid T productions. 187 | // We recursively check for more T productions here as there can be 188 | // multiple nested fields and each blocks containing should statements. 189 | do { 190 | node.right.children.push(T(tokens)); 191 | 192 | // Check if we have a closing brace to end the test block. 193 | if (util.isClosingBrace(tokens.next())) { 194 | break; 195 | } 196 | 197 | // If the last token was not a closing brace, then we need to 198 | // backup because it is the start of another T production. 199 | tokens.backup(); 200 | } while (true); 201 | 202 | return node; 203 | } 204 | 205 | // A => size | i | t | e 206 | function A(token) { 207 | if (util.isKeyword(token, 'size')) { 208 | return { 209 | type: "kwd", 210 | literal: "size", 211 | }; 212 | } else if (util.isIdentifier(token)) { 213 | return { 214 | type: "id", 215 | literal: token.literal, 216 | }; 217 | } else if (util.isStr(token)) { 218 | return { 219 | type: "strlit", 220 | literal: token.literal, 221 | }; 222 | } else { 223 | return null; 224 | } 225 | } 226 | 227 | // B => not | e 228 | function B(tokens) { 229 | let token = tokens.next(); 230 | 231 | if (util.isKeyword(token, 'not')) { 232 | return { 233 | type: "kwd", 234 | literal: "not", 235 | }; 236 | } 237 | 238 | // If we did not find the optional not operator, then we need to 239 | // backup, because it is the next piece of the should statement. 240 | tokens.backup(); 241 | return null; 242 | } 243 | 244 | // C => be | equal 245 | function C(tokens) { 246 | let token = tokens.next(); 247 | 248 | if (util.isKeyword(token, 'be')) { 249 | return { 250 | type: "kwd", 251 | literal: "be", 252 | }; 253 | } 254 | 255 | if (util.isKeyword(token, 'equal')) { 256 | return { 257 | type: "kwd", 258 | literal: "equal", 259 | }; 260 | } 261 | 262 | throwException(token, 'Expected either keyword `be` or `equal` to follow a `should` or a `not`.'); 263 | } 264 | 265 | // E => object | array | string | number | boolean | true | false | t 266 | function E(tokens) { 267 | let token = tokens.next(); 268 | 269 | if (util.isStr(token)) { 270 | return { 271 | type: 'strlit', 272 | literal: token.literal, 273 | } 274 | } 275 | if (util.isNumber(token)) { 276 | return { 277 | type: 'numlit', 278 | literal: token.literal, 279 | } 280 | } 281 | 282 | if (token.type === 'kwd' && equalKeywords.indexOf(token.literal) !== -1) { 283 | return { 284 | type: "kwd", 285 | literal: token.literal, 286 | } 287 | } 288 | 289 | throwException(token, 'Expected to find either a string literal, a number literal, or a value type at the end of a should statement. If the comparison value is more than one word, try enclosing it in quotes.'); 290 | } 291 | 292 | // F => i | t | e 293 | function F(tokens) { 294 | let token = tokens.next(); 295 | 296 | if (util.isStr(token)) { 297 | return { 298 | type: "strlit", 299 | literal: token.literal, 300 | }; 301 | } else if (util.isIdentifier(token)) { 302 | return { 303 | type: "id", 304 | literal: token.literal, 305 | }; 306 | } else { 307 | // If we did not find an optional identifier here, then we need to 308 | // backup, because it is the next piece of the T production. 309 | tokens.backup(); 310 | return null; 311 | } 312 | } -------------------------------------------------------------------------------- /run.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | let fs = require('fs'); 4 | let interpreter = require('./interpreter'); 5 | 6 | let sample = fs.readFileSync(__dirname + '/sample.txt', { 7 | encoding: 'utf8', 8 | }); 9 | 10 | interpreter(sample); -------------------------------------------------------------------------------- /sample.txt: -------------------------------------------------------------------------------- 1 | test "https://api.github.com/users/davesters" { 2 | 3 | should be object 4 | size should equal 30 5 | 6 | fields { 7 | "login" should be string 8 | id should equal 439674 9 | name should equal "David Corona" 10 | site_admin should not equal true 11 | } 12 | } 13 | 14 | test "https://api.github.com/users/davesters/repos" { 15 | 16 | should be array 17 | size should equal 17 18 | 19 | each { 20 | 21 | should be object 22 | size should equal 68 23 | 24 | fields { 25 | private should equal false 26 | 27 | fields owner { 28 | login should be string 29 | id should equal 439674 30 | site_admin should equal false 31 | } 32 | } 33 | 34 | } 35 | 36 | } -------------------------------------------------------------------------------- /test/lexerTests.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | let lexer = require('./../lexer'); 4 | let should = require('should'); 5 | 6 | describe('Lexer', function() { 7 | 8 | it('throws error on no input', function() { 9 | (lexer).should.throw('No input provided'); 10 | }); 11 | 12 | it('returns correct tokens for (Z => test t {)', function() { 13 | var tokens = lexer('test http://test.api {'); 14 | 15 | var token = tokens.next().value; 16 | token.type.should.equal('kwd'); 17 | token.literal.should.equal('test'); 18 | token.line.should.equal(1); 19 | token.char.should.equal(1); 20 | 21 | token = tokens.next().value; 22 | token.type.should.equal('id'); 23 | token.literal.should.equal('http://test.api'); 24 | token.line.should.equal(1); 25 | token.char.should.equal(6); 26 | 27 | token = tokens.next().value; 28 | token.type.should.equal('op'); 29 | token.literal.should.equal('{'); 30 | token.line.should.equal(1); 31 | token.char.should.equal(22); 32 | 33 | token = tokens.next().value; 34 | token.type.should.equal('eof'); 35 | }); 36 | 37 | it('returns correct tokens for (Z => test t {) when t is a string', function() { 38 | var tokens = lexer('test "http://test.api" {'); 39 | 40 | var token = tokens.next().value; 41 | token.type.should.equal('kwd'); 42 | token.literal.should.equal('test'); 43 | token.line.should.equal(1); 44 | token.char.should.equal(1); 45 | 46 | token = tokens.next().value; 47 | token.type.should.equal('str'); 48 | token.literal.should.equal('http://test.api'); 49 | token.line.should.equal(1); 50 | token.char.should.equal(6); 51 | 52 | token = tokens.next().value; 53 | token.type.should.equal('op'); 54 | token.literal.should.equal('{'); 55 | token.line.should.equal(1); 56 | token.char.should.equal(24); 57 | 58 | token = tokens.next().value; 59 | token.type.should.equal('eof'); 60 | }); 61 | 62 | it('returns correct tokens for (S => fields {)', function() { 63 | var tokens = lexer('fields {'); 64 | 65 | var token = tokens.next().value; 66 | token.type.should.equal('kwd'); 67 | token.literal.should.equal('fields'); 68 | token.line.should.equal(1); 69 | token.char.should.equal(1); 70 | 71 | token = tokens.next().value; 72 | token.type.should.equal('op'); 73 | token.literal.should.equal('{'); 74 | token.line.should.equal(1); 75 | token.char.should.equal(8); 76 | 77 | token = tokens.next().value; 78 | token.type.should.equal('eof'); 79 | }); 80 | 81 | it('returns correct tokens for (S => each {)', function() { 82 | var tokens = lexer('each {'); 83 | 84 | var token = tokens.next().value; 85 | token.type.should.equal('kwd'); 86 | token.literal.should.equal('each'); 87 | 88 | token = tokens.next().value; 89 | token.type.should.equal('op'); 90 | token.literal.should.equal('{'); 91 | 92 | token = tokens.next().value; 93 | token.type.should.equal('eof'); 94 | }); 95 | 96 | it('returns correct tokens for (S => })', function() { 97 | var tokens = lexer('}'); 98 | 99 | var token = tokens.next().value; 100 | token.type.should.equal('cl'); 101 | token.literal.should.equal('}'); 102 | 103 | token = tokens.next().value; 104 | token.type.should.equal('eof'); 105 | }); 106 | 107 | it('returns correct tokens for (S => A should BCE)', function() { 108 | var tokens = lexer('count should equal 10'); 109 | 110 | var token = tokens.next().value; 111 | token.type.should.equal('id'); 112 | token.literal.should.equal('count'); 113 | 114 | token = tokens.next().value; 115 | token.type.should.equal('kwd'); 116 | token.literal.should.equal('should'); 117 | 118 | token = tokens.next().value; 119 | token.type.should.equal('kwd'); 120 | token.literal.should.equal('equal'); 121 | 122 | token = tokens.next().value; 123 | token.type.should.equal('num'); 124 | token.literal.should.equal(10); 125 | 126 | token = tokens.next().value; 127 | token.type.should.equal('eof'); 128 | }); 129 | 130 | it('returns correct tokens for (S => A should BCE) with null A', function() { 131 | var tokens = lexer('should be object'); 132 | 133 | var token = tokens.next().value; 134 | token.type.should.equal('kwd'); 135 | token.literal.should.equal('should'); 136 | 137 | token = tokens.next().value; 138 | token.type.should.equal('kwd'); 139 | token.literal.should.equal('be'); 140 | 141 | token = tokens.next().value; 142 | token.type.should.equal('kwd'); 143 | token.literal.should.equal('object'); 144 | 145 | token = tokens.next().value; 146 | token.type.should.equal('eof'); 147 | }); 148 | 149 | it('returns token with negative number', function() { 150 | var tokens = lexer('-10'); 151 | 152 | var token = tokens.next().value; 153 | token.type.should.equal('num'); 154 | token.literal.should.equal(-10); 155 | 156 | token = tokens.next().value; 157 | token.type.should.equal('eof'); 158 | }); 159 | 160 | it('returns token with decimal number', function() { 161 | var tokens = lexer('99.9'); 162 | 163 | var token = tokens.next().value; 164 | token.type.should.equal('num'); 165 | token.literal.should.equal(99.9); 166 | 167 | token = tokens.next().value; 168 | token.type.should.equal('eof'); 169 | }); 170 | 171 | it('returns correct tokens for multiline block', function() { 172 | var tokens = lexer('test http://test.api {\n\n}'); 173 | 174 | var token = tokens.next().value; 175 | token.type.should.equal('kwd'); 176 | token.literal.should.equal('test'); 177 | 178 | token = tokens.next().value; 179 | token.type.should.equal('id'); 180 | token.literal.should.equal('http://test.api'); 181 | 182 | token = tokens.next().value; 183 | token.type.should.equal('op'); 184 | token.literal.should.equal('{'); 185 | 186 | token = tokens.next().value; 187 | token.type.should.equal('cl'); 188 | token.literal.should.equal('}'); 189 | 190 | token = tokens.next().value; 191 | token.type.should.equal('eof'); 192 | }); 193 | 194 | it('throws unidentified character error line 3', function() { 195 | (function() { 196 | lexer('test {\n\n $'); 197 | }).should.throw('Unidentified character "$" on line: 3, char: 2'); 198 | }); 199 | 200 | it('throws unidentified character error on first char', function() { 201 | (function() { 202 | lexer('$'); 203 | }).should.throw('Unidentified character "$" on line: 1, char: 1'); 204 | }); 205 | 206 | it('throws unidentified character error on fifth char', function() { 207 | (function() { 208 | lexer('test $'); 209 | }).should.throw('Unidentified character "$" on line: 1, char: 6'); 210 | }); 211 | 212 | it('throws unexpected character error on third char', function() { 213 | (function() { 214 | lexer('10.0.'); 215 | }).should.throw('Unexpected character "." on line: 1, char: 5'); 216 | }); 217 | 218 | }); -------------------------------------------------------------------------------- /test/parserTests.js: -------------------------------------------------------------------------------- 1 | var lexer = require('./../lexer'); 2 | var parser = require('./../parser'); 3 | var should = require('should'); 4 | 5 | describe('Parser', function() { 6 | }); -------------------------------------------------------------------------------- /util.js: -------------------------------------------------------------------------------- 1 | "use strict"; 2 | 3 | exports.keywords = [ 'test', 'fields', 'size', 'each', 'object', 'array', 'string', 'number', 'boolean', 'true', 'false', 'should', 'not', 'be', 'equal' ]; 4 | 5 | exports.lexTable = [ 6 | [ /[a-zA-Z_:/]/, /[0-9]/, /\{/, /\}/, /"/, /\./, /-/, /\s/ ], // 0: Regexes 7 | //[ i d { } " . - sp \n ] 8 | [ 2, 4, 12, 13, 6, 0, 11, 1, 1 ], // 1: Starting State 9 | [ 2, 2, 3, 3, 3, 2, 2, 3, 3 ], // 2: In Identifier 10 | [ 1, 1, 1, 1, 1, 1, 1, 1, 1 ], // 3: End Identifier * 11 | [ 5, 4, 5, 5, 5, 8, 5, 5, 5 ], // 4: In Number 12 | [ 1, 1, 1, 1, 1, 1, 1, 1, 1 ], // 5: End Number * 13 | [ 6, 6, 6, 6, 7, 6, 6, 6, 6 ], // 6: In String 14 | [ 1, 1, 1, 1, 1, 1, 1, 1, 1 ], // 7: End String * 15 | [ 0, 9, 0, 0, 0, 0, 0, 0, 0 ], // 8: Found Decimal Point 16 | [ 10, 9, 10, 10, 10, 10, 10, 10, 10 ], // 9: In Decimal 17 | [ 1, 1, 1, 1, 1, 1, 1, 1, 1 ], // 10: End Decimal * 18 | [ 0, 4, 0, 0, 0, 0, 0, 0, 0 ], // 11: Found minus sign 19 | [ 1, 1, 1, 1, 1, 1, 1, 1, 1 ], // 12: Found Start Block * 20 | [ 1, 1, 1, 1, 1, 1, 1, 1, 1 ], // 13: Found End Block * 21 | ]; 22 | 23 | exports.isStr = (token) => { 24 | return token.type === 'str'; 25 | }; 26 | 27 | exports.isNumber = (token) => { 28 | return token.type === 'num'; 29 | }; 30 | 31 | exports.isIdentifier = (token) => { 32 | return token.type === 'id'; 33 | }; 34 | 35 | exports.isOpeningBrace = (token) => { 36 | return token.type === 'op'; 37 | }; 38 | 39 | exports.isClosingBrace = (token) => { 40 | return token.type === 'cl'; 41 | }; 42 | 43 | exports.isEOF = (token) => { 44 | return token.type === 'eof'; 45 | }; 46 | 47 | exports.isKeyword = (token, kwd) => { 48 | return token.type === 'kwd' && token.literal === kwd; 49 | }; --------------------------------------------------------------------------------