├── .gitignore ├── README.md ├── api-vcr.jpg ├── cli.js ├── docs ├── architecture-diagram.bmml ├── architecture-diagram.png ├── assets │ └── api-vcr.jpg └── json-in-finder.png ├── package.json └── src ├── config.coffee ├── config.js ├── fileIO.coffee ├── fileIO.js ├── staticData.coffee ├── staticData.js ├── vcr.coffee └── vcr.js /.gitignore: -------------------------------------------------------------------------------- 1 | data/ 2 | api-vcr-data/ 3 | .idea/ 4 | node_modules/ -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Record and Play Back your API 2 | 3 | ![api-vcr logo](https://raw.githubusercontent.com/SimplGy/api-vcr/master/api-vcr.jpg) 4 | > Drawing credit: [dancingpirates](https://instagram.com/dancingpirates/) 5 | 6 | This records API responses so you can play them back later, quickly and reliably. 7 | 8 | If you're a front-end developer this means you can develop without an API server--even without the internet. 9 | 10 | You can also write tests for front end components that have some API dependencies but shouldn't break if the API server has hiccups. 11 | 12 | This is a node app with no database dependency. It's easy to integrate with `grunt` or `gulp`, or use from the command line. 13 | 14 | It's good for: 15 | 16 | * **Testing** (Responses are always the same, and fast) 17 | * **Nomads** (Work on JS apps without internet) 18 | * **Unstable APIs** (the VCR smooths out API downtimes) 19 | * **Throttled APIs** (don't get shut down for querying too much while developing) 20 | * **Coordinating** with frequent back end changes that require 30 minutes of API downtime to rebuild and deploy (ahem). 21 | 22 | ## How it Works 23 | 24 | This works by standing up a server in between your app and your API. If you have control over your app's API url, it's easy--just point it to `api-vcr` instead. 25 | 26 | ![api-vcr architecture diagram](docs/architecture-diagram.png) 27 | 28 | If you have multiple API servers, no problem. You stand up an instance of the vcr for each one. Each VCR will have a different port so you set them up as different urls in your app, just like you're probably already doing. 29 | 30 | If you do not have control over the API server being requested (if you're using library code, for example), you can modify your `etc/hosts` file to accomplish the same flow. I'm thinking about a script to make this easier, but I have some security concerns and I'm not sure about the design. I don't want to mess up anybody's system. 31 | 32 | Whenever you make reqeuests to api-vcr in `--record` mode, it passes the request on to your destination api server and records the response. You can see JSON files for your recorded responses in finder any time: 33 | 34 | ![api-vcr json files in finder](docs/json-in-finder.png) 35 | 36 | ## Precedent 37 | 38 | This is similar to [some](https://github.com/vcr/vcr) [other](http://www.mock-server.com/) [projects](https://github.com/assaf/node-replay). 39 | Other projects might be better for your needs, but some things make this one different: 40 | 41 | * Other solutions are focused on testing. That's great and valid, but I want to **develop** against something fast, deterministic, and reliable, too. 42 | * This is written in Node, so it's easy to fit in with a front end developer's workflow. 43 | * I store the responses as plain text JSON files so you can modify them whenever you want, and see very clearly where I create bugs or mismatch your expectations on API paths. 44 | * You can fake out your own API by making a tree of `json` files. No recording necessary. That could be pretty useful, huh? 45 | * Supports multiple API servers. You just run multiple instances and it stores data in a folder tree by hostname and port. 46 | 47 | 48 | ## Installation 49 | 50 | This module is a dev tool with a command line interface, so it is installed globally. 51 | 52 | npm install -g api-vcr 53 | 54 | I'd actually prefer it to be local to a project, but there's no way to provide a nice CLI API in a locally installed node module. 55 | You'd have to type `./node_modules/.bin/api-vcr` all the time (plus args), and I don't want to do that to you. 56 | This is actually why [grunt-cli](https://github.com/gruntjs/grunt-cli) exists, I now understand. 57 | 58 | 59 | ## Quick Start Guide 60 | 61 | First let's launch in `--record` mode so we can pass through requests and record the responses: 62 | 63 | api-vcr http://jsonplaceholder.typicode.com/ --record 64 | 65 | Now lets hit some endpoints so we can get a nice set of data. 66 | 67 | In a browser, you can drop in these urls, or any from the [jsonplaceholder api](http://jsonplaceholder.typicode.com/): 68 | 69 | * [http://localhost:59007/users](http://localhost:59007/users) 70 | * [http://localhost:59007/users/1](http://localhost:59007/users/1) 71 | * [http://localhost:59007/posts/2](http://localhost:59007/posts/2) 72 | * [http://localhost:59007/posts/1/comments](http://localhost:59007/posts/1/comments) 73 | 74 | For each request, the VCR will create a single JSON file, using the request information to name the file. 75 | 76 | For example, the request: 77 | 78 | GET http://localhost:59007/users/1 79 | 80 | Is mapped to the file: 81 | 82 | ./api-vcr-data/jsonplaceholder.typicode.com/80/users/1.json 83 | 84 | After recording a little, you can stop the server with `Ctrl+C`. Armed with recorded data, you're ready to run in offline playback mode. 85 | 86 | api-vcr http://jsonplaceholder.typicode.com 87 | 88 | Now any request you've previously recorded returns instantly and reliably from disk: 89 | 90 | * [http://localhost:59007/users](http://localhost:59007/users) 91 | * [http://localhost:59007/users/1](http://localhost:59007/users/1) 92 | 93 | Similar requests you haven't recorded yet return their best guess: 94 | 95 | * [http://localhost:59007/users/9999](http://localhost:59007/users/9999) 96 | 97 | Totally new requests return a 404: 98 | 99 | * [http://localhost:59007/the/rain/in/spain](http://localhost:59007/the/rain/in/spain) 100 | 101 | 102 | ## Options 103 | 104 | ### `--port` 105 | 106 | api-vcr http://myapi.com:8080 --port=1337 107 | 108 | You can specify a port for the vcr server to listen on. 109 | This is useful for running more than once instance. 110 | The port of the API server is used by default (unless that port is 80), 111 | this makes keeping proxies straight a little easier if you have remote APIs identified only by port number. 112 | 113 | ### `--data` 114 | 115 | api-vcr http://myapi.com:8080 --data=~/sites/myApp/testData 116 | 117 | By default, data is stored wherever you run the command. If you always run the command from your project dir, this works well. You can also set a hard-coded data path. 118 | 119 | 120 | ### `--noSiblings` 121 | 122 | api-vcr http://myapi.com:8080 --noSiblings 123 | 124 | By default the vcr looks for a sibling if it can't find a requested object. If you ask for `user/7`, for example, it will return `user/42` if it has one. 125 | This is awesome if you just want to keep working and it doesn't matter too much that it's the wrong user/product/sprocket/whatever. 126 | 127 | Not everyone wants this behavior though, so this option lets you turn it off. 128 | 129 | 130 | ## Seeding data 131 | 132 | Data is all in the `./api-vcr-data` folder local to where you run the command, or where you configure it with the `--data` option. If you run this as a build task inside your project this works well--your api data is stored with your project. 133 | 134 | You can create or modify data at will and distribute it in git with your project for consistent testing and front-end development across machines. 135 | 136 | Please be aware that if you run in record mode, it will overwrite any file with the same name, so if you make extensive manual edits to the data you should protect them. 137 | 138 | 139 | ## Developing 140 | 141 | Running the local version is the same except instead of `api-vcr`, you use `./cli.js` from this folder. 142 | 143 | You can bump the version with npm's awesome `npm version patch && npm publish`. 144 | 145 | 146 | ## TODO 147 | 148 | - [x] Start the app a `record` option 149 | - [x] Logs all requests to console 150 | - [x] Pass requests on to the recorded server 151 | - [x] Create a directory structure that matches requests (namespace by server and port to support multiple APIs) 152 | - [x] Store request responses as JSON files 153 | - [x] Support missing objects (eg: if you have recorded `surfboard/3` and they request `5`, return `3` instead) 154 | - [x] Print version on startup 155 | - [x] Switch to a global install for easier run API by packages that depend on this one. 156 | - [x] Bug: encode colons to `%3A` or something else in filenames 157 | 158 | - [ ] Bug: returning similar siblings returns deep nested JSON if there is a child folder (eg: posts/777 returns posts/1/comments.json) 159 | - [ ] Bug: returning siblings shouldn't return list objects for id requests and visa-versa 160 | - [ ] Handle non JSON gracefully 161 | - [ ] Screenshot of requests, mapping to screenshot of files in finder 162 | - [ ] Script to add and remove from etc/hosts. (design? will this work?) 163 | - [ ] Chrome extension difficulty estimation. https://github.com/chrisaljoudi/uBlock. Dev tools tab, monitor xhr and check boxes for ones to record? 164 | - [ ] Support a config file, maybe in `./api-vcr-config.json` (is there an option parsing library that supports fall-through from args to json file to defaults?) 165 | - [ ] Print the fully resolved path name for data, otherwise it's unclear where `.` is 166 | - [ ] Have a simple index page with list of all routes we know about, for easy debugging/transparency 167 | - [ ] Support query params 168 | - [ ] Option to trim large array responses to 20-100 max (default true) 169 | - [ ] Support response types other than JSON gracefully 170 | - [ ] Support POST, PUT, DELETE (at least don't error, and return a sensible object) -------------------------------------------------------------------------------- /api-vcr.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SimplGy/api-vcr/b01dcba8f3cc7373873932b26fe0f643c27ac54a/api-vcr.jpg -------------------------------------------------------------------------------- /cli.js: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env node 2 | 3 | // This file can't be in coffeescript, because of some baloney with parsing a she-bang comment as a comment: 4 | // https://github.com/jashkenas/coffeescript/issues/2215 5 | // The script must have a shebang in order to create the .cmd shim on windows: 6 | // https://github.com/ForbesLindesay/cmd-shim/blob/f59af911f1373239a7537072641d55ff882c3701/index.js#L22 7 | 8 | var config, minimist, options, url, vcr, pkg; 9 | 10 | vcr = require('./src/vcr'); 11 | minimist = require('minimist'); 12 | url = require('url'); 13 | config = require('./src/config'); 14 | fileIO = require('./src/fileIO'); 15 | pkg = require('./package.json'); 16 | 17 | 18 | 19 | console.log(''); 20 | console.log('apiVCR '+ pkg.version +' Starting'); 21 | console.log('----------------------'); 22 | 23 | options = minimist(process.argv.slice(2)); 24 | 25 | mightBeApi = options._[0] 26 | if (!mightBeApi) { 27 | throw "Need an API server specified. eg: `api-vcr http://api.pickle.com`"; 28 | } 29 | config.api = url.parse(mightBeApi); 30 | if (config.api.pathname === config.api.href && config.api.path === config.api.href) { 31 | throw "API server specified doesn't look like a url: " + mightBeApi; 32 | } 33 | 34 | if (options.noSiblings) { 35 | config.sameSameSiblings = false; 36 | } 37 | if (config.api.port) { 38 | config.port = config.api.port; 39 | } 40 | if (options.port) { 41 | config.port = options.port; 42 | } 43 | if (options.data) { 44 | config.rootPath = options.data; 45 | } 46 | 47 | config.computeFilePath(); 48 | 49 | if (options.record) { 50 | vcr.record(); 51 | } else if (!fileIO.count()) { 52 | process.exit(12) // if we aren't recording AND there are no files, this is an error 53 | } 54 | 55 | vcr.start(); 56 | 57 | -------------------------------------------------------------------------------- /docs/architecture-diagram.bmml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | Your%20App%0AJavaScript%20Client 6 | 7 | 8 | 9 | 10 | 0 11 | false 12 | RESTful%20XHR%20Requests 13 | 14 | 15 | 16 | 17 | 18 | 19 | 14540253 20 | 21 | 22 | 23 | 24 | 25 | 28 26 | ./assets/api-vcr.jpg 27 | 28 | 29 | 30 | 31 | 32 | 33 | Your%20API%20Server%0AJSON%20responses 34 | 35 | 36 | 37 | 38 | 0 39 | false 40 | Proxy 41 | 42 | 43 | 44 | 45 | 46 | 47 | 0 48 | true 49 | false 50 | %20%20%20%20%20Recorded%20Response 51 | 52 | 53 | 54 | 55 | 13576743 56 | DotIcon%7Csmall 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 0 66 | true 67 | false 68 | %20%20%20%20Proxy%20or%20Playback 69 | 70 | 71 | 72 | 73 | 40463 74 | RightFillTriangleIcon%7Cxsmall 75 | 76 | 77 | 78 | 79 | 80 | 81 | 6710886 82 | eg%3A%20http%3A//localhost%3A59007 83 | 84 | 85 | 86 | 87 | 6710886 88 | eg%3A%20http%3A//myapi.com%3A8080 89 | 90 | 91 | 92 | 93 | none 94 | 95 | 96 | 97 | -------------------------------------------------------------------------------- /docs/architecture-diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SimplGy/api-vcr/b01dcba8f3cc7373873932b26fe0f643c27ac54a/docs/architecture-diagram.png -------------------------------------------------------------------------------- /docs/assets/api-vcr.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SimplGy/api-vcr/b01dcba8f3cc7373873932b26fe0f643c27ac54a/docs/assets/api-vcr.jpg -------------------------------------------------------------------------------- /docs/json-in-finder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SimplGy/api-vcr/b01dcba8f3cc7373873932b26fe0f643c27ac54a/docs/json-in-finder.png -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "api-vcr", 3 | "version": "0.1.27", 4 | "description": "Record API responses for later. Work from a plane, island, or submarine. No internet needed.", 5 | "homepage": "https://github.com/SimplGy/api-vcr", 6 | "repository": { 7 | "type": "git", 8 | "url": "https://github.com/SimplGy/api-vcr" 9 | }, 10 | "preferGlobal": true, 11 | "bin": "./cli.js", 12 | "keywords": [ 13 | "VCR", 14 | "offline", 15 | "testing", 16 | "mocks", 17 | "replay", 18 | "nomad", 19 | "api", 20 | "REST" 21 | ], 22 | "author": "@simplgy", 23 | "dependencies": { 24 | "express": "^4.12.3", 25 | "express-http-proxy": "^0.3.1", 26 | "fs-extra": "^0.17.0", 27 | "lodash": "^3.6.0", 28 | "minimist": "^1.1.1" 29 | } 30 | } 31 | -------------------------------------------------------------------------------- /src/config.coffee: -------------------------------------------------------------------------------- 1 | fs = require 'fs-extra' 2 | 3 | config = 4 | port: 59007 5 | rootPath: './api-vcr-data' 6 | sameSameSiblings: true # If the exact object requested isn't available, should it return sibling ids? "Siblings are same same but different" 7 | ignore: [ 8 | '.DS_Store' 9 | ] 10 | # Compute the path using `root / hostname / port` 11 | computeFilePath: -> 12 | config.filePath = [ 13 | config.rootPath 14 | config.api.hostname 15 | config.api.port || 80 # TODO: handle https 16 | ].join '/' 17 | console.log "Using file path: `#{config.filePath}`" 18 | fs.ensureDirSync config.filePath 19 | config.filePath 20 | 21 | 22 | 23 | module.exports = config 24 | -------------------------------------------------------------------------------- /src/config.js: -------------------------------------------------------------------------------- 1 | // Generated by CoffeeScript 1.8.0 2 | (function() { 3 | var config, fs; 4 | 5 | fs = require('fs-extra'); 6 | 7 | config = { 8 | port: 59007, 9 | rootPath: './api-vcr-data', 10 | sameSameSiblings: true, 11 | ignore: ['.DS_Store'], 12 | computeFilePath: function() { 13 | config.filePath = [config.rootPath, config.api.hostname, config.api.port || 80].join('/'); 14 | console.log("Using file path: `" + config.filePath + "`"); 15 | fs.ensureDirSync(config.filePath); 16 | return config.filePath; 17 | } 18 | }; 19 | 20 | module.exports = config; 21 | 22 | }).call(this); 23 | -------------------------------------------------------------------------------- /src/fileIO.coffee: -------------------------------------------------------------------------------- 1 | ### 2 | Stateless utility methods for accessing api-vcr files. 3 | Any CRUD that is file system specific should go here 4 | ### 5 | 6 | fs = require 'fs' 7 | config = require './config' 8 | path = require 'path' 9 | _ = require 'lodash' 10 | querystring = require 'querystring' 11 | 12 | # config 13 | queryStringIndicator = '__&__' # This file system safe flag is how I know there's a query string in the filename. It's a bit ghetto. # Using an `&` instead of `?` because it's more file name safe. 14 | 15 | # Get a list of all the folders 16 | # Recursively scan all files in the path to get a list of json files 17 | readFiles = (filePath, jsonFiles) -> 18 | filePath = filePath || config.filePath 19 | jsonFiles = jsonFiles || [] 20 | return if config.ignore.indexOf(path.basename filePath) isnt -1 21 | try 22 | contents = fs.readdirSync filePath # TODO: refactor to remove the blocking sync method 23 | catch 24 | console.log "`#{filePath}` is not a folder, skipping" 25 | return 26 | 27 | for file in contents 28 | fullPath = "#{filePath}/#{file}" 29 | # JSON File 30 | if file.indexOf('.json') >= 0 31 | jsonFiles.push fullPath 32 | # Folder 33 | else 34 | readFiles fullPath, jsonFiles 35 | jsonFiles 36 | 37 | 38 | # Given a full file path and a callback, try to find a sibling. 39 | # Assume the file does not exist, as that's the reason to call this method. 40 | # TODO: First try to find any ID match without query params. If none of that matches, return any sibling 41 | getSiblingName = (filename) -> 42 | dirContents = readFiles path.dirname filename 43 | if dirContents 44 | # TODO Is there one with the same ID but no query params? 45 | # TODO Is there one with the same id but different query params? 46 | return dirContents[0] # ::shrug:: The first one. TODO: search for "most similar" by nearest ID or something? 47 | else 48 | return undefined 49 | 50 | 51 | count = -> 52 | count = readFiles().length 53 | if count is 0 54 | console.log "No JSON files found in `#{config.filePath}`" 55 | if config.isRecording 56 | console.log "Good thing you're recording" 57 | else 58 | console.log "You should probably add some or run in `--record` mode first" 59 | else if count is 1 60 | console.log "Only found one file. What is this, don't trust me yet?" 61 | else 62 | console.log "Found #{count} JSON files. You're ready to jam." 63 | return count 64 | 65 | # Given a param object, turn it into a string that is always the same no matter the param order, and is safe for a filename 66 | stringifyParams = (params) -> 67 | return '' unless params 68 | strings = [] 69 | for own prop of params 70 | strings.push "#{querystring.escape prop}=#{querystring.escape params[prop]}" 71 | return '' if strings.length is 0 72 | sorted = _.sortBy strings, (s) -> s.charCodeAt 0 73 | return queryStringIndicator + sorted.join('&') 74 | 75 | 76 | # Given an HTTP request, convert it to a filename. 77 | convertReqToFilename = (req) -> 78 | file = config.filePath + req.path # root path 79 | file += stringifyParams req.query 80 | file += '.json' 81 | 82 | # Given a file on the file system, which api path should it respond to? 83 | #convertFilenameToRequestPath = (filePath) -> 84 | # apiPath = path = filePath.split('.json').join '' 85 | # apiPath.split(defaultPath).join '' 86 | 87 | 88 | module.exports = 89 | count: count 90 | getSiblingName: getSiblingName 91 | list: readFiles 92 | convertReqToFilename: convertReqToFilename 93 | -------------------------------------------------------------------------------- /src/fileIO.js: -------------------------------------------------------------------------------- 1 | // Generated by CoffeeScript 1.8.0 2 | 3 | /* 4 | Stateless utility methods for accessing api-vcr files. 5 | Any CRUD that is file system specific should go here 6 | */ 7 | 8 | (function() { 9 | var config, convertReqToFilename, count, fs, getSiblingName, path, queryStringIndicator, querystring, readFiles, stringifyParams, _, 10 | __hasProp = {}.hasOwnProperty; 11 | 12 | fs = require('fs'); 13 | 14 | config = require('./config'); 15 | 16 | path = require('path'); 17 | 18 | _ = require('lodash'); 19 | 20 | querystring = require('querystring'); 21 | 22 | queryStringIndicator = '__&__'; 23 | 24 | readFiles = function(filePath, jsonFiles) { 25 | var contents, file, fullPath, _i, _len; 26 | filePath = filePath || config.filePath; 27 | jsonFiles = jsonFiles || []; 28 | if (config.ignore.indexOf(path.basename(filePath)) !== -1) { 29 | return; 30 | } 31 | try { 32 | contents = fs.readdirSync(filePath); 33 | } catch (_error) { 34 | console.log("`" + filePath + "` is not a folder, skipping"); 35 | return; 36 | } 37 | for (_i = 0, _len = contents.length; _i < _len; _i++) { 38 | file = contents[_i]; 39 | fullPath = "" + filePath + "/" + file; 40 | if (file.indexOf('.json') >= 0) { 41 | jsonFiles.push(fullPath); 42 | } else { 43 | readFiles(fullPath, jsonFiles); 44 | } 45 | } 46 | return jsonFiles; 47 | }; 48 | 49 | getSiblingName = function(filename) { 50 | var dirContents; 51 | dirContents = readFiles(path.dirname(filename)); 52 | if (dirContents) { 53 | return dirContents[0]; 54 | } else { 55 | return void 0; 56 | } 57 | }; 58 | 59 | count = function() { 60 | count = readFiles().length; 61 | if (count === 0) { 62 | console.log("No JSON files found in `" + config.filePath + "`"); 63 | if (config.isRecording) { 64 | console.log("Good thing you're recording"); 65 | } else { 66 | console.log("You should probably add some or run in `--record` mode first"); 67 | } 68 | } else if (count === 1) { 69 | console.log("Only found one file. What is this, don't trust me yet?"); 70 | } else { 71 | console.log("Found " + count + " JSON files. You're ready to jam."); 72 | } 73 | return count; 74 | }; 75 | 76 | stringifyParams = function(params) { 77 | var prop, sorted, strings; 78 | if (!params) { 79 | return ''; 80 | } 81 | strings = []; 82 | for (prop in params) { 83 | if (!__hasProp.call(params, prop)) continue; 84 | strings.push("" + (querystring.escape(prop)) + "=" + (querystring.escape(params[prop]))); 85 | } 86 | if (strings.length === 0) { 87 | return ''; 88 | } 89 | sorted = _.sortBy(strings, function(s) { 90 | return s.charCodeAt(0); 91 | }); 92 | return queryStringIndicator + sorted.join('&'); 93 | }; 94 | 95 | convertReqToFilename = function(req) { 96 | var file; 97 | file = config.filePath + req.path; 98 | file += stringifyParams(req.query); 99 | return file += '.json'; 100 | }; 101 | 102 | module.exports = { 103 | count: count, 104 | getSiblingName: getSiblingName, 105 | list: readFiles, 106 | convertReqToFilename: convertReqToFilename 107 | }; 108 | 109 | }).call(this); 110 | -------------------------------------------------------------------------------- /src/staticData.coffee: -------------------------------------------------------------------------------- 1 | # Includes 2 | fs = require 'fs-extra' 3 | config = require './config' 4 | fileIO = require './fileIO' 5 | path = require 'path' 6 | 7 | 8 | 9 | # Given an API peth, fetch a corresponding file on the filesystem 10 | # If a file isn't found, attempts to return one in the same path with a different id (config.bestAvailable) 11 | ### 12 | TODO: Ideal similarity search when recorded data is not found: 13 | Individual resource 14 | 1. Return exact item 15 | 2. Return sibling with nearby id 16 | 3. Return first item of a list with the same resource name (eg: request for animal/7 can return JSON.parse(animals.json)[0]) 17 | 4. 404 18 | Individual resource w/ query params 19 | 1. Return exact item with exact query params 20 | 2. Find same resource with a similar query (scoring similar queries should be fun) 21 | 3. Find same resource with no query 22 | 4. Find sibling resource with exact query params 23 | 5. Find sibling resource with similar query 24 | 6. Find sibling resource with no query 25 | 7. 404 26 | Collection 27 | 1. return exact collection 28 | 2. Search for a child item(s). If there are any, add them all to an array and return that. (eg: [1.json, 2.json]) 29 | 3. 404 30 | Collection w/ query params 31 | 1. return exact collection w/ matching params 32 | 2. return exact collection with similar query 33 | 3. exact colleciton no query 34 | 4. child items merged into an array 35 | 5. 404 36 | ### 37 | get = (req, res, next) -> 38 | file = fileIO.convertReqToFilename req 39 | fileCallback = (err, data) -> 40 | if err?.code is 'ENOENT' and config.sameSameSiblings 41 | console.log " Didn't find `#{file}`. Looking for similar siblings..." 42 | sibling = fileIO.getSiblingName file 43 | if sibling 44 | console.log " Found a sibling. Returning `#{path.basename sibling}`" 45 | fs.readJson sibling, fileCallback 46 | else 47 | next() 48 | else if err?.code is 'ENOENT' 49 | console.log " File not found: #{file}" 50 | next() 51 | else if err 52 | console.log " Unhandled error", { err: err, data: data } 53 | next() 54 | else 55 | res.setHeader 'Access-Control-Allow-Methods', 'GET' 56 | res.setHeader 'Access-Control-Allow-Origin', '*' 57 | res.send data 58 | next() 59 | 60 | 61 | fs.readJson file, fileCallback # https://github.com/jprichardson/node-fs-extra#readjsonfile-options-callback 62 | 63 | 64 | # if config.sameSameSiblings 65 | 66 | 67 | # Given an api request and a data object, persist the data object to disk, or update the existing object 68 | save = (req, data) -> 69 | filename = fileIO.convertReqToFilename req 70 | fs.outputJson filename, data, (err) -> 71 | if err 72 | console.log "Couldn't write file `#{filename}`", err 73 | else 74 | console.log "Wrote file: `#{filename}`" 75 | 76 | 77 | METHODS = 78 | GET: get 79 | POST: (req, res, next) -> next(); console.warn "staticData.POST not yet supported", arguments 80 | PUT: (req, res, next) -> next(); console.warn "staticData.PUT not yet supported", arguments 81 | DELETE: (req, res, next) -> next(); console.warn "staticData.DELETE not yet supported", arguments 82 | 83 | 84 | module.exports = 85 | save: save 86 | fetchDataForRequest: (req) -> 87 | console.log "#{req.method} localhost:#{config.port} #{req.path}" # TODO: print the query params, too 88 | METHODS[req.method].apply this, arguments 89 | # METHODS[req.method] req.path, (err, data) -> 90 | # if data then res.send data 91 | # next() 92 | -------------------------------------------------------------------------------- /src/staticData.js: -------------------------------------------------------------------------------- 1 | // Generated by CoffeeScript 1.10.0 2 | (function() { 3 | var METHODS, config, fileIO, fs, get, path, save; 4 | 5 | fs = require('fs-extra'); 6 | 7 | config = require('./config'); 8 | 9 | fileIO = require('./fileIO'); 10 | 11 | path = require('path'); 12 | 13 | 14 | /* 15 | TODO: Ideal similarity search when recorded data is not found: 16 | Individual resource 17 | 1. Return exact item 18 | 2. Return sibling with nearby id 19 | 3. Return first item of a list with the same resource name (eg: request for animal/7 can return JSON.parse(animals.json)[0]) 20 | 4. 404 21 | Individual resource w/ query params 22 | 1. Return exact item with exact query params 23 | 2. Find same resource with a similar query (scoring similar queries should be fun) 24 | 3. Find same resource with no query 25 | 4. Find sibling resource with exact query params 26 | 5. Find sibling resource with similar query 27 | 6. Find sibling resource with no query 28 | 7. 404 29 | Collection 30 | 1. return exact collection 31 | 2. Search for a child item(s). If there are any, add them all to an array and return that. (eg: [1.json, 2.json]) 32 | 3. 404 33 | Collection w/ query params 34 | 1. return exact collection w/ matching params 35 | 2. return exact collection with similar query 36 | 3. exact colleciton no query 37 | 4. child items merged into an array 38 | 5. 404 39 | */ 40 | 41 | get = function(req, res, next) { 42 | var file, fileCallback; 43 | file = fileIO.convertReqToFilename(req); 44 | fileCallback = function(err, data) { 45 | var sibling; 46 | if ((err != null ? err.code : void 0) === 'ENOENT' && config.sameSameSiblings) { 47 | console.log(" Didn't find `" + file + "`. Looking for similar siblings..."); 48 | sibling = fileIO.getSiblingName(file); 49 | if (sibling) { 50 | console.log(" Found a sibling. Returning `" + (path.basename(sibling)) + "`"); 51 | return fs.readJson(sibling, fileCallback); 52 | } else { 53 | return next(); 54 | } 55 | } else if ((err != null ? err.code : void 0) === 'ENOENT') { 56 | console.log(" File not found: " + file); 57 | return next(); 58 | } else if (err) { 59 | console.log(" Unhandled error", { 60 | err: err, 61 | data: data 62 | }); 63 | return next(); 64 | } else { 65 | res.setHeader('Access-Control-Allow-Methods', 'GET'); 66 | res.setHeader('Access-Control-Allow-Origin', '*'); 67 | res.send(data); 68 | return next(); 69 | } 70 | }; 71 | return fs.readJson(file, fileCallback); 72 | }; 73 | 74 | save = function(req, data) { 75 | var filename; 76 | filename = fileIO.convertReqToFilename(req); 77 | return fs.outputJson(filename, data, function(err) { 78 | if (err) { 79 | return console.log("Couldn't write file `" + filename + "`", err); 80 | } else { 81 | return console.log("Wrote file: `" + filename + "`"); 82 | } 83 | }); 84 | }; 85 | 86 | METHODS = { 87 | GET: get, 88 | POST: function(req, res, next) { 89 | next(); 90 | return console.warn("staticData.POST not yet supported", arguments); 91 | }, 92 | PUT: function(req, res, next) { 93 | next(); 94 | return console.warn("staticData.PUT not yet supported", arguments); 95 | }, 96 | DELETE: function(req, res, next) { 97 | next(); 98 | return console.warn("staticData.DELETE not yet supported", arguments); 99 | } 100 | }; 101 | 102 | module.exports = { 103 | save: save, 104 | fetchDataForRequest: function(req) { 105 | console.log(req.method + " localhost:" + config.port + " " + req.path); 106 | return METHODS[req.method].apply(this, arguments); 107 | } 108 | }; 109 | 110 | }).call(this); 111 | -------------------------------------------------------------------------------- /src/vcr.coffee: -------------------------------------------------------------------------------- 1 | # Includes 2 | express = require 'express' 3 | http = require 'http' 4 | config = require './config' 5 | staticData = require './staticData' 6 | fileIO = require './fileIO' 7 | proxy = require 'express-http-proxy' 8 | 9 | # -------------------------------------------------- Local Variables 10 | app = express() 11 | server = undefined # scope control 12 | 13 | 14 | # -------------------------------------------------- Private Methods 15 | onError = (err) -> 16 | if err.code is 'EADDRINUSE' 17 | console.log config.port + " is in use. Can't start the server. Change the port with the `--port=1234` option" 18 | else if err.syscall isnt 'listen' 19 | throw err 20 | 21 | onListening = -> 22 | addr = server.address(); 23 | bind = if typeof addr is 'string' then 'pipe ' + addr else 'port ' + addr.port; 24 | console.log "(つ -‘ _ ‘- )つ Listening on #{bind} " 25 | console.log '' 26 | 27 | startServer = -> 28 | console.log "Creating the `api-vcr` express server" 29 | server = http.createServer app 30 | server.listen config.port 31 | server.on 'error', onError 32 | server.on 'listening', onListening 33 | 34 | decorateProxiedRequest = (req) -> 35 | # console.log "decorating", req 36 | # Fixes issue: https://github.com/villadora/express-http-proxy/issues/9 37 | req.headers[ 'Accept-Encoding' ] = 'utf8' #TODO: only accept application/json? 38 | delete req.headers['if-modified-since'] 39 | delete req.headers['if-none-match'] 40 | req 41 | 42 | interceptProxiedResponse = (data, req, res, callback) -> 43 | data = data.toString 'utf8' 44 | try 45 | data = JSON.parse data 46 | staticData.save req, data 47 | callback null, JSON.stringify(data) # Continue on to the proxy library 48 | catch 49 | console.warn "unable to parse JSON response from API server", 50 | req_path: req.path 51 | data: data 52 | callback null, data 53 | 54 | 55 | # -------------------------------------------------- Public Methods/Exports 56 | # Pass requests on to an api server 57 | record = -> 58 | # proxy-express can't deal with a trailing slash. Strip it out. 59 | if config.api.href[config.api.href.length - 1] is '/' 60 | safeHref = config.api.href.substr 0, config.api.href.length - 1 61 | console.log '' 62 | console.log "Recording #{safeHref} ᕙ༼ ,,ԾܫԾ,, ༽ᕗ " 63 | console.log '' 64 | config.isRecording = true 65 | app.use proxy safeHref, 66 | decorateRequest: decorateProxiedRequest 67 | intercept: interceptProxiedResponse 68 | 69 | # Start without recording 70 | start = -> 71 | app.use staticData.fetchDataForRequest 72 | # console.log "Found the following JSON data: ", JSON.stringify fileIO.get(), null, 2 73 | startServer() 74 | 75 | module.exports = 76 | record: record 77 | start: start 78 | 79 | 80 | 81 | 82 | -------------------------------------------------------------------------------- /src/vcr.js: -------------------------------------------------------------------------------- 1 | // Generated by CoffeeScript 1.8.0 2 | (function() { 3 | var app, config, decorateProxiedRequest, express, fileIO, http, interceptProxiedResponse, onError, onListening, proxy, record, server, start, startServer, staticData; 4 | 5 | express = require('express'); 6 | 7 | http = require('http'); 8 | 9 | config = require('./config'); 10 | 11 | staticData = require('./staticData'); 12 | 13 | fileIO = require('./fileIO'); 14 | 15 | proxy = require('express-http-proxy'); 16 | 17 | app = express(); 18 | 19 | server = void 0; 20 | 21 | onError = function(err) { 22 | if (err.code === 'EADDRINUSE') { 23 | return console.log(config.port + " is in use. Can't start the server. Change the port with the `--port=1234` option"); 24 | } else if (err.syscall !== 'listen') { 25 | throw err; 26 | } 27 | }; 28 | 29 | onListening = function() { 30 | var addr, bind; 31 | addr = server.address(); 32 | bind = typeof addr === 'string' ? 'pipe ' + addr : 'port ' + addr.port; 33 | console.log("(つ -‘ _ ‘- )つ Listening on " + bind + " "); 34 | return console.log(''); 35 | }; 36 | 37 | startServer = function() { 38 | console.log("Creating the `api-vcr` express server"); 39 | server = http.createServer(app); 40 | server.listen(config.port); 41 | server.on('error', onError); 42 | return server.on('listening', onListening); 43 | }; 44 | 45 | decorateProxiedRequest = function(req) { 46 | req.headers['Accept-Encoding'] = 'utf8'; 47 | delete req.headers['if-modified-since']; 48 | delete req.headers['if-none-match']; 49 | return req; 50 | }; 51 | 52 | interceptProxiedResponse = function(data, req, res, callback) { 53 | data = data.toString('utf8'); 54 | try { 55 | data = JSON.parse(data); 56 | staticData.save(req, data); 57 | return callback(null, JSON.stringify(data)); 58 | } catch (_error) { 59 | console.warn("unable to parse JSON response from API server", { 60 | req_path: req.path, 61 | data: data 62 | }); 63 | return callback(null, data); 64 | } 65 | }; 66 | 67 | record = function() { 68 | var safeHref; 69 | if (config.api.href[config.api.href.length - 1] === '/') { 70 | safeHref = config.api.href.substr(0, config.api.href.length - 1); 71 | } 72 | console.log(''); 73 | console.log("Recording " + safeHref + " ᕙ༼ ,,ԾܫԾ,, ༽ᕗ "); 74 | console.log(''); 75 | config.isRecording = true; 76 | return app.use(proxy(safeHref, { 77 | decorateRequest: decorateProxiedRequest, 78 | intercept: interceptProxiedResponse 79 | })); 80 | }; 81 | 82 | start = function() { 83 | app.use(staticData.fetchDataForRequest); 84 | return startServer(); 85 | }; 86 | 87 | module.exports = { 88 | record: record, 89 | start: start 90 | }; 91 | 92 | }).call(this); 93 | --------------------------------------------------------------------------------