├── .babelrc ├── .editorconfig ├── .eslintignore ├── .eslintrc ├── .gitattributes ├── .gitignore ├── .npmignore ├── .npmrc ├── .tern-project ├── .travis.yml ├── LICENSE ├── README.md ├── appveyor.yml ├── package-lock.json ├── package.json ├── src ├── ElectronManager.js ├── ElectronWorker.js ├── checkIpcStatus.js ├── checkPortStatus.js └── index.js └── test ├── .eslintrc ├── electron-script ├── ipc │ ├── concurrency.js │ ├── custom-args.js │ ├── env.js │ ├── recycle-on-exit.js │ ├── script.js │ ├── slowstart.js │ ├── stdio.js │ ├── timeout.js │ └── workerId.js └── server │ ├── concurrency.js │ ├── custom-args.js │ ├── env.js │ ├── just-port.js │ ├── recycle-on-exit.js │ ├── script.js │ ├── slowstart.js │ ├── stdio.js │ ├── timeout.js │ └── workerId.js ├── mocha.opts └── test.js /.babelrc: -------------------------------------------------------------------------------- 1 | { 2 | "stage": 0, 3 | "loose": "all" 4 | } 5 | -------------------------------------------------------------------------------- /.editorconfig: -------------------------------------------------------------------------------- 1 | # EditorConfig helps developers define and maintain consistent 2 | # coding styles between different editors and IDEs 3 | # editorconfig.org 4 | 5 | root = true 6 | 7 | 8 | [*] 9 | # Change these settings to your own preference 10 | indent_style = space 11 | indent_size = 2 12 | 13 | # Recommend you to keep these unchanged 14 | end_of_line = lf 15 | charset = utf-8 16 | trim_trailing_whitespace = true 17 | insert_final_newline = true 18 | 19 | [*.py] 20 | indent_style = space 21 | indent_size = 4 22 | 23 | [*.js] 24 | indent_style = space 25 | indent_size = 2 26 | 27 | [*.css] 28 | indent_style = space 29 | indent_size = 2 30 | 31 | [*.jade] 32 | indent_style = space 33 | indent_size = 2 34 | 35 | [*.md] 36 | trim_trailing_whitespace = false 37 | -------------------------------------------------------------------------------- /.eslintignore: -------------------------------------------------------------------------------- 1 | lib 2 | **/node_modules 3 | -------------------------------------------------------------------------------- /.eslintrc: -------------------------------------------------------------------------------- 1 | { 2 | "extends": "eslint-config-airbnb-base", 3 | 4 | // Allow the following global variables 5 | "env": { 6 | "node": true // Node.js global variables and Node.js scoping. 7 | }, 8 | 9 | "parserOptions": { 10 | "ecmaFeatures": { 11 | experimentalObjectRestSpread: true 12 | } 13 | }, 14 | 15 | "rules": { 16 | "strict": [2, "safe"], 17 | 18 | /** 19 | * ES6 20 | */ 21 | "prefer-const": 0, 22 | 23 | /** 24 | * Variables 25 | */ 26 | "no-shadow": [2, { 27 | "builtinGlobals": true 28 | }], 29 | "no-unused-vars": [2, { 30 | "vars": "all", 31 | "args": "after-used" 32 | }], 33 | "no-use-before-define": [2, "nofunc"], 34 | 35 | /** 36 | * Possible errors 37 | */ 38 | "comma-dangle": [2, "never"], 39 | "no-inner-declarations": [2, "both"], 40 | 41 | /** 42 | * Best practices 43 | */ 44 | "consistent-return": 0, 45 | "curly": 2, 46 | "dot-notation": [2, { 47 | "allowKeywords": true, 48 | "allowPattern": "^[a-z]+(_[a-z]+)+$" 49 | }], 50 | "eqeqeq": [2, "allow-null"], 51 | "no-eq-null": 0, 52 | "no-redeclare": [2, { 53 | "builtinGlobals": true 54 | }], 55 | "wrap-iife": [2, "inside"], 56 | "max-len": [2, 130, 2, {"ignoreUrls": true}], 57 | 58 | /** 59 | * Style 60 | */ 61 | "indent": [2, 2, { 62 | "VariableDeclarator": { 63 | "var": 2, 64 | "let": 2, 65 | "const": 3 66 | }, 67 | "SwitchCase": 1 68 | }], 69 | "func-names": 0, 70 | "no-multiple-empty-lines": [2, { 71 | "max": 1 72 | }], 73 | "no-extra-parens": [2, "functions"], 74 | "one-var": 0, 75 | "space-before-function-paren": [2, "never"], 76 | "no-underscore-dangle": 0 77 | } 78 | } 79 | -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | # Git to autodetect text files and normalise their line endings to LF when they are checked into your repository. 2 | * text=auto 3 | 4 | # 5 | # The above will handle all files NOT found below 6 | # 7 | 8 | # These files are text and should be normalized (Convert crlf => lf) 9 | *.php text 10 | *.css text 11 | *.js text 12 | *.htm text 13 | *.html text 14 | *.xml text 15 | *.txt text 16 | *.ini text 17 | *.inc text 18 | .htaccess text 19 | 20 | # These files are binary and should be left untouched 21 | # (binary is a macro for -text -diff) 22 | *.png binary 23 | *.jpg binary 24 | *.jpeg binary 25 | *.gif binary 26 | *.ico binary 27 | *.mov binary 28 | *.mp4 binary 29 | *.mp3 binary 30 | *.flv binary 31 | *.fla binary 32 | *.swf binary 33 | *.gz binary 34 | *.zip binary 35 | *.7z binary 36 | *.ttf binary 37 | 38 | # Documents 39 | *.doc diff=astextplain 40 | *.DOC diff=astextplain 41 | *.docx diff=astextplain 42 | *.DOCX diff=astextplain 43 | *.dot diff=astextplain 44 | *.DOT diff=astextplain 45 | *.pdf diff=astextplain 46 | *.PDF diff=astextplain 47 | *.rtf diff=astextplain 48 | *.RTF diff=astextplain 49 | 50 | # These files are text and should be normalized (Convert crlf => lf) 51 | *.gitattributes text 52 | .gitignore text -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Logs 2 | log 3 | *.log 4 | npm-debug.log 5 | 6 | # Runtime data 7 | pids 8 | *.pid 9 | *.seed 10 | 11 | # Directory for instrumented libs generated by jscoverage/JSCover 12 | lib-cov 13 | 14 | # Coverage directory used by tools like istanbul 15 | coverage 16 | 17 | # Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files) 18 | .grunt 19 | 20 | # Compiled binary addons (http://nodejs.org/api/addons.html) 21 | build/Release 22 | 23 | # Dependency directory 24 | # Deployed apps should consider commenting this line out: 25 | # see https://npmjs.org/doc/faq.html#Should-I-check-my-node_modules-folder-into-git 26 | node_modules 27 | bower_components 28 | 29 | # Temp directory 30 | .DS_Store 31 | .tmp 32 | .sass-cache 33 | 34 | # compiled es5 code 35 | lib 36 | -------------------------------------------------------------------------------- /.npmignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | *.log 3 | src 4 | test 5 | -------------------------------------------------------------------------------- /.npmrc: -------------------------------------------------------------------------------- 1 | save-exact=true 2 | -------------------------------------------------------------------------------- /.tern-project: -------------------------------------------------------------------------------- 1 | { 2 | "ecmaVersion": 6, 3 | "libs": [], 4 | "plugins": { 5 | "node": {}, 6 | "modules": {}, 7 | "es_modules": {} 8 | } 9 | } -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: node_js 2 | node_js: 3 | - "6.9" 4 | - "4.1" 5 | - "4.0" 6 | - "0.10" 7 | install: 8 | - export DISPLAY=':99.0' 9 | - Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 & 10 | - npm install 11 | addons: 12 | apt: 13 | packages: 14 | - xvfb 15 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 BJR Matos 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | electron-workers 2 | ================ 3 | 4 | [![NPM Version](http://img.shields.io/npm/v/electron-workers.svg?style=flat-square)](https://npmjs.com/package/electron-workers)[![License](http://img.shields.io/npm/l/electron-workers.svg?style=flat-square)](http://opensource.org/licenses/MIT)[![Build Status](https://travis-ci.org/bjrmatos/electron-workers.png?branch=master)](https://travis-ci.org/bjrmatos/electron-workers) 5 | 6 | > **Run electron scripts in managed workers** 7 | 8 | This module lets you run an electron script with scalability in mind, useful if you have to rely on electron to do heavy or long running tasks in parallel (web scraping, taking screenshots, generating PDFs, etc). 9 | 10 | *Works with electron@>=0.35.x including electron@1.x.x* 11 | 12 | > Note: This package is designed to work when used from a node.js process/app, since **the main use case is to manage a pool of electron processes from a node.js process**. if for some reason you need to use this package to manage a pool of electron processes **from an electron process** then PR are welcome 😃 13 | 14 | Requeriments 15 | ------------ 16 | 17 | - Install [electron](http://electron.atom.io/) >= 0.35.x including electron@1, the easy way to install 18 | electron in your app is `npm install electron --save` or `npm install electron-prebuilt --save` 19 | (or you can pass the path to your `electron` executable using the `pathToElectron` option, see [options](#options)) 20 | 21 | 22 | Modes 23 | ----- 24 | 25 | There are two ways to communicate and distribute tasks between workers, each mode has its own way to use. 26 | 27 | - `server` -> Communication and task distribution will be doing using an embedded web server inside the electron process. 28 | - `ipc` -> Communication and task distribution will be doing using an ipc channel. 29 | 30 | The best mode to use will depend of how your electron app is implemented, however the recommended option is to use the `ipc` mode. 31 | 32 | ### How to use server mode 33 | 34 | 1.- First create an electron script wrapped in a webserver 35 | 36 | *script.js* 37 | 38 | ```js 39 | var http = require('http'), 40 | app = require('electron').app; 41 | 42 | // every worker gets unique port, get it from a process environment variables 43 | var port = process.env.ELECTRON_WORKER_PORT, 44 | host = process.env.ELECTRON_WORKER_HOST, 45 | workerId = process.env.ELECTRON_WORKER_ID; // worker id useful for logging 46 | 47 | console.log('Hello from worker', workerId); 48 | 49 | app.on('ready', function() { 50 | // you can use any webserver library/framework you like (connect, express, hapi, etc) 51 | var server = http.createServer(function(req, res) { 52 | // You can respond with a status `500` if you want to indicate that something went wrong 53 | res.writeHead(200, {'Content-Type': 'application/json'}); 54 | // data passed to `electronWorkers.execute` will be available in req body 55 | req.pipe(res); 56 | }); 57 | 58 | server.listen(port, host); 59 | }); 60 | ``` 61 | 62 | 2.- Start electron workers 63 | 64 | ```js 65 | var electronWorkers = require('electron-workers')({ 66 | connectionMode: 'server', 67 | pathToScript: 'script.js', 68 | timeout: 5000, 69 | numberOfWorkers: 5 70 | }); 71 | 72 | electronWorkers.start(function(startErr) { 73 | if (startErr) { 74 | return console.error(startErr); 75 | } 76 | 77 | // `electronWorkers` will send your data in a POST request to your electron script 78 | electronWorkers.execute({ someData: 'someData' }, function(err, data) { 79 | if (err) { 80 | return console.error(err); 81 | } 82 | 83 | console.log(JSON.stringify(data)); // { someData: 'someData' } 84 | electronWorkers.kill(); // kill all workers explicitly 85 | }); 86 | }); 87 | ``` 88 | 89 | ### How to use ipc mode 90 | 91 | 1.- First create an electron script 92 | 93 | You will have an ipc channel available, what this means is that you can use `process.send`, and listen `process.on('message', function() {})` inside your script 94 | 95 | *script.js* 96 | 97 | ```js 98 | var app = require('electron').app; 99 | 100 | var workerId = process.env.ELECTRON_WORKER_ID; // worker id useful for logging 101 | 102 | console.log('Hello from worker', workerId); 103 | 104 | app.on('ready', function() { 105 | // first you will need to listen the `message` event in the process object 106 | process.on('message', function(data) { 107 | if (!data) { 108 | return; 109 | } 110 | 111 | // `electron-workers` will try to verify is your worker is alive sending you a `ping` event 112 | if (data.workerEvent === 'ping') { 113 | // responding the ping call.. this will notify `electron-workers` that your process is alive 114 | process.send({ workerEvent: 'pong' }); 115 | } else if (data.workerEvent === 'task') { // when a new task is executed, you will recive a `task` event 116 | 117 | 118 | console.log(data); //data -> { workerEvent: 'task', taskId: '....', payload: } 119 | 120 | console.log(data.payload.someData); // -> someData 121 | 122 | // you can do whatever you want here.. 123 | 124 | // when the task has been processed, 125 | // respond with a `taskResponse` event, the `taskId` that you have received, and a custom `response`. 126 | // You can specify an `error` field if you want to indicate that something went wrong 127 | process.send({ 128 | workerEvent: 'taskResponse', 129 | taskId: data.taskId, 130 | response: { 131 | value: data.payload.someData 132 | } 133 | }); 134 | } 135 | }); 136 | }); 137 | ``` 138 | 139 | 2.- Start electron workers 140 | 141 | ```js 142 | var electronWorkers = require('electron-workers')({ 143 | connectionMode: 'ipc', 144 | pathToScript: 'script.js', 145 | timeout: 5000, 146 | numberOfWorkers: 5 147 | }); 148 | 149 | electronWorkers.start(function(startErr) { 150 | if (startErr) { 151 | return console.error(startErr); 152 | } 153 | 154 | // `electronWorkers` will send your data in a POST request to your electron script 155 | electronWorkers.execute({ someData: 'someData' }, function(err, data) { 156 | if (err) { 157 | return console.error(err); 158 | } 159 | 160 | console.log(JSON.stringify(data)); // { value: 'someData' } 161 | electronWorkers.kill(); // kill all workers explicitly 162 | }); 163 | }); 164 | ``` 165 | 166 | Options 167 | ------- 168 | 169 | `connectionMode` - `server`, `ipc` mode, defaults to `server` mode if no specified.`pathToScript` (required) - path to the electron script. 170 | 171 | `pathToElectron` - path to the electron executable, by default we will try to find the path using the value returned from the `electron` or `electron-prebuilt` packages (if any of them are found), otherwhise we will try to find it in your `$PATH` env var. 172 | 173 | `debug` Number - pass debug port to electron process,[see electron's debugging guide](http://electron.atom.io/docs/v0.34.0/tutorial/debugging-main-process/). 174 | 175 | `debugBrk` Number - pass debug-brk port to electron process, [see electron's debugging guide](http://electron.atom.io/docs/v0.34.0/tutorial/debugging-main-process/) 176 | 177 | `electronArgs` Array - pass custom arguments to the electron executable. ej: `electronArgs: ['--some-value=2', '--enable-some-behaviour']`. 178 | 179 | `env` Object - pass custom env vars to workers. ej: `env: { CUSTOM_ENV: 'foo' }`. 180 | 181 | `stdio` pass custom stdio option to worker's child process. see [node.js documentation](https://nodejs.org/api/child_process.html#child_process_options_stdio) for details. 182 | 183 | `killSignal` String - when calling `electronWorkers.kill()` this value will be used to [kill the child process](https://nodejs.org/api/child_process.html#child_process_child_kill_signal) attached to the worker. see node.js docs for [more info on signal events](https://nodejs.org/api/process.html#process_signal_events) 184 | 185 | `pingTimeout` Number - time in ms to wait for worker response in order to be considered alive, note that we retry the ping to a worker several times, this value is the interval between those pings. Default: 100 186 | 187 | `timeout` - execution timeout in ms. 188 | 189 | `numberOfWorkers` - number of electron instances, by default it will be the number of cores in the machine. 190 | 191 | `maxConcurrencyPerWorker` - number of tasks a worker can handle at the same time, default `Infinity` 192 | 193 | `host` - ip or hostname where to start listening electron web server, default localhost 194 | 195 | `portLeftBoundary` - don't specify if you just want to take any random free port 196 | 197 | `portRightBoundary` - don't specify if you just want to take any random free port 198 | 199 | `hostEnvVarName` - customize the name of the environment variable passed to the electron script that specifies the worker host. defaults to `ELECTRON_WORKER_HOST` 200 | 201 | `portEnvVarName` - customize the name of the environment variable passed to the electron script that specifies the worker port. defaults to `ELECTRON_WORKER_PORT` 202 | 203 | Troubleshooting 204 | --------------- 205 | 206 | If you are using node with [nvm](https://github.com/creationix/nvm) and you have installed electron with `npm install -g electron-prebuilt` you probably will see an error or log with `env: node: No such file or directory`, this is because the electron executable installed by `electron-prebuilt` is a node CLI spawning the real electron executable internally, since nvm don't install/symlink node to `/usr/bin/env/node` when the electron executable installed by `electron-prebuilt` tries to run, it will fail because `node` won't be found in that context. 207 | 208 | Solution 209 | -------- 210 | 211 | 1.- Install `electron-prebuilt` as a dependency in your app, this is the **recommended** option because you probably want to ensure your app will always run with the exact version you tested, and you probably don't want to install electron globally on your system. 212 | 213 | 2.- You can make a symlink to `/usr/bin/env/node` but this is **not recommended** by nvm authors, because you will lose all the power that nvm brings. 214 | 215 | 3.- Put the path to the **real electron executable** in your `$PATH`. 216 | 217 | License 218 | ------- 219 | 220 | See [license](https://github.com/bjrmatos/electron-workers/blob/master/LICENSE) 221 | -------------------------------------------------------------------------------- /appveyor.yml: -------------------------------------------------------------------------------- 1 | 2 | # Test against this version of Node.js 3 | environment: 4 | matrix: 5 | # node.js 6 | - nodejs_version: "0.10" 7 | - nodejs_version: "4.6" 8 | - nodejs_version: "6.9" 9 | 10 | # Install scripts. (runs after repo cloning) 11 | install: 12 | # Get the latest stable version of Node.js or io.js 13 | - ps: Install-Product node $env:nodejs_version 14 | # install modules 15 | - npm install 16 | 17 | # Post-install test scripts. 18 | test_script: 19 | # Output useful info for debugging. 20 | - node --version 21 | - npm --version 22 | # run tests 23 | - npm test 24 | 25 | # Don't actually build. 26 | build: off 27 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "electron-workers", 3 | "version": "1.10.3", 4 | "description": "Run electron scripts in managed workers", 5 | "main": "lib/index.js", 6 | "scripts": { 7 | "clean": "rimraf lib", 8 | "build": "babel src --out-dir lib", 9 | "lint": "eslint src test", 10 | "test": "mocha --timeout 9000", 11 | "prepublish": "in-publish && npm-run-all lint clean build || not-in-publish" 12 | }, 13 | "author": { 14 | "name": "BJR Matos", 15 | "email": "bjrmatos@gmail.com" 16 | }, 17 | "license": "MIT", 18 | "keywords": [ 19 | "electron", 20 | "headless", 21 | "workers", 22 | "electron spawn" 23 | ], 24 | "homepage": "https://github.com/bjrmatos/electron-workers", 25 | "repository": { 26 | "type": "git", 27 | "url": "git@github.com:bjrmatos/electron-workers.git" 28 | }, 29 | "dependencies": { 30 | "debug": "2.3.3", 31 | "lodash.findindex": "4.6.0", 32 | "net-cluster": "0.0.2", 33 | "portscanner": "1.2.0", 34 | "uuid": "3.0.1", 35 | "which": "1.2.12" 36 | }, 37 | "devDependencies": { 38 | "babel": "5.8.38", 39 | "electron": "1.6.6", 40 | "eslint": "2.13.1", 41 | "eslint-config-airbnb-base": "3.0.1", 42 | "eslint-plugin-import": "1.16.0", 43 | "in-publish": "2.0.0", 44 | "mocha": "2.5.3", 45 | "npm-run-all": "2.3.0", 46 | "rimraf": "2.5.4", 47 | "should": "9.0.2" 48 | } 49 | } 50 | -------------------------------------------------------------------------------- /src/ElectronManager.js: -------------------------------------------------------------------------------- 1 | 2 | /** 3 | * ElectronManager is responsible of managing pool of electron worker processes 4 | * and distributing tasks to them. 5 | */ 6 | 7 | import { EventEmitter } from 'events'; 8 | import os from 'os'; 9 | import debug from 'debug'; 10 | import which from 'which'; 11 | import findIndex from 'lodash.findindex'; 12 | import ElectronWorker from './ElectronWorker'; 13 | import { name as pkgName } from '../package.json'; 14 | 15 | const numCPUs = os.cpus().length, 16 | debugManager = debug(`${pkgName}:manager`); 17 | 18 | let ELECTRON_PATH; 19 | 20 | function getElectronPath() { 21 | let electron; 22 | 23 | if (ELECTRON_PATH) { 24 | debugManager('getting electron path from cache'); 25 | return ELECTRON_PATH; 26 | } 27 | 28 | // first try to find the electron executable if it is installed from `electron`.. 29 | electron = getElectronPathFromPackage('electron'); 30 | 31 | if (electron == null) { 32 | // second try to find the electron executable if it is installed from `electron-prebuilt`.. 33 | electron = getElectronPathFromPackage('electron-prebuilt'); 34 | } 35 | 36 | if (electron == null) { 37 | // last try to find the electron executable, trying using which module 38 | debugManager('trying to get electron path from $PATH..'); 39 | 40 | try { 41 | electron = which.sync('electron'); 42 | } catch (whichErr) { 43 | throw new Error( 44 | 'Couldn\'t find the path to the electron executable automatically, ' + 45 | 'try installing the `electron` or `electron-prebuilt` package, ' + 46 | 'or set the `pathToElectron` option to specify the path manually' 47 | ); 48 | } 49 | } 50 | 51 | ELECTRON_PATH = electron; 52 | 53 | return electron; 54 | } 55 | 56 | function getElectronPathFromPackage(moduleName) { 57 | let electronPath; 58 | 59 | try { 60 | debugManager(`trying to get electron path from "${moduleName}" module..`); 61 | 62 | // eslint-disable-next-line global-require 63 | electronPath = require(moduleName); 64 | 65 | return electronPath; 66 | } catch (err) { 67 | if (err.code === 'MODULE_NOT_FOUND') { 68 | return electronPath; 69 | } 70 | 71 | throw err; 72 | } 73 | } 74 | 75 | class ElectronManager extends EventEmitter { 76 | constructor(options = {}) { 77 | super(); 78 | 79 | let instance = this; 80 | 81 | this._electronInstances = []; 82 | this._electronInstancesTasksCount = {}; 83 | this.options = { ...options }; 84 | this.options.connectionMode = this.options.connectionMode || 'server'; 85 | this.options.electronArgs = this.options.electronArgs || []; 86 | this.options.pathToElectron = this.options.pathToElectron || getElectronPath(); 87 | this.options.numberOfWorkers = this.options.numberOfWorkers || numCPUs; 88 | this.options.maxConcurrencyPerWorker = this.options.maxConcurrencyPerWorker || Infinity; 89 | this.options.pingTimeout = this.options.pingTimeout || 100; 90 | this.options.timeout = this.options.timeout || 10000; 91 | this.options.host = this.options.host || 'localhost'; 92 | this.options.hostEnvVarName = this.options.hostEnvVarName || 'ELECTRON_WORKER_HOST'; 93 | this.options.portEnvVarName = this.options.portEnvVarName || 'ELECTRON_WORKER_PORT'; 94 | this._timeouts = []; 95 | this.tasksQueue = []; 96 | 97 | if (isNaN(this.options.maxConcurrencyPerWorker) || 98 | typeof this.options.maxConcurrencyPerWorker !== 'number') { 99 | throw new Error('`maxConcurrencyPerWorker` option must be a number'); 100 | } 101 | 102 | if (this.options.maxConcurrencyPerWorker <= 0) { 103 | throw new Error('`maxConcurrencyPerWorker` option must be greater than 0'); 104 | } 105 | 106 | function processExitHandler() { 107 | debugManager('process exit: trying to kill workers..'); 108 | instance.kill(); 109 | } 110 | 111 | this._processExitHandler = processExitHandler; 112 | 113 | process.once('exit', processExitHandler); 114 | } 115 | 116 | start(cb) { 117 | let started = 0, 118 | workerErrors = [], 119 | { numberOfWorkers, connectionMode } = this.options, 120 | couldNotStartWorkersErr; 121 | 122 | if (connectionMode !== 'server' && connectionMode !== 'ipc') { 123 | return cb(new Error(`invalid connection mode: ${connectionMode}`)); 124 | } 125 | 126 | debugManager(`starting ${numberOfWorkers} worker(s), mode: ${connectionMode}..`); 127 | 128 | function startHandler(err) { 129 | if (err) { 130 | workerErrors.push(err); 131 | } 132 | 133 | started++; 134 | 135 | if (started === numberOfWorkers) { 136 | if (workerErrors.length) { 137 | couldNotStartWorkersErr = new Error('electron manager could not start all workers..'); 138 | couldNotStartWorkersErr.workerErrors = workerErrors; 139 | debugManager('electron manager could not start all workers..'); 140 | return cb(couldNotStartWorkersErr); 141 | } 142 | 143 | debugManager('all workers started correctly'); 144 | cb(null); 145 | } 146 | } 147 | 148 | for (let ix = 0; ix < numberOfWorkers; ix++) { 149 | let workerPortLeftBoundary = this.options.portLeftBoundary, 150 | workerOptions, 151 | workerInstance; 152 | 153 | // prevent that workers start with the same left boundary 154 | if (workerPortLeftBoundary != null) { 155 | workerPortLeftBoundary += ix; 156 | } 157 | 158 | workerOptions = { 159 | debug: this.options.debug, 160 | debugBrk: this.options.debugBrk, 161 | env: this.options.env, 162 | stdio: this.options.stdio, 163 | connectionMode: this.options.connectionMode, 164 | pingTimeout: this.options.pingTimeout, 165 | killSignal: this.options.killSignal, 166 | electronArgs: this.options.electronArgs, 167 | pathToElectron: this.options.pathToElectron, 168 | pathToScript: this.options.pathToScript, 169 | hostEnvVarName: this.options.hostEnvVarName, 170 | portEnvVarName: this.options.portEnvVarName, 171 | host: this.options.host, 172 | portLeftBoundary: workerPortLeftBoundary, 173 | portRightBoundary: this.options.portRightBoundary 174 | }; 175 | 176 | debugManager(`creating worker ${ix + 1} with options:`, workerOptions); 177 | workerInstance = new ElectronWorker(workerOptions); 178 | 179 | workerInstance.on('processCreated', () => { 180 | this.emit('workerProcessCreated', workerInstance, workerInstance._childProcess); 181 | }); 182 | 183 | workerInstance.on('recycling', () => { 184 | if (this._electronInstancesTasksCount[workerInstance.id] != null) { 185 | this._electronInstancesTasksCount[workerInstance.id] = 0; 186 | } 187 | 188 | this.emit('workerRecycling', workerInstance); 189 | }); 190 | 191 | workerInstance.on('recyclingError', () => { 192 | this.emit('workerRecyclingError', workerInstance); 193 | this.tryFlushQueue(); 194 | }); 195 | 196 | workerInstance.on('recycled', () => { 197 | this.emit('workerRecycled', workerInstance); 198 | this.tryFlushQueue(); 199 | }); 200 | 201 | workerInstance.on('kill', () => { 202 | if (this._electronInstancesTasksCount[workerInstance.id] != null) { 203 | this._electronInstancesTasksCount[workerInstance.id] = 0; 204 | } 205 | }); 206 | 207 | this._electronInstances.push(workerInstance); 208 | this._electronInstancesTasksCount[workerInstance.id] = 0; 209 | 210 | this._electronInstances[ix].start(startHandler); 211 | } 212 | } 213 | 214 | execute(data, ...args) { 215 | let availableWorkerInstanceIndex, 216 | availableWorkerInstance, 217 | options, 218 | cb; 219 | 220 | if (args.length > 1) { 221 | options = args[0]; 222 | cb = args[1]; 223 | } else { 224 | cb = args[0]; 225 | } 226 | 227 | debugManager('getting new task..'); 228 | 229 | // simple round robin balancer across workers 230 | // on each execute, get the first available worker from the list... 231 | availableWorkerInstanceIndex = findIndex(this._electronInstances, { 232 | isBusy: false 233 | }); 234 | 235 | if (availableWorkerInstanceIndex !== -1) { 236 | availableWorkerInstance = this._electronInstances.splice(availableWorkerInstanceIndex, 1)[0]; 237 | 238 | this._manageTaskStartInWorker(availableWorkerInstance); 239 | 240 | debugManager(`worker [${availableWorkerInstance.id}] has been choosen for the task..`); 241 | 242 | this._executeInWorker(availableWorkerInstance, data, options, cb); 243 | // ..and then the worker we have used becomes the last item in the list 244 | this._electronInstances.push(availableWorkerInstance); 245 | return; 246 | } 247 | 248 | debugManager('no workers available, storing the task for later processing..'); 249 | // if no available worker save task for later processing 250 | this.tasksQueue.push({ data, options, cb }); 251 | } 252 | 253 | _manageTaskStartInWorker(worker) { 254 | const maxConcurrencyPerWorker = this.options.maxConcurrencyPerWorker; 255 | 256 | if (this._electronInstancesTasksCount[worker.id] == null) { 257 | this._electronInstancesTasksCount[worker.id] = 0; 258 | } 259 | 260 | if (this._electronInstancesTasksCount[worker.id] < maxConcurrencyPerWorker) { 261 | this._electronInstancesTasksCount[worker.id]++; 262 | } 263 | 264 | // "equality check" is just enough here but we apply the "greater than" check just in case.. 265 | if (this._electronInstancesTasksCount[worker.id] >= maxConcurrencyPerWorker) { 266 | worker.isBusy = true; // eslint-disable-line no-param-reassign 267 | } 268 | } 269 | 270 | _manageTaskEndInWorker(worker) { 271 | const maxConcurrencyPerWorker = this.options.maxConcurrencyPerWorker; 272 | 273 | if (this._electronInstancesTasksCount[worker.id] == null) { 274 | this._electronInstancesTasksCount[worker.id] = 0; 275 | } 276 | 277 | if (this._electronInstancesTasksCount[worker.id] > 0) { 278 | this._electronInstancesTasksCount[worker.id]--; 279 | } 280 | 281 | if (this._electronInstancesTasksCount[worker.id] < maxConcurrencyPerWorker) { 282 | worker.isBusy = false; // eslint-disable-line no-param-reassign 283 | } 284 | } 285 | 286 | _executeInWorker(worker, data, options = {}, cb) { 287 | let workerTimeout; 288 | 289 | if (options.timeout != null) { 290 | workerTimeout = options.timeout; 291 | } else { 292 | workerTimeout = this.options.timeout; 293 | } 294 | 295 | if (worker.shouldRevive) { 296 | debugManager(`trying to revive worker [${worker.id}]..`); 297 | 298 | worker.start((startErr) => { 299 | if (startErr) { 300 | debugManager(`worker [${worker.id}] could not revive..`); 301 | this.tryFlushQueue(); 302 | return cb(startErr); 303 | } 304 | 305 | debugManager(`worker [${worker.id}] has revived..`); 306 | executeTask.call(this); 307 | }); 308 | } else { 309 | executeTask.call(this); 310 | } 311 | 312 | function executeTask() { 313 | let isDone = false; 314 | 315 | let timeoutId = setTimeout(() => { 316 | this._timeouts.splice(this._timeouts.indexOf(timeoutId), 1); 317 | 318 | if (isDone) { 319 | return; 320 | } 321 | 322 | debugManager(`task timeout in worker [${worker.id}] has been reached..`); 323 | 324 | isDone = true; 325 | 326 | this._manageTaskEndInWorker(worker); 327 | 328 | this.emit('workerTimeout', worker); 329 | 330 | let error = new Error(); 331 | error.workerTimeout = true; 332 | error.message = `Worker Timeout, the worker process does not respond after ${workerTimeout} ms`; 333 | cb(error); 334 | 335 | this.tryFlushQueue(); 336 | }, workerTimeout); 337 | 338 | debugManager(`executing task in worker [${worker.id}] with timeout:`, workerTimeout); 339 | 340 | this._timeouts.push(timeoutId); 341 | 342 | worker.execute(data, (err, result) => { 343 | if (isDone) { 344 | return; 345 | } 346 | 347 | this._manageTaskEndInWorker(worker); 348 | 349 | // clear timeout 350 | this._timeouts.splice(this._timeouts.indexOf(timeoutId), 1); 351 | clearTimeout(timeoutId); 352 | 353 | if (err) { 354 | debugManager(`task has failed in worker [${worker.id}]..`); 355 | this.tryFlushQueue(); 356 | cb(err); 357 | return; 358 | } 359 | 360 | isDone = true; 361 | debugManager(`task executed correctly in worker [${worker.id}]..`); 362 | this.tryFlushQueue(); 363 | cb(null, result); 364 | }); 365 | } 366 | } 367 | 368 | tryFlushQueue() { 369 | let availableWorkerInstanceIndex, 370 | availableWorkerInstance, 371 | task; 372 | 373 | debugManager('trying to flush queue of pending tasks..'); 374 | 375 | if (this.tasksQueue.length === 0) { 376 | debugManager('there is no pending tasks..'); 377 | return; 378 | } 379 | 380 | // simple round robin balancer across workers 381 | // get the first available worker from the list... 382 | availableWorkerInstanceIndex = findIndex(this._electronInstances, { 383 | isBusy: false 384 | }); 385 | 386 | if (availableWorkerInstanceIndex === -1) { 387 | debugManager('no workers available to process pending task..'); 388 | return; 389 | } 390 | 391 | task = this.tasksQueue.shift(); 392 | availableWorkerInstance = this._electronInstances.splice(availableWorkerInstanceIndex, 1)[0]; 393 | 394 | this._manageTaskStartInWorker(availableWorkerInstance); 395 | 396 | debugManager(`worker [${availableWorkerInstance.id}] has been choosen for process pending task..`); 397 | 398 | this._executeInWorker(availableWorkerInstance, task.data, task.options, task.cb); 399 | // ..and then the worker we have used becomes the last item in the list 400 | this._electronInstances.push(availableWorkerInstance); 401 | } 402 | 403 | kill() { 404 | debugManager('killing all workers..'); 405 | 406 | this._timeouts.forEach((tId) => { 407 | clearTimeout(tId); 408 | }); 409 | 410 | this._electronInstances.forEach((workerInstance) => { 411 | workerInstance.kill(true); 412 | }); 413 | 414 | process.removeListener('exit', this._processExitHandler); 415 | } 416 | } 417 | 418 | export default ElectronManager; 419 | -------------------------------------------------------------------------------- /src/ElectronWorker.js: -------------------------------------------------------------------------------- 1 | 2 | import { EventEmitter } from 'events'; 3 | import childProcess from 'child_process'; 4 | import cluster from 'cluster'; 5 | import http from 'http'; 6 | import debugPkg from 'debug'; 7 | import netCluster from 'net-cluster'; 8 | import portScanner from 'portscanner'; 9 | import uuid from 'uuid'; 10 | import checkPortStatus from './checkPortStatus'; 11 | import checkIpcStatus from './checkIpcStatus'; 12 | import { name as pkgName } from '../package.json'; 13 | 14 | const debugWorker = debugPkg(`${pkgName}:worker`); 15 | 16 | function findFreePort(host, cb) { 17 | let server = netCluster.createServer(), 18 | port = 0; 19 | 20 | debugWorker('trying to find free port..'); 21 | 22 | server.on('listening', () => { 23 | port = server.address().port; 24 | server.close(); 25 | }); 26 | 27 | server.on('close', () => { 28 | cb(null, port); 29 | }); 30 | 31 | server.listen(0, host); 32 | } 33 | 34 | function findFreePortInRange(host, portLeftBoundary, portRightBoundary, cb) { 35 | let newPortLeftBoundary = portLeftBoundary; 36 | 37 | // in cluster we don't want ports to collide, so we make a special space for every 38 | // worker assuming max number of cluster workers is 5 39 | if (cluster.worker) { 40 | newPortLeftBoundary = portLeftBoundary + (((portRightBoundary - portLeftBoundary) / 5) * (cluster.worker.id - 1)); 41 | } 42 | 43 | debugWorker(`trying to find free port in range ${newPortLeftBoundary}-${portRightBoundary}`); 44 | 45 | portScanner.findAPortNotInUse(newPortLeftBoundary, portRightBoundary, host, (error, port) => { 46 | cb(error, port); 47 | }); 48 | } 49 | 50 | function isValidConnectionMode(mode) { 51 | if (mode !== 'server' && mode !== 'ipc') { 52 | return false; 53 | } 54 | 55 | return true; 56 | } 57 | 58 | class ElectronWorker extends EventEmitter { 59 | constructor(options) { 60 | super(); 61 | 62 | this.options = options; 63 | this.firstStart = false; 64 | this.shouldRevive = false; 65 | this.exit = false; 66 | this.isBusy = false; 67 | this.isRecycling = false; 68 | this.id = uuid.v1(); 69 | this._hardKill = false; 70 | this._earlyError = false; 71 | this._taskCallback = {}; 72 | 73 | this.onWorkerProcessError = this.onWorkerProcessError.bind(this); 74 | this.onWorkerProcessExitTryToRecyle = this.onWorkerProcessExitTryToRecyle.bind(this); 75 | this.onWorkerProcessIpcMessage = this.onWorkerProcessIpcMessage.bind(this); 76 | 77 | if (options.connectionMode === 'ipc') { 78 | this.findFreePort = function(cb) { 79 | cb(null); 80 | }; 81 | } else { 82 | if (options.portLeftBoundary && options.portRightBoundary) { 83 | this.findFreePort = function(cb) { 84 | findFreePortInRange(options.host, options.portLeftBoundary, options.portRightBoundary, cb); 85 | }; 86 | } else { 87 | this.findFreePort = function(cb) { 88 | findFreePort(options.host, cb); 89 | }; 90 | } 91 | } 92 | } 93 | 94 | onWorkerProcessError(workerProcessErr) { 95 | debugWorker(`worker [${this.id}] electron process error callback: ${workerProcessErr.message}`); 96 | 97 | // don't handle early errors (errors between spawning the process and the first checkAlive call) in this handler 98 | if (this._earlyError) { 99 | debugWorker(`worker [${this.id}] ignoring error because it was handled previously (early): ${workerProcessErr.message}`); 100 | return; 101 | } 102 | 103 | // try revive the process when an error is received, 104 | // note that could not be spawn errors are not handled here.. 105 | if (this.firstStart && !this.isRecycling && !this.shouldRevive) { 106 | debugWorker(`worker [${this.id}] the process will be revived because an error: ${workerProcessErr.message}`); 107 | this.shouldRevive = true; 108 | } 109 | } 110 | 111 | onWorkerProcessExitTryToRecyle(code, signal) { 112 | debugWorker(`worker [${this.id}] onWorkerProcessExitTryToRecyle callback..`); 113 | 114 | if (code != null || signal != null) { 115 | debugWorker(`worker [${this.id}] electron process exit with code: ${code} and signal: ${signal}`); 116 | } 117 | 118 | // we only recycle the process on exit and if it is not in the middle 119 | // of another recycling 120 | if (this.firstStart && !this.isRecycling) { 121 | debugWorker(`trying to recycle worker [${this.id}], reason: process exit..`); 122 | 123 | this.exit = true; 124 | this.firstStart = false; 125 | 126 | this.recycle(() => { 127 | this.exit = false; 128 | }); 129 | } 130 | } 131 | 132 | onWorkerProcessIpcMessage(payload) { 133 | let callback, 134 | responseData; 135 | 136 | if (payload && payload.workerEvent === 'taskResponse') { 137 | debugWorker(`task in worker [${this.id}] has ended..`); 138 | 139 | callback = this._taskCallback[payload.taskId]; 140 | responseData = payload.response; 141 | 142 | if (!callback || typeof callback !== 'function') { 143 | debugWorker(`worker [${this.id}] - callback registered for the task's response (${payload.taskId}) is not a function`); 144 | return; 145 | } 146 | 147 | if (payload.error) { 148 | let errorSerialized = JSON.stringify(payload.error); 149 | 150 | debugWorker(`task in worker [${this.id}] ended with error: ${errorSerialized}`); 151 | 152 | return callback(new Error( 153 | payload.error.message || 154 | `An error has occurred when trying to process the task: ${errorSerialized}` 155 | )); 156 | } 157 | 158 | debugWorker(`task in worker [${this.id}] ended successfully`); 159 | 160 | callback(null, responseData); 161 | } 162 | } 163 | 164 | start(cb) { 165 | let isDone = false; 166 | 167 | if (!isValidConnectionMode(this.options.connectionMode)) { 168 | return cb(new Error(`invalid connection mode: ${this.options.connectionMode}`)); 169 | } 170 | 171 | debugWorker(`starting worker [${this.id}]..`); 172 | 173 | this.findFreePort((err, port) => { 174 | let childArgs, 175 | childOpts; 176 | 177 | let { 178 | electronArgs, 179 | pathToElectron, 180 | pathToScript, 181 | hostEnvVarName, 182 | portEnvVarName, 183 | host, 184 | debug, 185 | debugBrk, 186 | env, 187 | stdio, 188 | connectionMode 189 | } = this.options; 190 | 191 | if (!env) { 192 | env = {}; 193 | } 194 | 195 | childArgs = electronArgs.slice(); 196 | childArgs.unshift(pathToScript); 197 | 198 | if (debugBrk != null) { 199 | childArgs.unshift(`--debug-brk=${debugBrk}`); 200 | } else if (debug != null) { 201 | childArgs.unshift(`--debug=${debug}`); 202 | } 203 | 204 | if (err) { 205 | debugWorker(`couldn't find free port for worker [${this.id}]..`); 206 | return cb(err); 207 | } 208 | 209 | this.port = port; 210 | 211 | childOpts = { 212 | env: { 213 | ...env, 214 | ELECTRON_WORKER_ID: this.id, 215 | // propagate the DISPLAY env var to make it work on LINUX 216 | DISPLAY: process.env.DISPLAY 217 | } 218 | }; 219 | 220 | // we send host and port as env vars to child process in server mode 221 | if (connectionMode === 'server') { 222 | childOpts.stdio = 'pipe'; 223 | childOpts.env[hostEnvVarName] = host; 224 | childOpts.env[portEnvVarName] = port; 225 | } else if (connectionMode === 'ipc') { 226 | childOpts.stdio = ['pipe', 'pipe', 'pipe', 'ipc']; 227 | } 228 | 229 | if (stdio != null) { 230 | childOpts.stdio = stdio; 231 | } 232 | 233 | debugWorker(`spawning process for worker [${this.id}] with args:`, childArgs, 'and options:', childOpts); 234 | 235 | this._childProcess = childProcess.spawn(pathToElectron, childArgs, childOpts); 236 | 237 | debugWorker(`electron process pid for worker [${this.id}]:`, this._childProcess.pid); 238 | 239 | // ipc connection is required for ipc mode 240 | if (connectionMode === 'ipc' && !this._childProcess.send) { 241 | return cb(new Error( 242 | 'ipc mode requires a ipc connection, if you\'re using stdio option make sure you are setting up ipc' 243 | )); 244 | } 245 | 246 | this._handleSpawnError = function(spawnError) { 247 | debugWorker(`worker [${this.id}] spawn error callback..`); 248 | 249 | if (!this.firstStart) { 250 | isDone = true; 251 | this._earlyError = true; 252 | debugWorker(`worker [${this.id}] start was canceled because an early error: ${spawnError.message}`); 253 | cb(spawnError); 254 | } 255 | }; 256 | 257 | this._handleSpawnError = this._handleSpawnError.bind(this); 258 | 259 | this._childProcess.once('error', this._handleSpawnError); 260 | 261 | this._childProcess.on('error', this.onWorkerProcessError); 262 | 263 | this._childProcess.on('exit', this.onWorkerProcessExitTryToRecyle); 264 | 265 | if (connectionMode === 'ipc') { 266 | this._childProcess.on('message', this.onWorkerProcessIpcMessage); 267 | } 268 | 269 | this.emit('processCreated'); 270 | 271 | setImmediate(() => { 272 | // the workers were killed explicitly by the user 273 | if (this._hardKill || isDone) { 274 | return; 275 | } 276 | 277 | if (this._childProcess == null) { 278 | debugWorker(`There is no child process for worker [${this.id}]..`); 279 | return cb(new Error('There is no child process for worker')); 280 | } 281 | 282 | debugWorker(`checking if worker [${this.id}] is alive..`); 283 | 284 | this.checkAlive((checkAliveErr) => { 285 | if (isDone) { 286 | return; 287 | } 288 | 289 | if (checkAliveErr) { 290 | debugWorker(`worker [${this.id}] is not alive..`); 291 | return cb(checkAliveErr); 292 | } 293 | 294 | this._earlyError = false; 295 | this._childProcess.removeListener('error', this._handleSpawnError); 296 | 297 | if (!this.firstStart) { 298 | this.firstStart = true; 299 | } 300 | 301 | debugWorker(`worker [${this.id}] is alive..`); 302 | cb(); 303 | }); 304 | }); 305 | }); 306 | } 307 | 308 | checkAlive(cb, shot) { 309 | let shotCount = shot || 1, 310 | connectionMode = this.options.connectionMode; 311 | 312 | function statusHandler(err, statusWorker) { 313 | if (!err && statusWorker === 'open') { 314 | return cb(); 315 | } 316 | 317 | if (connectionMode === 'server' && shotCount > 50) { 318 | return cb(new Error(`Unable to reach electron worker - mode: ${connectionMode}, ${(err || {}).message}`)); 319 | } 320 | 321 | if (connectionMode === 'ipc' && err) { 322 | return cb(err); 323 | } 324 | 325 | shotCount++; 326 | 327 | // re-try check 328 | if (connectionMode === 'server') { 329 | setTimeout(() => { 330 | this.checkAlive(cb, shotCount); 331 | }, 100); 332 | } 333 | } 334 | 335 | if (connectionMode === 'server') { 336 | checkPortStatus(this.options.pingTimeout, this.port, this.options.host, statusHandler.bind(this)); 337 | } else if (connectionMode === 'ipc') { 338 | checkIpcStatus(this.options.pingTimeout, this._childProcess, statusHandler.bind(this)); 339 | } 340 | } 341 | 342 | execute(data, cb) { 343 | let connectionMode = this.options.connectionMode, 344 | httpOpts, 345 | req, 346 | json, 347 | taskId; 348 | 349 | debugWorker(`new task for worker [${this.id}]..`); 350 | 351 | this.emit('task'); 352 | 353 | if (this._hardKill) { 354 | debugWorker(`task execution stopped because worker [${this.id}] was killed by the user..`); 355 | return; 356 | } 357 | 358 | if (connectionMode === 'ipc') { 359 | debugWorker(`creating ipc task message for worker [${this.id}]..`); 360 | 361 | taskId = uuid.v1(); 362 | 363 | this._taskCallback[taskId] = (...args) => { 364 | this.emit('taskEnd'); 365 | cb.apply(undefined, args); 366 | }; 367 | 368 | return this._childProcess.send({ 369 | workerEvent: 'task', 370 | taskId, 371 | payload: data 372 | }); 373 | } 374 | 375 | debugWorker(`creating request for worker [${this.id}]..`); 376 | 377 | httpOpts = { 378 | hostname: this.options.host, 379 | port: this.port, 380 | path: '/', 381 | method: 'POST' 382 | }; 383 | 384 | req = http.request(httpOpts, (res) => { 385 | let result = ''; 386 | 387 | res.on('data', (chunk) => { 388 | result += chunk; 389 | }); 390 | 391 | res.on('end', () => { 392 | let responseData; 393 | 394 | debugWorker(`request in worker [${this.id}] has ended..`); 395 | 396 | this.emit('taskEnd'); 397 | 398 | try { 399 | debugWorker(`trying to parse worker [${this.id}] response..`); 400 | responseData = result ? JSON.parse(result) : null; 401 | } catch (err) { 402 | debugWorker(`couldn't parse response for worker [${this.id}]..`); 403 | return cb(err); 404 | } 405 | 406 | debugWorker(`response has been parsed correctly for worker [${this.id}]..`); 407 | cb(null, responseData); 408 | }); 409 | }); 410 | 411 | req.setHeader('Content-Type', 'application/json'); 412 | json = JSON.stringify(data); 413 | req.setHeader('Content-Length', Buffer.byteLength(json)); 414 | 415 | debugWorker(`trying to communicate with worker [${this.id}], request options:`, httpOpts, 'data:', json); 416 | 417 | req.write(json); 418 | 419 | req.on('error', (err) => { 420 | debugWorker(`error when trying to communicate with worker [${this.id}]..`); 421 | cb(err); 422 | }); 423 | 424 | req.end(); 425 | } 426 | 427 | recycle(...args) { 428 | let cb, 429 | revive; 430 | 431 | debugWorker(`recycling worker [${this.id}]..`); 432 | 433 | if (args.length < 2) { 434 | cb = args[0]; 435 | revive = true; 436 | } else { 437 | cb = args[1]; 438 | revive = args[0]; 439 | } 440 | 441 | if (this._childProcess) { 442 | this.isRecycling = true; 443 | // mark worker as busy before recycling 444 | this.isBusy = true; 445 | 446 | this.emit('recycling'); 447 | 448 | if (this._hardKill) { 449 | debugWorker(`recycling was stopped because worker [${this.id}] was killed by the user..`); 450 | return; 451 | } 452 | 453 | this.kill(); 454 | 455 | debugWorker(`trying to re-start child process for worker [${this.id}]..`); 456 | 457 | this.start((startErr) => { 458 | this.isRecycling = false; 459 | // mark worker as free after recycling 460 | this.isBusy = false; 461 | 462 | // if there is a error on worker recycling, revive it on next execute 463 | if (startErr) { 464 | this.shouldRevive = Boolean(revive); 465 | 466 | debugWorker(`couldn't recycle worker [${this.id}], should revive: ${this.shouldRevive}`); 467 | 468 | cb(startErr); 469 | this.emit('recyclingError', startErr); 470 | return; 471 | } 472 | 473 | debugWorker(`worker [${this.id}] has been recycled..`); 474 | 475 | this.shouldRevive = false; 476 | 477 | cb(); 478 | 479 | this.emit('recycled'); 480 | }); 481 | } else { 482 | debugWorker(`there is no child process to recycle - worker [${this.id}]`); 483 | } 484 | } 485 | 486 | kill(hardKill) { 487 | let connectionMode = this.options.connectionMode; 488 | 489 | debugWorker(`killing worker [${this.id}]..`); 490 | 491 | this.emit('kill'); 492 | 493 | this._hardKill = Boolean(hardKill); 494 | 495 | if (this._childProcess) { 496 | if (this._childProcess.connected) { 497 | debugWorker(`closing ipc connection - worker [${this.id}]..`); 498 | this._childProcess.disconnect(); 499 | } 500 | 501 | // clean previous listeners 502 | if (this._handleSpawnError) { 503 | this._childProcess.removeListener('error', this._handleSpawnError); 504 | } 505 | 506 | this._childProcess.removeListener('error', this.onWorkerProcessError); 507 | this._childProcess.removeListener('exit', this.onWorkerProcessExitTryToRecyle); 508 | 509 | if (connectionMode === 'ipc') { 510 | this._childProcess.removeListener('message', this.onWorkerProcessIpcMessage); 511 | } 512 | 513 | // guard against closing a process that has been closed before 514 | if (!this.exit) { 515 | if (this.options.killSignal) { 516 | debugWorker(`killing worker [${this.id}] with custom signal:`, this.options.killSignal); 517 | this._childProcess.kill(this.options.killSignal); 518 | } else { 519 | this._childProcess.kill(); 520 | } 521 | 522 | if (!hardKill) { 523 | this.onWorkerProcessExitTryToRecyle(); 524 | } 525 | } 526 | 527 | this._childProcess = undefined; 528 | } else { 529 | debugWorker(`there is no child process to kill - worker [${this.id}]`); 530 | } 531 | } 532 | } 533 | 534 | export default ElectronWorker; 535 | -------------------------------------------------------------------------------- /src/checkIpcStatus.js: -------------------------------------------------------------------------------- 1 | 2 | export default function(timeout, processObj, cb) { 3 | let isDone = false, 4 | timeoutId; 5 | 6 | function pongHandler(payload) { 7 | if (payload && payload.workerEvent === 'pong') { 8 | isDone = true; 9 | clearTimeout(timeoutId); 10 | processObj.removeListener('message', pongHandler); 11 | cb(null, 'open'); 12 | } 13 | } 14 | 15 | processObj.on('message', pongHandler); 16 | 17 | tryCommunication(); 18 | 19 | function tryCommunication(shotCount = 1) { 20 | if (isDone) { 21 | return; 22 | } 23 | 24 | processObj.send({ 25 | workerEvent: 'ping' 26 | }, undefined, (err) => { 27 | if (isDone) { 28 | return; 29 | } 30 | 31 | if (shotCount <= 200) { 32 | return; 33 | } 34 | 35 | if (err) { 36 | isDone = true; 37 | clearTimeout(timeoutId); 38 | cb(new Error('ipc message could not be sent to electron process')); 39 | } 40 | }); 41 | 42 | timeoutId = setTimeout(() => { 43 | if (isDone) { 44 | return; 45 | } 46 | 47 | if (shotCount > 200) { 48 | isDone = true; 49 | return cb(new Error(`Worker timeout (${timeout} ms) ocurred waiting for ipc connection to be available`)); 50 | } 51 | 52 | tryCommunication(shotCount + 1); 53 | }, timeout); 54 | } 55 | } 56 | -------------------------------------------------------------------------------- /src/checkPortStatus.js: -------------------------------------------------------------------------------- 1 | 2 | import { Socket } from 'net'; 3 | 4 | export default function(timeout, port, host, cb) { 5 | let connectionRefused = false, 6 | portStatus = null, 7 | error = null, 8 | socket; 9 | 10 | socket = new Socket(); 11 | 12 | // Socket connection established, port is open 13 | socket.on('connect', () => { 14 | portStatus = 'open'; 15 | socket.destroy(); 16 | }); 17 | 18 | // If no response, assume port is not listening 19 | socket.setTimeout(timeout); 20 | 21 | socket.on('timeout', () => { 22 | portStatus = 'closed'; 23 | error = new Error(`Worker timeout (${timeout} ms) ocurred waiting for ${host}:${port} to be available`); 24 | socket.destroy(); 25 | }); 26 | 27 | socket.on('error', (exception) => { 28 | if (exception.code !== 'ECONNREFUSED') { 29 | error = exception; 30 | } else { 31 | connectionRefused = true; 32 | } 33 | 34 | portStatus = 'closed'; 35 | }); 36 | 37 | // Return after the socket has closed 38 | socket.on('close', (exception) => { 39 | if (exception && !connectionRefused) { 40 | error = exception; 41 | } else { 42 | error = null; 43 | } 44 | 45 | cb(error, portStatus); 46 | }); 47 | 48 | socket.connect(port, host); 49 | } 50 | -------------------------------------------------------------------------------- /src/index.js: -------------------------------------------------------------------------------- 1 | 2 | import debug from 'debug'; 3 | import ElectronManager from './ElectronManager'; 4 | import { name as pkgName } from '../package.json'; 5 | 6 | const debugMe = debug(pkgName); 7 | 8 | function createManager(options) { 9 | let manager = new ElectronManager(options); 10 | debugMe('Creating a new manager with options:', manager.options); 11 | return manager; 12 | } 13 | 14 | function electronManager(options) { 15 | return createManager(options); 16 | } 17 | 18 | export default electronManager; 19 | -------------------------------------------------------------------------------- /test/.eslintrc: -------------------------------------------------------------------------------- 1 | { 2 | "env": { 3 | "mocha": true // adds all of the Mocha testing global variables. 4 | } 5 | } 6 | -------------------------------------------------------------------------------- /test/electron-script/ipc/concurrency.js: -------------------------------------------------------------------------------- 1 | 2 | // disabling eslint import because `electron` is a buil-in module 3 | // eslint-disable-next-line import/no-unresolved 4 | const { app } = require('electron'); 5 | 6 | const JOB_DURATION_MS = parseInt(process.env.JOB_DURATION_MS, 10); 7 | 8 | app.on('ready', () => { 9 | // first you will need to listen the `message` event in the process object 10 | process.on('message', (data) => { 11 | if (!data) { 12 | return; 13 | } 14 | 15 | if (data.workerEvent === 'ping') { 16 | process.send({ workerEvent: 'pong' }); 17 | } else if (data.workerEvent === 'task') { 18 | let started = Date.now(); 19 | 20 | // simulate 500ms duration 21 | setTimeout(() => { 22 | process.send({ 23 | workerEvent: 'taskResponse', 24 | taskId: data.taskId, 25 | response: { 26 | started, 27 | ended: Date.now() 28 | } 29 | }); 30 | }, JOB_DURATION_MS); 31 | } 32 | }); 33 | }); 34 | -------------------------------------------------------------------------------- /test/electron-script/ipc/custom-args.js: -------------------------------------------------------------------------------- 1 | 2 | // disabling eslint import because `electron` is a buil-in module 3 | // eslint-disable-next-line import/no-unresolved 4 | const { app } = require('electron'); 5 | 6 | app.on('ready', () => { 7 | process.on('message', (data) => { 8 | if (!data) { 9 | return; 10 | } 11 | 12 | if (data.workerEvent === 'ping') { 13 | process.send({ workerEvent: 'pong' }); 14 | } else if (data.workerEvent === 'task') { 15 | process.send({ 16 | workerEvent: 'taskResponse', 17 | taskId: data.taskId, 18 | response: process.argv.slice(2) 19 | }); 20 | } 21 | }); 22 | }); 23 | -------------------------------------------------------------------------------- /test/electron-script/ipc/env.js: -------------------------------------------------------------------------------- 1 | 2 | // disabling eslint import because `electron` is a buil-in module 3 | // eslint-disable-next-line import/no-unresolved 4 | const { app } = require('electron'); 5 | 6 | const foo = process.env.FOO, 7 | customEnv = process.env.CUSTOM_ENV; 8 | 9 | app.on('ready', () => { 10 | process.on('message', (data) => { 11 | if (!data) { 12 | return; 13 | } 14 | 15 | if (data.workerEvent === 'ping') { 16 | process.send({ workerEvent: 'pong' }); 17 | } else if (data.workerEvent === 'task') { 18 | process.send({ 19 | workerEvent: 'taskResponse', 20 | taskId: data.taskId, 21 | response: { 22 | FOO: foo, 23 | CUSTOM_ENV: customEnv 24 | } 25 | }); 26 | } 27 | }); 28 | }); 29 | -------------------------------------------------------------------------------- /test/electron-script/ipc/recycle-on-exit.js: -------------------------------------------------------------------------------- 1 | 2 | // disabling eslint import because `electron` is a buil-in module 3 | // eslint-disable-next-line import/no-unresolved 4 | const { app } = require('electron'); 5 | 6 | app.on('ready', () => { 7 | process.on('message', (data) => { 8 | if (!data) { 9 | return; 10 | } 11 | 12 | if (data.workerEvent === 'ping') { 13 | process.send({ workerEvent: 'pong' }); 14 | } else if (data.workerEvent === 'task') { 15 | process.send({ 16 | workerEvent: 'taskResponse', 17 | taskId: data.taskId, 18 | response: { 19 | ok: true 20 | } 21 | }); 22 | app.quit(); 23 | } 24 | }); 25 | }); 26 | -------------------------------------------------------------------------------- /test/electron-script/ipc/script.js: -------------------------------------------------------------------------------- 1 | 2 | // disabling eslint import because `electron` is a buil-in module 3 | // eslint-disable-next-line import/no-unresolved 4 | const { app } = require('electron'); 5 | 6 | if (!process.send) { 7 | app.quit(); 8 | } 9 | 10 | app.on('ready', () => { 11 | process.on('message', (data) => { 12 | if (!data) { 13 | return; 14 | } 15 | 16 | if (data.workerEvent === 'ping') { 17 | process.send({ workerEvent: 'pong' }); 18 | } else if (data.workerEvent === 'task') { 19 | process.send({ 20 | workerEvent: 'taskResponse', 21 | taskId: data.taskId, 22 | response: data.payload 23 | }); 24 | } 25 | }); 26 | }); 27 | -------------------------------------------------------------------------------- /test/electron-script/ipc/slowstart.js: -------------------------------------------------------------------------------- 1 | 2 | // disabling eslint import because `electron` is a buil-in module 3 | // eslint-disable-next-line import/no-unresolved 4 | const { app } = require('electron'); 5 | 6 | app.on('ready', () => { 7 | setTimeout(() => { 8 | process.on('message', (data) => { 9 | if (!data) { 10 | return; 11 | } 12 | 13 | if (data.workerEvent === 'ping') { 14 | process.send({ workerEvent: 'pong' }); 15 | } else if (data.workerEvent === 'task') { 16 | process.send({ 17 | workerEvent: 'taskResponse', 18 | taskId: data.taskId, 19 | response: { 20 | ok: true 21 | } 22 | }); 23 | } 24 | }); 25 | }, 2000); 26 | }); 27 | -------------------------------------------------------------------------------- /test/electron-script/ipc/stdio.js: -------------------------------------------------------------------------------- 1 | 2 | // disabling eslint import because `electron` is a buil-in module 3 | // eslint-disable-next-line import/no-unresolved 4 | const { app } = require('electron'); 5 | 6 | app.on('ready', () => { 7 | process.on('message', (data) => { 8 | if (!data) { 9 | return; 10 | } 11 | 12 | if (data.workerEvent === 'ping') { 13 | process.send({ workerEvent: 'pong' }); 14 | } else if (data.workerEvent === 'task') { 15 | process.send({ 16 | workerEvent: 'taskResponse', 17 | taskId: data.taskId, 18 | response: {} 19 | }); 20 | } 21 | }); 22 | }); 23 | -------------------------------------------------------------------------------- /test/electron-script/ipc/timeout.js: -------------------------------------------------------------------------------- 1 | 2 | // disabling eslint import because `electron` is a buil-in module 3 | // eslint-disable-next-line import/no-unresolved 4 | const { app } = require('electron'); 5 | 6 | app.on('ready', () => { 7 | setTimeout(() => { 8 | process.on('message', (data) => { 9 | if (!data) { 10 | return; 11 | } 12 | 13 | if (data.workerEvent === 'ping') { 14 | process.send({ workerEvent: 'pong' }); 15 | } 16 | }); 17 | }, 2000); 18 | }); 19 | -------------------------------------------------------------------------------- /test/electron-script/ipc/workerId.js: -------------------------------------------------------------------------------- 1 | 2 | // disabling eslint import because `electron` is a buil-in module 3 | // eslint-disable-next-line import/no-unresolved 4 | const { app } = require('electron'); 5 | 6 | const workerId = process.env.ELECTRON_WORKER_ID; 7 | 8 | app.on('ready', () => { 9 | process.on('message', (data) => { 10 | if (!data) { 11 | return; 12 | } 13 | 14 | if (data.workerEvent === 'ping') { 15 | process.send({ workerEvent: 'pong' }); 16 | } else if (data.workerEvent === 'task') { 17 | process.send({ 18 | workerEvent: 'taskResponse', 19 | taskId: data.taskId, 20 | response: workerId 21 | }); 22 | } 23 | }); 24 | }); 25 | -------------------------------------------------------------------------------- /test/electron-script/server/concurrency.js: -------------------------------------------------------------------------------- 1 | 2 | const http = require('http'), 3 | // disabling eslint import because `electron` is a buil-in module 4 | // eslint-disable-next-line import/no-unresolved 5 | { app } = require('electron'); 6 | 7 | const port = process.env.ELECTRON_WORKER_PORT, 8 | JOB_DURATION_MS = parseInt(process.env.JOB_DURATION_MS, 10); 9 | 10 | app.on('ready', () => { 11 | const server = http.createServer((req, res) => { 12 | let started = Date.now(); 13 | 14 | res.writeHead(200, { 'Content-Type': 'application/json' }); 15 | 16 | // simulate 500ms duration 17 | setTimeout(() => { 18 | res.end(JSON.stringify({ 19 | started, 20 | ended: Date.now() 21 | })); 22 | }, JOB_DURATION_MS); 23 | }); 24 | 25 | server.listen(port); 26 | }); 27 | -------------------------------------------------------------------------------- /test/electron-script/server/custom-args.js: -------------------------------------------------------------------------------- 1 | 2 | const http = require('http'), 3 | // disabling eslint import because `electron` is a buil-in module 4 | // eslint-disable-next-line import/no-unresolved 5 | { app } = require('electron'); 6 | 7 | const port = process.env.ELECTRON_WORKER_PORT, 8 | host = process.env.ELECTRON_WORKER_HOST; 9 | 10 | app.on('ready', () => { 11 | const server = http.createServer((req, res) => { 12 | res.writeHead(200, { 'Content-Type': 'application/json' }); 13 | res.end(JSON.stringify(process.argv.slice(2))); 14 | }); 15 | 16 | server.listen(port, host); 17 | }); 18 | -------------------------------------------------------------------------------- /test/electron-script/server/env.js: -------------------------------------------------------------------------------- 1 | 2 | const http = require('http'), 3 | // disabling eslint import because `electron` is a buil-in module 4 | // eslint-disable-next-line import/no-unresolved 5 | { app } = require('electron'); 6 | 7 | const port = process.env.ELECTRON_WORKER_PORT, 8 | host = process.env.ELECTRON_WORKER_HOST, 9 | foo = process.env.FOO, 10 | customEnv = process.env.CUSTOM_ENV; 11 | 12 | app.on('ready', () => { 13 | const server = http.createServer((req, res) => { 14 | res.writeHead(200, { 'Content-Type': 'application/json' }); 15 | res.end(JSON.stringify({ FOO: foo, CUSTOM_ENV: customEnv })); 16 | }); 17 | 18 | server.listen(port, host); 19 | }); 20 | -------------------------------------------------------------------------------- /test/electron-script/server/just-port.js: -------------------------------------------------------------------------------- 1 | 2 | const http = require('http'), 3 | // disabling eslint import because `electron` is a buil-in module 4 | // eslint-disable-next-line import/no-unresolved 5 | { app } = require('electron'); 6 | 7 | const port = process.env.ELECTRON_WORKER_PORT; 8 | 9 | app.on('ready', () => { 10 | const server = http.createServer((req, res) => { 11 | res.writeHead(200, { 'Content-Type': 'application/json' }); 12 | req.pipe(res); 13 | }); 14 | 15 | server.listen(port); 16 | }); 17 | -------------------------------------------------------------------------------- /test/electron-script/server/recycle-on-exit.js: -------------------------------------------------------------------------------- 1 | 2 | const http = require('http'), 3 | // disabling eslint import because `electron` is a buil-in module 4 | // eslint-disable-next-line import/no-unresolved 5 | { app } = require('electron'); 6 | 7 | const port = process.env.ELECTRON_WORKER_PORT, 8 | host = process.env.ELECTRON_WORKER_HOST; 9 | 10 | app.on('ready', () => { 11 | const server = http.createServer((req, res) => { 12 | res.writeHead(200, { 'Content-Type': 'application/json' }); 13 | res.end(JSON.stringify({ ok: true })); 14 | app.quit(); 15 | }); 16 | 17 | server.listen(port, host); 18 | }); 19 | -------------------------------------------------------------------------------- /test/electron-script/server/script.js: -------------------------------------------------------------------------------- 1 | 2 | const http = require('http'), 3 | // disabling eslint import because `electron` is a buil-in module 4 | // eslint-disable-next-line import/no-unresolved 5 | { app } = require('electron'); 6 | 7 | const port = process.env.ELECTRON_WORKER_PORT, 8 | host = process.env.ELECTRON_WORKER_HOST; 9 | 10 | app.on('ready', () => { 11 | const server = http.createServer((req, res) => { 12 | res.writeHead(200, { 'Content-Type': 'application/json' }); 13 | req.pipe(res); 14 | }); 15 | 16 | server.listen(port, host); 17 | }); 18 | -------------------------------------------------------------------------------- /test/electron-script/server/slowstart.js: -------------------------------------------------------------------------------- 1 | 2 | const http = require('http'), 3 | // disabling eslint import because `electron` is a buil-in module 4 | // eslint-disable-next-line import/no-unresolved 5 | { app } = require('electron'); 6 | 7 | const port = process.env.ELECTRON_WORKER_PORT, 8 | host = process.env.ELECTRON_WORKER_HOST; 9 | 10 | app.on('ready', () => { 11 | setTimeout(() => { 12 | const server = http.createServer((req, res) => { 13 | res.writeHead(200, { 'Content-Type': 'application/json' }); 14 | res.end(JSON.stringify({ ok: true })); 15 | }); 16 | 17 | server.listen(port, host); 18 | }, 2000); 19 | }); 20 | -------------------------------------------------------------------------------- /test/electron-script/server/stdio.js: -------------------------------------------------------------------------------- 1 | 2 | const http = require('http'), 3 | // disabling eslint import because `electron` is a buil-in module 4 | // eslint-disable-next-line import/no-unresolved 5 | { app } = require('electron'); 6 | 7 | const port = process.env.ELECTRON_WORKER_PORT, 8 | host = process.env.ELECTRON_WORKER_HOST; 9 | 10 | app.on('ready', () => { 11 | const server = http.createServer((req, res) => { 12 | process.send('ping'); 13 | 14 | res.writeHead(200, { 'Content-Type': 'application/json' }); 15 | res.end(JSON.stringify({})); 16 | }); 17 | 18 | server.listen(port, host); 19 | }); 20 | -------------------------------------------------------------------------------- /test/electron-script/server/timeout.js: -------------------------------------------------------------------------------- 1 | 2 | const http = require('http'), 3 | // disabling eslint import because `electron` is a buil-in module 4 | // eslint-disable-next-line import/no-unresolved 5 | { app } = require('electron'); 6 | 7 | const port = process.env.ELECTRON_WORKER_PORT, 8 | host = process.env.ELECTRON_WORKER_HOST; 9 | 10 | app.on('ready', () => { 11 | setTimeout(() => { 12 | const server = http.createServer((req, res) => res); 13 | 14 | server.listen(port, host); 15 | }, 2000); 16 | }); 17 | -------------------------------------------------------------------------------- /test/electron-script/server/workerId.js: -------------------------------------------------------------------------------- 1 | 2 | const http = require('http'), 3 | // disabling eslint import because `electron` is a buil-in module 4 | // eslint-disable-next-line import/no-unresolved 5 | { app } = require('electron'); 6 | 7 | const port = process.env.ELECTRON_WORKER_PORT, 8 | host = process.env.ELECTRON_WORKER_HOST, 9 | workerId = process.env.ELECTRON_WORKER_ID; 10 | 11 | app.on('ready', () => { 12 | const server = http.createServer((req, res) => { 13 | res.writeHead(200, { 'Content-Type': 'application/json' }); 14 | res.end(JSON.stringify(workerId)); 15 | }); 16 | 17 | server.listen(port, host); 18 | }); 19 | -------------------------------------------------------------------------------- /test/mocha.opts: -------------------------------------------------------------------------------- 1 | --compilers js:babel/register 2 | -------------------------------------------------------------------------------- /test/test.js: -------------------------------------------------------------------------------- 1 | 2 | import path from 'path'; 3 | import os from 'os'; 4 | import should from 'should'; 5 | import createManager from '../src/index'; 6 | 7 | /* eslint-disable padded-blocks */ 8 | /* eslint-disable prefer-arrow-callback */ 9 | describe('electron-workers', () => { 10 | 11 | describe('server mode', () => { 12 | common('server'); 13 | 14 | it('should initialize workers correctly with port boundary', function(done) { 15 | let isDone = false; 16 | 17 | let electronManager = createManager({ 18 | pathToScript: path.join(__dirname, 'electron-script/server', 'script.js'), 19 | numberOfWorkers: 2, 20 | portLeftBoundary: 10000, 21 | portRightBoundary: 15000 22 | }); 23 | 24 | electronManager.on('workerRecycling', function() { 25 | if (isDone) { 26 | return; 27 | } 28 | 29 | isDone = true; 30 | done(new Error('worker was recycled when trying to use port boundary')); 31 | }); 32 | 33 | electronManager.start((startErr) => { 34 | if (isDone) { 35 | return; 36 | } 37 | 38 | if (startErr) { 39 | isDone = true; 40 | return done(startErr); 41 | } 42 | 43 | should(electronManager._electronInstances.length).be.eql(2); 44 | should(electronManager._electronInstances[0].port).not.be.eql(electronManager._electronInstances[1].port); 45 | electronManager.kill(); 46 | done(); 47 | }); 48 | }); 49 | 50 | it('should be able to start electron in a port range', function(done) { 51 | let electronManager = createManager({ 52 | pathToScript: path.join(__dirname, 'electron-script/server', 'script.js'), 53 | numberOfWorkers: 1, 54 | portLeftBoundary: 10000, 55 | portRightBoundary: 11000 56 | }); 57 | 58 | electronManager.start((startErr) => { 59 | if (startErr) { 60 | return done(startErr); 61 | } 62 | 63 | should(electronManager._electronInstances[0].port).be.within(10000, 11000); 64 | electronManager.kill(); 65 | done(); 66 | }); 67 | }); 68 | 69 | it('should be able to communicate with just-port script', function(done) { 70 | let electronManager = createManager({ 71 | pathToScript: path.join(__dirname, 'electron-script/server', 'just-port.js'), 72 | numberOfWorkers: 1 73 | }); 74 | 75 | electronManager.start((startErr) => { 76 | if (startErr) { 77 | return done(startErr); 78 | } 79 | 80 | electronManager.execute({ foo: 'test' }, (executeErr, data) => { 81 | if (executeErr) { 82 | return done(executeErr); 83 | } 84 | 85 | should(data).be.eql({ foo: 'test' }); 86 | electronManager.kill(); 87 | done(); 88 | }); 89 | }); 90 | }); 91 | }); 92 | 93 | describe('ipc mode', () => { 94 | common('ipc'); 95 | 96 | it('should cb with error when not initialized with a ipc connection', function(done) { 97 | let electronManager = createManager({ 98 | connectionMode: 'ipc', 99 | pathToScript: path.join(__dirname, 'electron-script/ipc', 'script.js'), 100 | numberOfWorkers: 1, 101 | stdio: 'inherit' 102 | }); 103 | 104 | electronManager.start((startErr) => { 105 | if (startErr) { 106 | electronManager.kill(); 107 | return done(); 108 | } 109 | 110 | electronManager.kill(); 111 | done(new Error('should not start')); 112 | }); 113 | }); 114 | }); 115 | 116 | function common(mode) { 117 | it('should be able to communicate with electron', function(done) { 118 | let electronManager = createManager({ 119 | connectionMode: mode, 120 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'script.js'), 121 | numberOfWorkers: 1 122 | }); 123 | 124 | electronManager.start((startErr) => { 125 | if (startErr) { 126 | return done(startErr); 127 | } 128 | 129 | electronManager.execute({ foo: 'test' }, (executeErr, data) => { 130 | if (executeErr) { 131 | return done(executeErr); 132 | } 133 | 134 | should(data).be.eql({ foo: 'test' }); 135 | electronManager.kill(); 136 | done(); 137 | }); 138 | }); 139 | }); 140 | 141 | it('should be able to communicate with slowly starting electron', function(done) { 142 | this.timeout(5000); 143 | 144 | let electronManager = createManager({ 145 | connectionMode: mode, 146 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'slowstart.js'), 147 | numberOfWorkers: 1 148 | }); 149 | 150 | electronManager.start((startErr) => { 151 | if (startErr) { 152 | return done(startErr); 153 | } 154 | 155 | electronManager.execute({ foo: 'test' }, (executeErr, data) => { 156 | if (executeErr) { 157 | return done(executeErr); 158 | } 159 | 160 | should(data).be.eql({ ok: true }); 161 | electronManager.kill(); 162 | done(); 163 | }); 164 | }); 165 | }); 166 | 167 | it('should pass worker id as an env var', function(done) { 168 | let electronManager = createManager({ 169 | connectionMode: mode, 170 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'workerId.js'), 171 | numberOfWorkers: 2 172 | }); 173 | 174 | electronManager.start((startErr) => { 175 | let workersResponseId = [], 176 | isDone = false, 177 | executeCount = 0; 178 | 179 | if (startErr) { 180 | return done(startErr); 181 | } 182 | 183 | function executeTask() { 184 | electronManager.execute({}, (executeErr, workerIdResp) => { 185 | if (isDone) { 186 | return; 187 | } 188 | 189 | if (executeErr) { 190 | isDone = true; 191 | done(executeErr); 192 | return; 193 | } 194 | 195 | workersResponseId.push(workerIdResp); 196 | 197 | executeCount++; 198 | 199 | if (executeCount === electronManager._electronInstances.length) { 200 | let workerIds, 201 | workersNotFound; 202 | 203 | workerIds = electronManager._electronInstances.map((worker) => worker.id); 204 | 205 | workersNotFound = workerIds.filter((workerId) => workersResponseId.indexOf(workerId) === -1); 206 | 207 | should(workersNotFound.length).be.eql(0); 208 | electronManager.kill(); 209 | done(); 210 | } 211 | }); 212 | } 213 | 214 | for (let ix = 0; ix < electronManager._electronInstances.length; ix++) { 215 | executeTask(); 216 | } 217 | }); 218 | }); 219 | 220 | it('should pass env vars', function(done) { 221 | let electronManager = createManager({ 222 | connectionMode: mode, 223 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'env.js'), 224 | numberOfWorkers: 1, 225 | env: { 226 | FOO: 'FOO', 227 | CUSTOM_ENV: 'FOO' 228 | } 229 | }); 230 | 231 | electronManager.start((startErr) => { 232 | if (startErr) { 233 | return done(startErr); 234 | } 235 | 236 | electronManager.execute({}, (executeErr, data) => { 237 | if (executeErr) { 238 | return done(executeErr); 239 | } 240 | 241 | should(data).be.eql({ FOO: 'FOO', CUSTOM_ENV: 'FOO' }); 242 | electronManager.kill(); 243 | done(); 244 | }); 245 | }); 246 | }); 247 | 248 | it('should pass custom kill signal for worker process', function(done) { 249 | let electronManager = createManager({ 250 | connectionMode: mode, 251 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'script.js'), 252 | killSignal: 'SIGKILL' 253 | }); 254 | 255 | electronManager.start((startErr) => { 256 | if (startErr) { 257 | return done(startErr); 258 | } 259 | 260 | should(electronManager._electronInstances[0].options.killSignal).be.eql('SIGKILL'); 261 | electronManager.kill(); 262 | done(); 263 | }); 264 | }); 265 | 266 | it('should pass custom stdio to worker child process', function(done) { 267 | let customStdio = [null, null, null, 'ipc']; 268 | let isDone = false; 269 | 270 | let electronManager = createManager({ 271 | connectionMode: mode, 272 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'stdio.js'), 273 | numberOfWorkers: 1, 274 | stdio: customStdio 275 | }); 276 | 277 | electronManager.start((startErr) => { 278 | if (startErr) { 279 | return done(startErr); 280 | } 281 | 282 | electronManager._electronInstances[0].options.stdio.forEach(function(value, index) { 283 | should(value).be.eql(customStdio[index]); 284 | }); 285 | 286 | if (mode === 'ipc') { 287 | electronManager.kill(); 288 | return done(); 289 | } 290 | 291 | if (mode === 'server') { 292 | electronManager._electronInstances[0]._childProcess.on('message', function(msg) { 293 | should(msg).be.eql('ping'); 294 | if (!isDone) { 295 | isDone = true; 296 | electronManager.kill(); 297 | done(); 298 | } 299 | }); 300 | } 301 | 302 | electronManager.execute({}, (executeErr) => { 303 | if (executeErr && !isDone) { 304 | isDone = true; 305 | return done(executeErr); 306 | } 307 | }); 308 | }); 309 | }); 310 | 311 | it('should pass custom arguments to the electron executable', function(done) { 312 | let electronManager = createManager({ 313 | connectionMode: mode, 314 | electronArgs: ['--some-value=2', '--enable-some-behaviour'], 315 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'custom-args.js'), 316 | numberOfWorkers: 1 317 | }); 318 | 319 | electronManager.start((startErr) => { 320 | if (startErr) { 321 | return done(startErr); 322 | } 323 | 324 | electronManager.execute({}, (executeErr, data) => { 325 | if (executeErr) { 326 | return done(executeErr); 327 | } 328 | 329 | should(data.join(' ')).be.eql('--some-value=2 --enable-some-behaviour'); 330 | electronManager.kill(); 331 | done(); 332 | }); 333 | }); 334 | }); 335 | 336 | it('should spin up number of workers equal to number of cores by default', function(done) { 337 | let electronManager = createManager({ 338 | connectionMode: mode, 339 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'script.js') 340 | }); 341 | 342 | electronManager.start((startErr) => { 343 | if (startErr) { 344 | return done(startErr); 345 | } 346 | 347 | should(electronManager._electronInstances.length).be.eql(os.cpus().length); 348 | electronManager.kill(); 349 | done(); 350 | }); 351 | }); 352 | 353 | it('should spin up specified number of workers', function(done) { 354 | let electronManager = createManager({ 355 | connectionMode: mode, 356 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'script.js'), 357 | numberOfWorkers: 3 358 | }); 359 | 360 | electronManager.start((startErr) => { 361 | if (startErr) { 362 | return done(startErr); 363 | } 364 | 365 | should(electronManager._electronInstances.length).be.eql(3); 366 | electronManager.kill(); 367 | done(); 368 | }); 369 | }); 370 | 371 | it('worker process creation should emit event', function(done) { 372 | let numberOfProcess = 0; 373 | let isDone = false; 374 | 375 | let electronManager = createManager({ 376 | connectionMode: mode, 377 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'script.js'), 378 | numberOfWorkers: 2 379 | }); 380 | 381 | electronManager.on('workerProcessCreated', function(worker) { 382 | numberOfProcess++; 383 | 384 | let matchWorker = electronManager._electronInstances.filter( 385 | (electronInstance) => electronInstance.id === worker.id 386 | ); 387 | 388 | should(matchWorker.length).be.eql(1); 389 | 390 | if (numberOfProcess === 2) { 391 | if (!isDone) { 392 | isDone = true; 393 | electronManager.kill(); 394 | done(); 395 | } 396 | } 397 | }); 398 | 399 | electronManager.start(() => { 400 | }); 401 | }); 402 | 403 | it('worker process recycle should emit event', function(done) { 404 | let isDone = false, 405 | recyclingWasCalled = false; 406 | 407 | let electronManager = createManager({ 408 | connectionMode: mode, 409 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'script.js'), 410 | numberOfWorkers: 1 411 | }); 412 | 413 | electronManager.on('workerRecycling', () => { 414 | recyclingWasCalled = true; 415 | }); 416 | 417 | electronManager.on('workerRecycled', (worker) => { 418 | if (isDone) { 419 | return; 420 | } 421 | 422 | isDone = true; 423 | should(recyclingWasCalled).be.eql(true); 424 | should(worker.id).be.eql(electronManager._electronInstances[0].id); 425 | electronManager.kill(); 426 | done(); 427 | }); 428 | 429 | electronManager.start((startErr) => { 430 | if (startErr) { 431 | return done(startErr); 432 | } 433 | 434 | electronManager._electronInstances[0].recycle(function(recycleErr) { 435 | if (isDone) { 436 | return; 437 | } 438 | 439 | if (recycleErr) { 440 | isDone = true; 441 | return done(recycleErr); 442 | } 443 | }); 444 | }); 445 | }); 446 | 447 | it('should recycle on worker\'s process exit', function(done) { 448 | let responseData; 449 | 450 | let electronManager = createManager({ 451 | connectionMode: mode, 452 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'recycle-on-exit.js'), 453 | numberOfWorkers: 1 454 | }); 455 | 456 | electronManager.on('workerRecycled', () => { 457 | should(responseData).be.eql({ ok: true }); 458 | electronManager.kill(); 459 | done(); 460 | }); 461 | 462 | electronManager.start((startErr) => { 463 | if (startErr) { 464 | return done(startErr); 465 | } 466 | 467 | electronManager.execute({}, function(executeErr, data) { 468 | responseData = data; 469 | if (executeErr) { 470 | return done(executeErr); 471 | } 472 | }); 473 | }); 474 | }); 475 | 476 | it('should initialize free workers', function(done) { 477 | let electronManager = createManager({ 478 | connectionMode: mode, 479 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'script.js'), 480 | numberOfWorkers: 2 481 | }); 482 | 483 | electronManager.start((startErr) => { 484 | let busyWorkers; 485 | 486 | if (startErr) { 487 | return done(startErr); 488 | } 489 | 490 | busyWorkers = electronManager._electronInstances.filter((worker) => worker.isBusy === true); 491 | 492 | should(busyWorkers.length).be.eql(0); 493 | electronManager.kill(); 494 | done(); 495 | }); 496 | }); 497 | 498 | it('should initialize with no workers in recycling', function(done) { 499 | let electronManager = createManager({ 500 | connectionMode: mode, 501 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'script.js'), 502 | numberOfWorkers: 2 503 | }); 504 | 505 | electronManager.start((startErr) => { 506 | let recyclingWorkers; 507 | 508 | if (startErr) { 509 | return done(startErr); 510 | } 511 | 512 | recyclingWorkers = electronManager._electronInstances.filter((worker) => worker.isRecycling === true); 513 | 514 | should(recyclingWorkers.length).be.eql(0); 515 | electronManager.kill(); 516 | done(); 517 | }); 518 | }); 519 | 520 | it('should distribute tasks across all workers', function(done) { 521 | let electronManager = createManager({ 522 | connectionMode: mode, 523 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'script.js'), 524 | numberOfWorkers: 4 525 | }); 526 | 527 | electronManager.start((startErr) => { 528 | let workersCalled = [], 529 | isDone = false, 530 | executeCount = 0; 531 | 532 | if (startErr) { 533 | return done(startErr); 534 | } 535 | 536 | electronManager._electronInstances.forEach((worker) => { 537 | worker.once('task', function() { 538 | workersCalled.push(worker.id); 539 | }); 540 | }); 541 | 542 | function executeTask() { 543 | electronManager.execute({}, (executeErr) => { 544 | if (isDone) { 545 | return; 546 | } 547 | 548 | if (executeErr) { 549 | isDone = true; 550 | done(executeErr); 551 | return; 552 | } 553 | 554 | executeCount++; 555 | 556 | if (executeCount === electronManager._electronInstances.length) { 557 | let workerIds, 558 | workersNotCalled; 559 | 560 | workerIds = electronManager._electronInstances.map((worker) => worker.id); 561 | 562 | workersNotCalled = workerIds.filter((workerId) => workersCalled.indexOf(workerId) === -1); 563 | 564 | should(workersNotCalled.length).be.eql(0); 565 | electronManager.kill(); 566 | done(); 567 | } 568 | }); 569 | } 570 | 571 | for (let ix = 0; ix < electronManager._electronInstances.length; ix++) { 572 | executeTask(); 573 | } 574 | }); 575 | }); 576 | 577 | it('should respect maxConcurrencyPerWorker option', function(done) { 578 | /* eslint-disable no-console */ 579 | const totalWorkers = 2, 580 | totalJobs = 6, 581 | jobDuration = 500; 582 | 583 | let electronManager = createManager({ 584 | connectionMode: mode, 585 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'concurrency.js'), 586 | numberOfWorkers: totalWorkers, 587 | maxConcurrencyPerWorker: 1, 588 | env: { 589 | JOB_DURATION_MS: jobDuration 590 | } 591 | }); 592 | 593 | electronManager.start((startErr) => { 594 | let workersCalled = [], 595 | isDone = false, 596 | executeCount = 0, 597 | jobsStarted; 598 | 599 | if (startErr) { 600 | return done(startErr); 601 | } 602 | 603 | jobsStarted = Date.now(); 604 | 605 | electronManager._electronInstances.forEach((worker) => { 606 | let taskDurationMsInWorker = 0, 607 | taskStartedInWorker, 608 | taskCountInWorker = 0; 609 | 610 | worker.once('task', () => { 611 | workersCalled.push(worker.id); 612 | }); 613 | 614 | worker.on('task', () => { 615 | console.log(`worker ${worker.id} has received new task..`); 616 | taskStartedInWorker = Date.now(); 617 | taskCountInWorker++; 618 | }); 619 | 620 | worker.on('taskEnd', () => { 621 | console.log(`worker ${worker.id} task has ended..`); 622 | taskDurationMsInWorker = Date.now() - taskStartedInWorker; 623 | 624 | // only one task per worker should be processed concurrently 625 | should(taskCountInWorker).be.eql(1); 626 | should(taskDurationMsInWorker).be.aboveOrEqual(jobDuration); 627 | 628 | taskCountInWorker = 0; 629 | }); 630 | }); 631 | 632 | function executeTask() { 633 | electronManager.execute({}, (executeErr, data) => { 634 | if (isDone) { 635 | return; 636 | } 637 | 638 | if (executeErr) { 639 | isDone = true; 640 | done(executeErr); 641 | return; 642 | } 643 | 644 | console.log( 645 | `started: ${data.started} | ended: ${Date.now()} | duration: ${((data.ended - data.started) / 1000)} secs` 646 | ); 647 | 648 | executeCount++; 649 | 650 | if (executeCount === totalJobs) { 651 | let workerIds, 652 | workersNotCalled; 653 | 654 | workerIds = electronManager._electronInstances.map((worker) => worker.id); 655 | 656 | workersNotCalled = workerIds.filter((workerId) => workersCalled.indexOf(workerId) === -1); 657 | 658 | console.log('-----'); 659 | console.log(`ESTIMATED DURATION: ${((totalJobs / totalWorkers) * (jobDuration / 1000))} secs`); 660 | console.log(`ACTUAL DURATION: ${((Date.now() - jobsStarted) / 1000)} secs`); 661 | 662 | should(workersNotCalled.length).be.eql(0); 663 | 664 | electronManager.kill(); 665 | done(); 666 | } 667 | }); 668 | } 669 | 670 | for (let i = totalJobs - 1; i >= 0; i--) { 671 | executeTask(); 672 | } 673 | }); 674 | /* eslint-enable no-console */ 675 | }); 676 | 677 | it('should be able to send just a simple string on input', function(done) { 678 | let electronManager = createManager({ 679 | connectionMode: mode, 680 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'script.js'), 681 | numberOfWorkers: 1 682 | }); 683 | 684 | electronManager.start((startErr) => { 685 | if (startErr) { 686 | return done(startErr); 687 | } 688 | 689 | electronManager.execute('test', (executeErr, data) => { 690 | if (executeErr) { 691 | return done(executeErr); 692 | } 693 | 694 | should(data).be.eql('test'); 695 | electronManager.kill(); 696 | done(); 697 | }); 698 | }); 699 | }); 700 | 701 | it('simple input string should not be stringified what is causing broken line endings', function(done) { 702 | let electronManager = createManager({ 703 | connectionMode: mode, 704 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'script.js'), 705 | numberOfWorkers: 1 706 | }); 707 | 708 | electronManager.start((startErr) => { 709 | if (startErr) { 710 | return done(startErr); 711 | } 712 | 713 | electronManager.execute('', (executeErr, data) => { 714 | if (executeErr) { 715 | return done(executeErr); 716 | } 717 | 718 | should(data).be.eql(''); 719 | electronManager.kill(); 720 | done(); 721 | }); 722 | }); 723 | }); 724 | 725 | it('timeout should emit event', function(done) { 726 | this.timeout(10000); 727 | 728 | let electronManager = createManager({ 729 | connectionMode: mode, 730 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'timeout.js'), 731 | numberOfWorkers: 1, 732 | timeout: 10 733 | }); 734 | 735 | let emitted = false, 736 | timeoutId; 737 | 738 | electronManager.on('workerTimeout', () => { 739 | emitted = true; 740 | clearTimeout(timeoutId); 741 | electronManager.kill(); 742 | done(); 743 | }); 744 | 745 | electronManager.start((startErr) => { 746 | if (startErr) { 747 | return done(startErr); 748 | } 749 | 750 | electronManager.execute({}, (executeErr) => { 751 | if (!executeErr) { 752 | return done(new Error('should not execute successfully')); 753 | } 754 | }); 755 | }); 756 | 757 | timeoutId = setTimeout(() => { 758 | electronManager.kill(); 759 | 760 | if (!emitted) { 761 | done(new Error('worker timeout was not emitted')); 762 | } 763 | }, 10000); 764 | }); 765 | 766 | it('timeout should cb with error', function(done) { 767 | this.timeout(10000); 768 | 769 | let electronManager = createManager({ 770 | connectionMode: mode, 771 | pathToScript: path.join(__dirname, `electron-script/${mode}`, 'timeout.js'), 772 | numberOfWorkers: 1, 773 | timeout: 100 774 | }); 775 | 776 | electronManager.start((startErr) => { 777 | if (startErr) { 778 | return done(startErr); 779 | } 780 | 781 | electronManager.execute({}, (executeErr) => { 782 | if (!executeErr) { 783 | return done(new Error('should not execute successfully')); 784 | } 785 | 786 | electronManager.kill(); 787 | should(executeErr.workerTimeout).be.eql(true); 788 | done(); 789 | }); 790 | }); 791 | }); 792 | } 793 | }); 794 | --------------------------------------------------------------------------------