3 |
2 |
3 |
Offload modules to Worker threads seamlessly using Comlink.
6 | 7 | 8 | ### Features 9 | 10 | - Offload almost any module into a Worker with little or no usage change 11 | - Supports arbitrary classes, objects & functions (`await new Foo()`) 12 | - Works beautifully with async/await 13 | - Built-in code-splitting: workers are lazy-loaded 14 | 15 | 16 | ## Installation 17 | 18 | ```sh 19 | npm install -D comlink-loader 20 | ``` 21 | 22 | 23 | ## Usage 24 | 25 | The goal of `comlink-loader` is to make the fact that a module is running inside a Worker nearly transparent to the developer. 26 | 27 | ### Factory Mode (default) 28 | 29 | In the example below, there are two changes we must make in order to import `MyClass` within a Worker via `comlink-loader`. 30 | 31 | 1. instantiation and method calls must be prefixed with `await`, since everything is inherently asynchronous. 32 | 2. the value we import from `comlink-loader!./my-class` is now a function that returns our module exports. 33 | > Calling this function creates a new instance of the Worker. 34 | 35 | **my-class.js**: _(gets moved into a worker)_ 36 | 37 | ```js 38 | // Dependencies get bundled into the worker: 39 | import rnd from 'random-int'; 40 | 41 | // Export as you would in a normal module: 42 | export function meaningOfLife() { 43 | return 42; 44 | } 45 | 46 | export class MyClass { 47 | constructor(value = rnd()) { 48 | this.value = value; 49 | } 50 | increment() { 51 | this.value++; 52 | } 53 | // Tip: async functions make the interface identical 54 | async getValue() { 55 | return this.value; 56 | } 57 | } 58 | ``` 59 | 60 | **main.js**: _(our demo, on the main thread)_ 61 | 62 | ```js 63 | import MyWorker from 'comlink-loader!./my-class'; 64 | 65 | // instantiate a new Worker with our code in it: 66 | const inst = new MyWorker(); 67 | 68 | // our module exports are exposed on the instance: 69 | await inst.meaningOfLife(); // 42 70 | 71 | // instantiate a class in the worker (does not create a new worker). 72 | // notice the `await` here: 73 | const obj = await new inst.MyClass(42); 74 | 75 | await obj.increment(); 76 | 77 | await obj.getValue(); // 43 78 | ``` 79 | 80 | ### Singleton Mode 81 | 82 | Comlink-loader also includes a `singleton` mode, which can be opted in on a per-module basis using Webpack's inline loader syntax, or globally in Webpack configuration. Singleton mode is designed to be the easiest possible way to use a Web Worker, but in doing so it only allows using a single Worker instance for each module. 83 | 84 | The benefit is that your module's exports can be used just like any other import, without the need for a constructor. It also supports TypeScript automatically, since the module being imported looks just like it would were it running on the main thread. The only change that is required in order to move a module into a Worker using singleton mode is to ensure all of your function calls use `await`. 85 | 86 | First, configure `comlink-loader` globally to apply to all `*.worker.js` files (or whichever pattern you choose). Here we're going to use TypeScript, just to show that it works out-of-the-box: 87 | 88 | **webpack.config.js**: 89 | 90 | ```js 91 | module.exports = { 92 | module: { 93 | rules: [ 94 | { 95 | test: /\.worker\.(js|ts)$/i, 96 | use: [{ 97 | loader: 'comlink-loader', 98 | options: { 99 | singleton: true 100 | } 101 | }] 102 | } 103 | ] 104 | } 105 | } 106 | ``` 107 | 108 | Now, let's write a simple module that we're going to load in a Worker: 109 | 110 | **greetings.worker.ts**: 111 | 112 | ```ts 113 | export async function greet(subject: string): string { 114 | return `Hello, ${subject}!`; 115 | } 116 | ``` 117 | 118 | We can import our the above module, and since the filename includes `.worker.ts`, it will be transparently loaded in a Web Worker! 119 | 120 | **index.ts**: 121 | 122 | ```ts 123 | import { greet } from './greetings.worker.ts'; 124 | 125 | async function demo() { 126 | console.log(await greet('dog')); 127 | } 128 | 129 | demo(); 130 | ``` 131 | 132 | 133 | ## License 134 | 135 | Apache-2.0 136 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "comlink-loader", 3 | "version": "2.0.0", 4 | "description": "Webpack loader: offload modules to Worker threads seamlessly using Comlink", 5 | "main": "dist/comlink-loader.js", 6 | "repository": "GoogleChromeLabs/comlink-loader", 7 | "scripts": { 8 | "build": "microbundle --inline none --format cjs --no-compress src/*.js", 9 | "prepublishOnly": "npm run build", 10 | "dev": "karmatic watch --no-headless", 11 | "test": "npm run build && karmatic --no-coverage", 12 | "release": "npm t && git commit -am $npm_package_version && git tag $npm_package_version && git push && git push --tags && npm publish" 13 | }, 14 | "eslintConfig": { 15 | "extends": "developit", 16 | "rules": { 17 | "import/no-webpack-loader-syntax": false, 18 | "indent": [ 19 | "error", 20 | 2 21 | ] 22 | } 23 | }, 24 | "files": [ 25 | "src", 26 | "dist" 27 | ], 28 | "keywords": [ 29 | "webpack", 30 | "loader", 31 | "worker", 32 | "web worker", 33 | "thread", 34 | "comlink" 35 | ], 36 | "author": "The Chromium Authors", 37 | "contributors": [ 38 | { 39 | "name": "Jason Miller", 40 | "email": "developit@google.com" 41 | }, 42 | { 43 | "name": "Jasper Palfree", 44 | "email": "jasper@wellcaffeinated.net" 45 | } 46 | ], 47 | "license": "Apache-2.0", 48 | "devDependencies": { 49 | "eslint": "^4.16.0", 50 | "eslint-config-developit": "^1.1.1", 51 | "jasmine-sinon": "^0.4.0", 52 | "karmatic": "^1.4.0", 53 | "microbundle": "^0.11.0", 54 | "sinon": "^8.0.4", 55 | "webpack": "^4.41.2" 56 | }, 57 | "dependencies": { 58 | "comlink": "^4.2.0", 59 | "loader-utils": "^1.1.0", 60 | "slash": "^3.0.0", 61 | "worker-loader": "^2.0.0" 62 | } 63 | } 64 | -------------------------------------------------------------------------------- /src/comlink-worker-loader.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2018 Google LLC 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); you may not 5 | * use this file except in compliance with the License. You may obtain a copy of 6 | * the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT 12 | * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the 13 | * License for the specific language governing permissions and limitations under 14 | * the License. 15 | */ 16 | 17 | export default function rpcWorkerLoader (content) { 18 | return `import { expose } from 'comlink'; 19 | ${content}; 20 | expose( 21 | Object.keys(__webpack_exports__).reduce(function(r,k){ 22 | if (k=='__esModule') return r; 23 | r[k] = __webpack_exports__[k]; 24 | return r 25 | },{}) 26 | )`; 27 | } 28 | -------------------------------------------------------------------------------- /src/index.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2018 Google LLC 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); you may not 5 | * use this file except in compliance with the License. You may obtain a copy of 6 | * the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT 12 | * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the 13 | * License for the specific language governing permissions and limitations under 14 | * the License. 15 | */ 16 | 17 | import path from 'path'; 18 | import loaderUtils from 'loader-utils'; 19 | import slash from 'slash'; 20 | 21 | const comlinkLoaderSpecificOptions = [ 22 | 'multiple', 'multi', // @todo: remove these 23 | 'singleton' 24 | ]; 25 | 26 | export default function loader () { } 27 | 28 | loader.pitch = function (request) { 29 | const options = loaderUtils.getOptions(this) || {}; 30 | const singleton = options.singleton; 31 | const workerLoaderOptions = {}; 32 | for (let i in options) { 33 | if (comlinkLoaderSpecificOptions.indexOf(i) === -1) { 34 | workerLoaderOptions[i] = options[i]; 35 | } 36 | } 37 | 38 | const workerLoader = `!worker-loader?${JSON.stringify(workerLoaderOptions)}!${slash(path.resolve(__dirname, 'comlink-worker-loader.js'))}`; 39 | 40 | const remainingRequest = JSON.stringify(workerLoader + '!' + request); 41 | 42 | // ?singleton mode: export an instance of the worker 43 | if (singleton === true) { 44 | return ` 45 | module.exports = require('comlink').wrap(require(${remainingRequest})()); 46 | ${options.module === false ? '' : 'module.exports.__esModule = true;'} 47 | `.replace(/\n\s*/g, ''); 48 | } 49 | 50 | // ?singleton=false mode: always return a new worker from the factory 51 | if (singleton === false) { 52 | return ` 53 | module.exports = function () { 54 | return require('comlink').wrap(require(${remainingRequest})()); 55 | }; 56 | `.replace(/\n\s*/g, ''); 57 | } 58 | 59 | return ` 60 | var wrap = require('comlink').wrap, 61 | Worker = require(${remainingRequest}), 62 | inst; 63 | module.exports = function f() { 64 | if (this instanceof f) return wrap(Worker()); 65 | return inst || (inst = wrap(Worker())); 66 | }; 67 | `.replace(/\n\s*/g, ''); 68 | }; 69 | -------------------------------------------------------------------------------- /test/index.test.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2018 Google LLC 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); you may not 5 | * use this file except in compliance with the License. You may obtain a copy of 6 | * the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT 12 | * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the 13 | * License for the specific language governing permissions and limitations under 14 | * the License. 15 | */ 16 | 17 | import sinon from 'sinon'; 18 | import 'jasmine-sinon'; 19 | import MyWorker from 'comlink-loader!./worker'; 20 | 21 | const OriginalWorker = self.Worker; 22 | self.Worker = sinon.spy((url, opts) => new OriginalWorker(url, opts)); 23 | 24 | describe('worker', () => { 25 | let worker, inst; 26 | 27 | it('should be a factory', async () => { 28 | worker = new MyWorker(); 29 | expect(self.Worker).toHaveBeenCalledOnce(); 30 | self.Worker.resetHistory(); 31 | expect(self.Worker).not.toHaveBeenCalled(); 32 | }); 33 | 34 | it('should be instantiable', async () => { 35 | inst = await new (worker.MyClass)(); 36 | expect(self.Worker).not.toHaveBeenCalled(); 37 | }); 38 | 39 | it('inst.foo()', async () => { 40 | const result = inst.foo(); 41 | expect(result instanceof Promise).toBe(true); 42 | expect(await result).toBe(1); 43 | }); 44 | 45 | it('inst.bar("a", "b")', async () => { 46 | let out = await inst.bar('a', 'b'); 47 | expect(out).toEqual('a [bar:3] b'); 48 | }); 49 | 50 | it('should propagate worker exceptions', async () => { 51 | try { 52 | await inst.throwError(); 53 | } 54 | catch (e) { 55 | expect(e).toMatch(/Error/); 56 | } 57 | }); 58 | 59 | it('should re-use Worker instances when the factory is invoked without `new`', async () => { 60 | self.Worker.resetHistory(); 61 | 62 | const firstWorker = MyWorker(); 63 | const firstInst = await new (firstWorker.MyClass)(); 64 | expect(await firstInst.foo()).toBe(1); 65 | 66 | expect(self.Worker).toHaveBeenCalledOnce(); 67 | 68 | self.Worker.resetHistory(); 69 | 70 | const secondWorker = MyWorker(); 71 | const secondInst = await new (secondWorker.MyClass)(); 72 | expect(await secondInst.foo()).toBe(1); 73 | expect(secondInst).not.toBe(inst); 74 | 75 | expect(self.Worker).not.toHaveBeenCalled(); 76 | }); 77 | }); 78 | -------------------------------------------------------------------------------- /test/other.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2018 Google LLC 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); you may not 5 | * use this file except in compliance with the License. You may obtain a copy of 6 | * the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT 12 | * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the 13 | * License for the specific language governing permissions and limitations under 14 | * the License. 15 | */ 16 | 17 | export function otherFoo () {} 18 | 19 | export const otherBar = 3; 20 | -------------------------------------------------------------------------------- /test/singleton.test.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2018 Google LLC 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); you may not 5 | * use this file except in compliance with the License. You may obtain a copy of 6 | * the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT 12 | * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the 13 | * License for the specific language governing permissions and limitations under 14 | * the License. 15 | */ 16 | 17 | import sinon from 'sinon'; 18 | import 'jasmine-sinon'; 19 | 20 | const OriginalWorker = self.Worker; 21 | self.Worker = sinon.spy((url, opts) => new OriginalWorker(url, opts)); 22 | 23 | describe('singleton', () => { 24 | let exported; 25 | 26 | it('should immediately instantiate the worker', async () => { 27 | // we're using dynamic import here so the Worker spy can be installed before-hand 28 | exported = require('comlink-loader?singleton!./worker'); 29 | 30 | expect(self.Worker).toHaveBeenCalledOnce(); 31 | 32 | self.Worker.resetHistory(); 33 | }); 34 | 35 | it('should function and not re-instantiate the Worker', async () => { 36 | const inst = await new exported.MyClass(); 37 | expect(await inst.foo()).toBe(1); 38 | 39 | expect(await exported.hello()).toBe('world'); 40 | 41 | expect(self.Worker).not.toHaveBeenCalled(); 42 | }); 43 | 44 | it('should propagate worker exceptions', async () => { 45 | const inst = await new exported.MyClass(); 46 | try { 47 | await inst.throwError(); 48 | } 49 | catch (e) { 50 | expect(e).toMatch(/Error/); 51 | } 52 | }); 53 | }); 54 | -------------------------------------------------------------------------------- /test/worker.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Copyright 2018 Google LLC 3 | * 4 | * Licensed under the Apache License, Version 2.0 (the "License"); you may not 5 | * use this file except in compliance with the License. You may obtain a copy of 6 | * the License at 7 | * 8 | * http://www.apache.org/licenses/LICENSE-2.0 9 | * 10 | * Unless required by applicable law or agreed to in writing, software 11 | * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT 12 | * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the 13 | * License for the specific language governing permissions and limitations under 14 | * the License. 15 | */ 16 | 17 | import { otherBar } from './other'; 18 | 19 | export function hello() { 20 | return Promise.resolve('world'); 21 | } 22 | 23 | export class MyClass { 24 | constructor ({ value = 41 } = {}) { 25 | this.myValue = value; 26 | } 27 | foo () { 28 | return 1; 29 | } 30 | bar (a, b) { 31 | return `${a} [bar:${otherBar}] ${b}`; 32 | } 33 | baz () { 34 | return `myValue = ${++this.myValue}`; 35 | } 36 | } 37 | --------------------------------------------------------------------------------