├── .data ├── transform-async-es3.json ├── transform-async-es5.json ├── transform-parallel-es3.json ├── transform-parallel-es5.json ├── transform-async-es2015.json ├── transform-async-es2016.json ├── transform-async-es2017.json ├── transform-async-es2018.json ├── transform-async-es2019.json ├── transform-async-es2020.json ├── transform-parallel-es2016.json ├── transform-parallel-es2019.json ├── transform-parallel-es2015.json ├── transform-parallel-es2017.json ├── transform-parallel-es2018.json ├── transform-parallel-es2020.json ├── transform-sync-es3.json ├── transform-sync-es5.json ├── transform-sync-es2015.json ├── transform-sync-es2016.json ├── transform-sync-es2017.json ├── transform-sync-es2018.json ├── transform-sync-es2019.json └── transform-sync-es2020.json ├── images └── transform │ ├── async │ ├── es3.png │ ├── es5.png │ ├── es2015.png │ ├── es2016.png │ ├── es2017.png │ ├── es2018.png │ ├── es2019.png │ └── es2020.png │ ├── sync │ ├── es3.png │ ├── es5.png │ ├── es2015.png │ ├── es2016.png │ ├── es2017.png │ ├── es2018.png │ ├── es2019.png │ └── es2020.png │ └── parallel │ ├── es3.png │ ├── es5.png │ ├── es2015.png │ ├── es2016.png │ ├── es2017.png │ ├── es2018.png │ ├── es2019.png │ └── es2020.png ├── index.ts ├── compat-transform ├── index.ts ├── README.md └── common.ts ├── package.json ├── scripts └── vis.py ├── util.ts ├── .gitignore ├── tsconfig.json └── README.md /.data/transform-async-es3.json: -------------------------------------------------------------------------------- 1 | {"swc (es3)":494.66588271005776} -------------------------------------------------------------------------------- /.data/transform-async-es5.json: -------------------------------------------------------------------------------- 1 | {"swc (es5)":515.9168311137706} -------------------------------------------------------------------------------- /.data/transform-parallel-es3.json: -------------------------------------------------------------------------------- 1 | {"swc (es3)":1445.670854686975} -------------------------------------------------------------------------------- /.data/transform-parallel-es5.json: -------------------------------------------------------------------------------- 1 | {"swc (es5)":1558.7855947362473} -------------------------------------------------------------------------------- /.data/transform-async-es2015.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2015)":633.5434936444149,"swc (es2015)":509.62676746015165} -------------------------------------------------------------------------------- /.data/transform-async-es2016.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2016)":711.8967645771112,"swc (es2016)":704.4432221282367} -------------------------------------------------------------------------------- /.data/transform-async-es2017.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2017)":691.9096662351187,"swc (es2017)":688.5347637730079} -------------------------------------------------------------------------------- /.data/transform-async-es2018.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2018)":702.4742332425185,"swc (es2018)":693.3686469089365} -------------------------------------------------------------------------------- /.data/transform-async-es2019.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2019)":708.8951769110444,"swc (es2019)":698.0345238059931} -------------------------------------------------------------------------------- /.data/transform-async-es2020.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2020)":697.4507374912309,"swc (es2020)":762.3602346514622} -------------------------------------------------------------------------------- /.data/transform-parallel-es2016.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2016)":1492.252738854041,"swc (es2016)":2118.326508954607} -------------------------------------------------------------------------------- /.data/transform-parallel-es2019.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2019)":1498.6514229170143,"swc (es2019)":2134.00315378253} -------------------------------------------------------------------------------- /.data/transform-parallel-es2015.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2015)":1331.0175272342167,"swc (es2015)":1513.4473405748427} -------------------------------------------------------------------------------- /.data/transform-parallel-es2017.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2017)":1449.2330861443888,"swc (es2017)":1928.8985215259147} -------------------------------------------------------------------------------- /.data/transform-parallel-es2018.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2018)":1482.0753962428064,"swc (es2018)":2110.0668900836167} -------------------------------------------------------------------------------- /.data/transform-parallel-es2020.json: -------------------------------------------------------------------------------- 1 | {"esbuild (es2020)":1477.1094446333052,"swc (es2020)":2321.6834820193403} -------------------------------------------------------------------------------- /images/transform/async/es3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/async/es3.png -------------------------------------------------------------------------------- /images/transform/async/es5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/async/es5.png -------------------------------------------------------------------------------- /images/transform/sync/es3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/sync/es3.png -------------------------------------------------------------------------------- /images/transform/sync/es5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/sync/es5.png -------------------------------------------------------------------------------- /index.ts: -------------------------------------------------------------------------------- 1 | import { runTransforms } from "./compat-transform"; 2 | 3 | (async () => { 4 | await runTransforms(); 5 | })() -------------------------------------------------------------------------------- /images/transform/async/es2015.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/async/es2015.png -------------------------------------------------------------------------------- /images/transform/async/es2016.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/async/es2016.png -------------------------------------------------------------------------------- /images/transform/async/es2017.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/async/es2017.png -------------------------------------------------------------------------------- /images/transform/async/es2018.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/async/es2018.png -------------------------------------------------------------------------------- /images/transform/async/es2019.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/async/es2019.png -------------------------------------------------------------------------------- /images/transform/async/es2020.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/async/es2020.png -------------------------------------------------------------------------------- /images/transform/parallel/es3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/parallel/es3.png -------------------------------------------------------------------------------- /images/transform/parallel/es5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/parallel/es5.png -------------------------------------------------------------------------------- /images/transform/sync/es2015.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/sync/es2015.png -------------------------------------------------------------------------------- /images/transform/sync/es2016.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/sync/es2016.png -------------------------------------------------------------------------------- /images/transform/sync/es2017.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/sync/es2017.png -------------------------------------------------------------------------------- /images/transform/sync/es2018.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/sync/es2018.png -------------------------------------------------------------------------------- /images/transform/sync/es2019.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/sync/es2019.png -------------------------------------------------------------------------------- /images/transform/sync/es2020.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/sync/es2020.png -------------------------------------------------------------------------------- /images/transform/parallel/es2015.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/parallel/es2015.png -------------------------------------------------------------------------------- /images/transform/parallel/es2016.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/parallel/es2016.png -------------------------------------------------------------------------------- /images/transform/parallel/es2017.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/parallel/es2017.png -------------------------------------------------------------------------------- /images/transform/parallel/es2018.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/parallel/es2018.png -------------------------------------------------------------------------------- /images/transform/parallel/es2019.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/parallel/es2019.png -------------------------------------------------------------------------------- /images/transform/parallel/es2020.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kdy1/benchmark-done-right/HEAD/images/transform/parallel/es2020.png -------------------------------------------------------------------------------- /.data/transform-sync-es3.json: -------------------------------------------------------------------------------- 1 | {"babel (preset-env)":58.2345676752695,"typescript (es3)":46.431997998112045,"swc (es3)":587.060029112075} -------------------------------------------------------------------------------- /.data/transform-sync-es5.json: -------------------------------------------------------------------------------- 1 | {"babel (preset-env)":68.66735248954869,"typescript (es5)":56.49099761939024,"swc (es5)":629.7896989949928} -------------------------------------------------------------------------------- /.data/transform-sync-es2015.json: -------------------------------------------------------------------------------- 1 | {"babel (preset-env)":72.63479033235706,"typescript (es2015)":64.98926793364551,"esbuild (es2015)":38.584140698223116,"swc (es2015)":635.885699580509} -------------------------------------------------------------------------------- /.data/transform-sync-es2016.json: -------------------------------------------------------------------------------- 1 | {"babel (preset-env)":71.8705081628644,"typescript (es2016)":65.74443005557461,"esbuild (es2016)":33.951201104833544,"swc (es2016)":889.6390738927712} -------------------------------------------------------------------------------- /.data/transform-sync-es2017.json: -------------------------------------------------------------------------------- 1 | {"babel (preset-env)":73.04988865770007,"typescript (es2017)":67.36789783641555,"esbuild (es2017)":27.421520899215764,"swc (es2017)":900.8418420414845} -------------------------------------------------------------------------------- /.data/transform-sync-es2018.json: -------------------------------------------------------------------------------- 1 | {"babel (preset-env)":72.72433607450668,"typescript (es2018)":68.44866418846755,"esbuild (es2018)":23.516841790790306,"swc (es2018)":893.8527869164253} -------------------------------------------------------------------------------- /.data/transform-sync-es2019.json: -------------------------------------------------------------------------------- 1 | {"babel (preset-env)":72.36252634005747,"typescript (es2019)":67.1315037304669,"esbuild (es2019)":20.51797488285517,"swc (es2019)":909.3530069068167} -------------------------------------------------------------------------------- /.data/transform-sync-es2020.json: -------------------------------------------------------------------------------- 1 | {"babel (preset-env)":72.20525012156936,"typescript (es2020)":67.05501065174656,"esbuild (es2020)":18.15588634829545,"swc (es2020)":984.3266393751551} -------------------------------------------------------------------------------- /compat-transform/index.ts: -------------------------------------------------------------------------------- 1 | import { run } from "./common"; 2 | 3 | export async function runTransforms() { 4 | await run('es3') 5 | await run('es5') 6 | await run('es2015') 7 | await run('es2016') 8 | await run('es2017') 9 | await run('es2018') 10 | await run('es2019') 11 | await run('es2020') 12 | } -------------------------------------------------------------------------------- /compat-transform/README.md: -------------------------------------------------------------------------------- 1 | # Compatibilty transforms 2 | 3 | This benchmark is about compatibility transforms, which is used to make code run on old browsers. 4 | As some environments (es3, es5) are not supported by some tools, those are exlcuded. 5 | 6 | Note: Because I'm lazy and busy, babel becnhmark suite does not do many work. 7 | I expect `es2016` benchmark suite of other tools should be compared to `preset-env` of babel. 8 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "benchmark-done-right", 3 | "private": true, 4 | "version": "0.1.0", 5 | "description": "Many benchmarks suck, because they don't align tasks the tool does.", 6 | "main": "index.js", 7 | "scripts": { 8 | "build": "tsc", 9 | "bench": "npm run build && node index.js" 10 | }, 11 | "keywords": [], 12 | "author": "", 13 | "license": "ISC", 14 | "devDependencies": { 15 | "@babel/core": "^7.14.2", 16 | "@babel/preset-env": "^7.14.2", 17 | "@babel/preset-typescript": "^7.13.0", 18 | "@swc/core": "^1.2.57", 19 | "@types/babel__core": "^7.1.14", 20 | "@types/benchmark": "^2.1.0", 21 | "@types/node": "^15.0.3", 22 | "benchmark": "^2.1.4", 23 | "esbuild": "^0.11.20", 24 | "rxjs": "^7.0.1", 25 | "typescript": "^4.2.4" 26 | } 27 | } 28 | -------------------------------------------------------------------------------- /scripts/vis.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import matplotlib.pyplot as plt 4 | import json 5 | import os 6 | 7 | 8 | def process(f): 9 | [op, test_type, target] = f.split('-') 10 | target = target.replace('.json', '') 11 | with open('.data/%s' % f, 'r') as f: 12 | data = json.load(f) 13 | keys = list(data.keys()) 14 | keys_without_target = list(map(lambda s: s.split('(')[0].strip(), keys)) 15 | values = list(data.values()) 16 | plt.bar(keys_without_target, values) 17 | plt.title('%s (%s)' % (op, test_type)) 18 | plt.xlabel('Toolchain') 19 | plt.ylabel('Op/s (Higher is better)') 20 | os.makedirs('images/%s/%s' % (op, test_type), exist_ok=True) 21 | plt.savefig('images/%s/%s/%s.png' % (op, test_type, target)) 22 | plt.close() 23 | 24 | 25 | files = os.listdir('.data') 26 | for f in files: 27 | print('Data file:', f) 28 | process(f) 29 | -------------------------------------------------------------------------------- /util.ts: -------------------------------------------------------------------------------- 1 | import Benchmark from 'benchmark' 2 | import { JscTarget } from '@swc/core'; 3 | import * as ts from 'typescript'; 4 | import * as path from 'path'; 5 | import { promises as fs } from 'fs'; 6 | 7 | export async function store(op: string, res: { [name: string]: number }) { 8 | const dataDir = path.join(__dirname, '.data'); 9 | await fs.mkdir(dataDir, { recursive: true }); 10 | 11 | const f = path.join(dataDir, `${op}.json`); 12 | await fs.writeFile(f, JSON.stringify(res), 'utf8') 13 | } 14 | 15 | export async function storeResult(op: string, res: Benchmark.Target & Benchmark.Suite) { 16 | const reduced = res.reduce((arg: any, v: any) => { 17 | return { 18 | [v.name]: v.hz, 19 | ...arg 20 | }; 21 | }, {}); 22 | 23 | 24 | await store(op, reduced) 25 | } 26 | 27 | export function toTsTarget(target: JscTarget): ts.ScriptTarget { 28 | switch (target) { 29 | case "es3": 30 | return ts.ScriptTarget.ES3 31 | case "es5": 32 | return ts.ScriptTarget.ES5 33 | case "es2015": 34 | return ts.ScriptTarget.ES2015 35 | case "es2016": 36 | return ts.ScriptTarget.ES2016 37 | case "es2017": 38 | return ts.ScriptTarget.ES2017 39 | case "es2018": 40 | return ts.ScriptTarget.ES2018 41 | case "es2019": 42 | return ts.ScriptTarget.ES2019 43 | case "es2020": 44 | return ts.ScriptTarget.ES2020 45 | } 46 | } -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Logs 2 | logs 3 | *.log 4 | npm-debug.log* 5 | yarn-debug.log* 6 | yarn-error.log* 7 | lerna-debug.log* 8 | 9 | # Diagnostic reports (https://nodejs.org/api/report.html) 10 | report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json 11 | 12 | # Runtime data 13 | pids 14 | *.pid 15 | *.seed 16 | *.pid.lock 17 | 18 | # Directory for instrumented libs generated by jscoverage/JSCover 19 | lib-cov 20 | 21 | # Coverage directory used by tools like istanbul 22 | coverage 23 | *.lcov 24 | 25 | # nyc test coverage 26 | .nyc_output 27 | 28 | # Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files) 29 | .grunt 30 | 31 | # Bower dependency directory (https://bower.io/) 32 | bower_components 33 | 34 | # node-waf configuration 35 | .lock-wscript 36 | 37 | # Compiled binary addons (https://nodejs.org/api/addons.html) 38 | build/Release 39 | 40 | # Dependency directories 41 | node_modules/ 42 | jspm_packages/ 43 | 44 | # Snowpack dependency directory (https://snowpack.dev/) 45 | web_modules/ 46 | 47 | # TypeScript cache 48 | *.tsbuildinfo 49 | 50 | # Optional npm cache directory 51 | .npm 52 | 53 | # Optional eslint cache 54 | .eslintcache 55 | 56 | # Microbundle cache 57 | .rpt2_cache/ 58 | .rts2_cache_cjs/ 59 | .rts2_cache_es/ 60 | .rts2_cache_umd/ 61 | 62 | # Optional REPL history 63 | .node_repl_history 64 | 65 | # Output of 'npm pack' 66 | *.tgz 67 | 68 | # Yarn Integrity file 69 | .yarn-integrity 70 | 71 | # dotenv environment variables file 72 | .env 73 | .env.test 74 | 75 | # parcel-bundler cache (https://parceljs.org/) 76 | .cache 77 | .parcel-cache 78 | 79 | # Next.js build output 80 | .next 81 | out 82 | 83 | # Nuxt.js build / generate output 84 | .nuxt 85 | dist 86 | 87 | # Gatsby files 88 | .cache/ 89 | # Comment in the public line in if your project uses Gatsby and not Next.js 90 | # https://nextjs.org/blog/next-9-1#public-directory-support 91 | # public 92 | 93 | # vuepress build output 94 | .vuepress/dist 95 | 96 | # Serverless directories 97 | .serverless/ 98 | 99 | # FuseBox cache 100 | .fusebox/ 101 | 102 | # DynamoDB Local files 103 | .dynamodb/ 104 | 105 | # TernJS port file 106 | .tern-port 107 | 108 | # Stores VSCode versions used for testing VSCode extensions 109 | .vscode-test 110 | 111 | # yarn v2 112 | .yarn/cache 113 | .yarn/unplugged 114 | .yarn/build-state.yml 115 | .yarn/install-state.gz 116 | .pnp.* 117 | *.js 118 | -------------------------------------------------------------------------------- /tsconfig.json: -------------------------------------------------------------------------------- 1 | { 2 | "compilerOptions": { 3 | /* Visit https://aka.ms/tsconfig.json to read more about this file */ 4 | 5 | /* Basic Options */ 6 | // "incremental": true, /* Enable incremental compilation */ 7 | "target": "es2016" /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019', 'ES2020', or 'ESNEXT'. */, 8 | "module": "commonjs" /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', 'es2020', or 'ESNext'. */, 9 | // "lib": [], /* Specify library files to be included in the compilation. */ 10 | // "allowJs": true, /* Allow javascript files to be compiled. */ 11 | // "checkJs": true, /* Report errors in .js files. */ 12 | // "jsx": "preserve", /* Specify JSX code generation: 'preserve', 'react-native', 'react', 'react-jsx' or 'react-jsxdev'. */ 13 | // "declaration": true, /* Generates corresponding '.d.ts' file. */ 14 | // "declarationMap": true, /* Generates a sourcemap for each corresponding '.d.ts' file. */ 15 | // "sourceMap": true, /* Generates corresponding '.map' file. */ 16 | // "outFile": "./", /* Concatenate and emit output to single file. */ 17 | // "outDir": "./", /* Redirect output structure to the directory. */ 18 | // "rootDir": "./", /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */ 19 | // "composite": true, /* Enable project compilation */ 20 | // "tsBuildInfoFile": "./", /* Specify file to store incremental compilation information */ 21 | // "removeComments": true, /* Do not emit comments to output. */ 22 | // "noEmit": true, /* Do not emit outputs. */ 23 | // "importHelpers": true, /* Import emit helpers from 'tslib'. */ 24 | // "downlevelIteration": true, /* Provide full support for iterables in 'for-of', spread, and destructuring when targeting 'ES5' or 'ES3'. */ 25 | // "isolatedModules": true, /* Transpile each file as a separate module (similar to 'ts.transpileModule'). */ 26 | 27 | /* Strict Type-Checking Options */ 28 | "strict": true /* Enable all strict type-checking options. */, 29 | // "noImplicitAny": true, /* Raise error on expressions and declarations with an implied 'any' type. */ 30 | // "strictNullChecks": true, /* Enable strict null checks. */ 31 | // "strictFunctionTypes": true, /* Enable strict checking of function types. */ 32 | // "strictBindCallApply": true, /* Enable strict 'bind', 'call', and 'apply' methods on functions. */ 33 | // "strictPropertyInitialization": true, /* Enable strict checking of property initialization in classes. */ 34 | // "noImplicitThis": true, /* Raise error on 'this' expressions with an implied 'any' type. */ 35 | // "alwaysStrict": true, /* Parse in strict mode and emit "use strict" for each source file. */ 36 | 37 | /* Additional Checks */ 38 | // "noUnusedLocals": true, /* Report errors on unused locals. */ 39 | // "noUnusedParameters": true, /* Report errors on unused parameters. */ 40 | // "noImplicitReturns": true, /* Report error when not all code paths in function return a value. */ 41 | // "noFallthroughCasesInSwitch": true, /* Report errors for fallthrough cases in switch statement. */ 42 | // "noUncheckedIndexedAccess": true, /* Include 'undefined' in index signature results */ 43 | // "noPropertyAccessFromIndexSignature": true, /* Require undeclared properties from index signatures to use element accesses. */ 44 | 45 | /* Module Resolution Options */ 46 | // "moduleResolution": "node", /* Specify module resolution strategy: 'node' (Node.js) or 'classic' (TypeScript pre-1.6). */ 47 | // "baseUrl": "./", /* Base directory to resolve non-absolute module names. */ 48 | // "paths": {}, /* A series of entries which re-map imports to lookup locations relative to the 'baseUrl'. */ 49 | // "rootDirs": [], /* List of root folders whose combined content represents the structure of the project at runtime. */ 50 | // "typeRoots": [], /* List of folders to include type definitions from. */ 51 | // "types": [], /* Type declaration files to be included in compilation. */ 52 | // "allowSyntheticDefaultImports": true, /* Allow default imports from modules with no default export. This does not affect code emit, just typechecking. */ 53 | "esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */, 54 | // "preserveSymlinks": true, /* Do not resolve the real path of symlinks. */ 55 | // "allowUmdGlobalAccess": true, /* Allow accessing UMD globals from modules. */ 56 | 57 | /* Source Map Options */ 58 | // "sourceRoot": "", /* Specify the location where debugger should locate TypeScript files instead of source locations. */ 59 | // "mapRoot": "", /* Specify the location where debugger should locate map files instead of generated locations. */ 60 | // "inlineSourceMap": true, /* Emit a single file with source maps instead of having a separate file. */ 61 | // "inlineSources": true, /* Emit the source alongside the sourcemaps within a single file; requires '--inlineSourceMap' or '--sourceMap' to be set. */ 62 | 63 | /* Experimental Options */ 64 | // "experimentalDecorators": true, /* Enables experimental support for ES7 decorators. */ 65 | // "emitDecoratorMetadata": true, /* Enables experimental support for emitting type metadata for decorators. */ 66 | 67 | /* Advanced Options */ 68 | "skipLibCheck": true /* Skip type checking of declaration files. */, 69 | "forceConsistentCasingInFileNames": true /* Disallow inconsistently-cased references to the same file. */ 70 | } 71 | } 72 | -------------------------------------------------------------------------------- /compat-transform/common.ts: -------------------------------------------------------------------------------- 1 | // Copied from https://github.com/Brooooooklyn/swc-node/blob/master/bench/index.ts 2 | 3 | 4 | import fs from 'fs' 5 | import os from 'os' 6 | import path from 'path' 7 | 8 | import * as swc from '@swc/core'; 9 | import * as babel from '@babel/core' 10 | // @ts-expect-error 11 | import * as envPreset from '@babel/preset-env' 12 | // @ts-expect-error 13 | import * as tsPreset from '@babel/preset-typescript' 14 | import Benchmark, { Suite } from 'benchmark' 15 | import chalk from 'chalk' 16 | import { transformSync as transformSyncEsbuild, transform as transformEsbuild } from 'esbuild' 17 | import ts from 'typescript' 18 | import { JscTarget } from '@swc/core'; 19 | import { store, storeResult, toTsTarget } from '../util'; 20 | 21 | 22 | const cpuCount = os.cpus().length - 1 23 | 24 | const SOURCE_PATH = path.join(__dirname, '../node_modules/rxjs/src/internal/Observable.ts') 25 | const SOURCE_CODE = fs.readFileSync(SOURCE_PATH, 'utf-8') 26 | 27 | export async function run(target: JscTarget) { 28 | const asyncSuite = new Suite(`[${target}] Transform rxjs/Observable.ts async benchmark`) 29 | const parallelSuite = new Suite(`[${target}] Transform rxjs/Observable.ts parallel benchmark`) 30 | 31 | 32 | console.log(`[${chalk.green(target)}] Running single thread benchmarks`); 33 | await runSync(target); 34 | 35 | console.log(`[${chalk.green(target)}] Running async benchmarks`); 36 | await runAsync(target, 1, asyncSuite, 'async'); 37 | 38 | console.log(`[${chalk.green(target)}] Running parallel benchmarks`); 39 | await runAsync(target, cpuCount, parallelSuite, 'parallel'); 40 | } 41 | 42 | 43 | async function runSync(target: JscTarget) { 44 | let defer: () => void 45 | const task = new Promise((resolve) => { 46 | defer = resolve 47 | }) 48 | 49 | 50 | const suite = new Suite(`[${target}] Transform rxjs/Observable.ts benchmark`) 51 | 52 | suite 53 | .add(`swc (${target})`, () => { 54 | swc.transformSync(SOURCE_CODE, { 55 | filename: SOURCE_PATH, 56 | jsc: { 57 | target, 58 | }, 59 | sourceMaps: true, 60 | }) 61 | }); 62 | 63 | if (target !== 'es3' && target !== 'es5') { 64 | suite.add(`esbuild (${target})`, () => { 65 | transformSyncEsbuild(SOURCE_CODE, { 66 | sourcefile: SOURCE_PATH, 67 | loader: 'ts', 68 | sourcemap: true, 69 | minify: false, 70 | target, 71 | }) 72 | }); 73 | } 74 | 75 | suite.add(`typescript (${target})`, () => { 76 | ts.transpileModule(SOURCE_CODE, { 77 | fileName: SOURCE_PATH, 78 | compilerOptions: { 79 | target: toTsTarget(target), 80 | isolatedModules: true, 81 | sourceMap: true, 82 | }, 83 | }) 84 | }) 85 | .add(`babel (preset-env)`, () => { 86 | babel.transform(SOURCE_CODE, { 87 | filename: SOURCE_PATH, 88 | // TODO 89 | presets: [tsPreset, [envPreset, { targets: { node: 'current' } }]], 90 | configFile: false, 91 | babelrc: false, 92 | sourceMaps: true, 93 | }) 94 | }) 95 | .on('cycle', function (event: Benchmark.Event) { 96 | console.info(String(event.target)) 97 | }) 98 | .on('complete', async function (this: Benchmark.Target & Benchmark.Suite) { 99 | await storeResult(`transform-sync-${target}`, this); 100 | 101 | console.info( 102 | `${this.name} bench suite: Fastest is ${chalk.green( 103 | this.filter('fastest').map((s: Benchmark.Target) => s.name).toString(), 104 | )}`, 105 | ) 106 | defer() 107 | }) 108 | .run(); 109 | 110 | await task; 111 | } 112 | 113 | async function runAsync(target: JscTarget, parallel: number, suite: Suite, name: string) { 114 | let defer: () => void 115 | const task = new Promise((resolve) => { 116 | defer = resolve 117 | }) 118 | suite 119 | .add({ 120 | name: `swc (${target})`, 121 | fn: (deferred: any) => { 122 | Promise.all( 123 | Array.from({ length: parallel }).map(() => { 124 | return swc.transform(SOURCE_CODE, { 125 | filename: SOURCE_PATH, 126 | jsc: { 127 | target, 128 | }, 129 | sourceMaps: true, 130 | }) 131 | }), 132 | ) 133 | .then(() => { 134 | deferred.resolve() 135 | }) 136 | .catch((e) => { 137 | console.error(e) 138 | }) 139 | }, 140 | defer: true, 141 | async: true, 142 | queued: true, 143 | }); 144 | 145 | if (target !== 'es3' && target !== 'es5') { 146 | suite 147 | .add({ 148 | name: `esbuild (${target})`, 149 | fn: (deferred: any) => { 150 | Promise.all( 151 | Array.from({ length: parallel }).map(() => 152 | transformEsbuild(SOURCE_CODE, { 153 | sourcefile: SOURCE_PATH, 154 | loader: 'ts', 155 | sourcemap: true, 156 | minify: false, 157 | target, 158 | }), 159 | ), 160 | ) 161 | .then(() => { 162 | deferred.resolve() 163 | }) 164 | .catch((e) => { 165 | console.error(e) 166 | }) 167 | }, 168 | defer: true, 169 | async: true, 170 | queued: true, 171 | }) 172 | } 173 | suite 174 | .on('cycle', function (event: Benchmark.Event) { 175 | event.target.hz = event.target!.hz! * parallel 176 | console.info(String(event.target)) 177 | }) 178 | .on('complete', async function (this: Benchmark.Target & Benchmark.Suite) { 179 | await storeResult(`transform-${name}-${target}`, this); 180 | console.info( 181 | `${this.name} bench suite: Fastest is ${chalk.green( 182 | this.filter('fastest').map((t: Benchmark.Target) => t.name).toString(), 183 | )}`, 184 | ) 185 | defer() 186 | }); 187 | 188 | suite.run() 189 | 190 | await task 191 | } -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Benchmark for js dev tools, done right 2 | 3 | This repo contains correct benchmark for various tools. 4 | 5 | Many benchmarks suck, because they don't align tasks the tool does. 6 | Car costs more than apple for good reasons, but many people compares price of apple and car. 7 | 8 | To be fair, benchmarks are grouped base on their tasks. 9 | 10 | Each directory contains simple description for tasks. 11 | (But don't expect too much, I don't like documenting stuffs) 12 | 13 | ## Running benchmarks 14 | 15 | ```sh 16 | npm i 17 | npm run bench 18 | ``` 19 | 20 | ## Benchmark results 21 | 22 | Run on `MacBook Pro (13-inch, 2020, Four Thunderbolt 3 ports)`, `2.3 GHz Quad Core Intel Core i7`, `32GB 3733 MHz LPDDR4X`. 23 | 24 | What really matters is parallel benchmark because it represents real word usages, assuming the build tool you are using is not a dumb. 25 | 26 | ### Compatibility transforms 27 | 28 | Note: `esbuild` does not support `es3` nor `es5`. 29 | 30 | ```sh 31 | [es3] Running single thread benchmarks 32 | swc (es3) x 587 ops/sec ±1.39% (88 runs sampled) 33 | typescript (es3) x 46.43 ops/sec ±9.63% (61 runs sampled) 34 | babel (preset-env) x 58.23 ops/sec ±5.42% (64 runs sampled) 35 | [es3] Transform rxjs/Observable.ts benchmark bench suite: Fastest is swc (es3) 36 | [es3] Running async benchmarks 37 | swc (es3) x 495 ops/sec ±1.07% (81 runs sampled) 38 | [es3] Transform rxjs/Observable.ts async benchmark bench suite: Fastest is swc (es3) 39 | [es3] Running parallel benchmarks 40 | swc (es3) x 1,446 ops/sec ±1.27% (79 runs sampled) 41 | [es3] Transform rxjs/Observable.ts parallel benchmark bench suite: Fastest is swc (es3) 42 | [es5] Running single thread benchmarks 43 | swc (es5) x 630 ops/sec ±0.95% (89 runs sampled) 44 | typescript (es5) x 56.49 ops/sec ±5.42% (63 runs sampled) 45 | babel (preset-env) x 68.67 ops/sec ±3.21% (66 runs sampled) 46 | [es5] Transform rxjs/Observable.ts benchmark bench suite: Fastest is swc (es5) 47 | [es5] Running async benchmarks 48 | swc (es5) x 516 ops/sec ±1.09% (81 runs sampled) 49 | [es5] Transform rxjs/Observable.ts async benchmark bench suite: Fastest is swc (es5) 50 | [es5] Running parallel benchmarks 51 | swc (es5) x 1,559 ops/sec ±1.11% (79 runs sampled) 52 | [es5] Transform rxjs/Observable.ts parallel benchmark bench suite: Fastest is swc (es5) 53 | [es2015] Running single thread benchmarks 54 | swc (es2015) x 636 ops/sec ±1.38% (90 runs sampled) 55 | esbuild (es2015) x 38.58 ops/sec ±0.47% (67 runs sampled) 56 | typescript (es2015) x 64.99 ops/sec ±5.74% (66 runs sampled) 57 | babel (preset-env) x 72.63 ops/sec ±2.08% (74 runs sampled) 58 | [es2015] Transform rxjs/Observable.ts benchmark bench suite: Fastest is swc (es2015) 59 | [es2015] Running async benchmarks 60 | swc (es2015) x 510 ops/sec ±1.31% (82 runs sampled) 61 | esbuild (es2015) x 634 ops/sec ±2.90% (75 runs sampled) 62 | [es2015] Transform rxjs/Observable.ts async benchmark bench suite: Fastest is esbuild (es2015) 63 | [es2015] Running parallel benchmarks 64 | swc (es2015) x 1,513 ops/sec ±1.15% (82 runs sampled) 65 | esbuild (es2015) x 1,331 ops/sec ±2.46% (75 runs sampled) 66 | [es2015] Transform rxjs/Observable.ts parallel benchmark bench suite: Fastest is swc (es2015) 67 | [es2016] Running single thread benchmarks 68 | swc (es2016) x 890 ops/sec ±0.81% (90 runs sampled) 69 | esbuild (es2016) x 33.95 ops/sec ±0.63% (59 runs sampled) 70 | typescript (es2016) x 65.74 ops/sec ±6.08% (68 runs sampled) 71 | babel (preset-env) x 71.87 ops/sec ±2.82% (70 runs sampled) 72 | [es2016] Transform rxjs/Observable.ts benchmark bench suite: Fastest is swc (es2016) 73 | [es2016] Running async benchmarks 74 | swc (es2016) x 704 ops/sec ±1.18% (81 runs sampled) 75 | esbuild (es2016) x 712 ops/sec ±0.92% (81 runs sampled) 76 | [es2016] Transform rxjs/Observable.ts async benchmark bench suite: Fastest is esbuild (es2016),swc (es2016) 77 | [es2016] Running parallel benchmarks 78 | swc (es2016) x 2,118 ops/sec ±0.99% (82 runs sampled) 79 | esbuild (es2016) x 1,492 ops/sec ±1.17% (82 runs sampled) 80 | [es2016] Transform rxjs/Observable.ts parallel benchmark bench suite: Fastest is swc (es2016) 81 | [es2017] Running single thread benchmarks 82 | swc (es2017) x 901 ops/sec ±0.77% (90 runs sampled) 83 | esbuild (es2017) x 27.42 ops/sec ±0.40% (49 runs sampled) 84 | typescript (es2017) x 67.37 ops/sec ±5.12% (71 runs sampled) 85 | babel (preset-env) x 73.05 ops/sec ±2.55% (66 runs sampled) 86 | [es2017] Transform rxjs/Observable.ts benchmark bench suite: Fastest is swc (es2017) 87 | [es2017] Running async benchmarks 88 | swc (es2017) x 689 ops/sec ±2.89% (80 runs sampled) 89 | esbuild (es2017) x 692 ops/sec ±1.50% (80 runs sampled) 90 | [es2017] Transform rxjs/Observable.ts async benchmark bench suite: Fastest is esbuild (es2017),swc (es2017) 91 | [es2017] Running parallel benchmarks 92 | swc (es2017) x 1,929 ops/sec ±1.64% (80 runs sampled) 93 | esbuild (es2017) x 1,449 ops/sec ±1.21% (80 runs sampled) 94 | [es2017] Transform rxjs/Observable.ts parallel benchmark bench suite: Fastest is swc (es2017) 95 | [es2018] Running single thread benchmarks 96 | swc (es2018) x 894 ops/sec ±0.92% (90 runs sampled) 97 | esbuild (es2018) x 23.52 ops/sec ±0.48% (43 runs sampled) 98 | typescript (es2018) x 68.45 ops/sec ±5.41% (68 runs sampled) 99 | babel (preset-env) x 72.72 ops/sec ±3.30% (66 runs sampled) 100 | [es2018] Transform rxjs/Observable.ts benchmark bench suite: Fastest is swc (es2018) 101 | [es2018] Running async benchmarks 102 | swc (es2018) x 693 ops/sec ±1.22% (81 runs sampled) 103 | esbuild (es2018) x 702 ops/sec ±1.06% (80 runs sampled) 104 | [es2018] Transform rxjs/Observable.ts async benchmark bench suite: Fastest is esbuild (es2018),swc (es2018) 105 | [es2018] Running parallel benchmarks 106 | swc (es2018) x 2,110 ops/sec ±1.41% (81 runs sampled) 107 | esbuild (es2018) x 1,482 ops/sec ±0.98% (81 runs sampled) 108 | [es2018] Transform rxjs/Observable.ts parallel benchmark bench suite: Fastest is swc (es2018) 109 | [es2019] Running single thread benchmarks 110 | swc (es2019) x 909 ops/sec ±0.57% (92 runs sampled) 111 | esbuild (es2019) x 20.52 ops/sec ±0.58% (38 runs sampled) 112 | typescript (es2019) x 67.13 ops/sec ±5.23% (69 runs sampled) 113 | babel (preset-env) x 72.36 ops/sec ±3.30% (67 runs sampled) 114 | [es2019] Transform rxjs/Observable.ts benchmark bench suite: Fastest is swc (es2019) 115 | [es2019] Running async benchmarks 116 | swc (es2019) x 698 ops/sec ±1.26% (82 runs sampled) 117 | esbuild (es2019) x 709 ops/sec ±1.02% (81 runs sampled) 118 | [es2019] Transform rxjs/Observable.ts async benchmark bench suite: Fastest is esbuild (es2019),swc (es2019) 119 | [es2019] Running parallel benchmarks 120 | swc (es2019) x 2,134 ops/sec ±1.18% (82 runs sampled) 121 | esbuild (es2019) x 1,499 ops/sec ±1.00% (82 runs sampled) 122 | [es2019] Transform rxjs/Observable.ts parallel benchmark bench suite: Fastest is swc (es2019) 123 | [es2020] Running single thread benchmarks 124 | swc (es2020) x 984 ops/sec ±0.81% (90 runs sampled) 125 | esbuild (es2020) x 18.16 ops/sec ±0.48% (49 runs sampled) 126 | typescript (es2020) x 67.06 ops/sec ±5.37% (68 runs sampled) 127 | babel (preset-env) x 72.21 ops/sec ±3.25% (68 runs sampled) 128 | [es2020] Transform rxjs/Observable.ts benchmark bench suite: Fastest is swc (es2020) 129 | [es2020] Running async benchmarks 130 | swc (es2020) x 762 ops/sec ±1.09% (80 runs sampled) 131 | esbuild (es2020) x 697 ops/sec ±1.08% (80 runs sampled) 132 | [es2020] Transform rxjs/Observable.ts async benchmark bench suite: Fastest is swc (es2020) 133 | [es2020] Running parallel benchmarks 134 | swc (es2020) x 2,322 ops/sec ±1.10% (80 runs sampled) 135 | esbuild (es2020) x 1,477 ops/sec ±1.04% (80 runs sampled) 136 | [es2020] Transform rxjs/Observable.ts parallel benchmark bench suite: Fastest is swc (es2020) 137 | ``` 138 | --------------------------------------------------------------------------------