├── .editorconfig ├── .gitattributes ├── .gitignore ├── LICENSE ├── README.md ├── assets └── images │ └── gh-cover.png ├── benchmark ├── _standardProcessing.js ├── _turbitProcessing.js └── speedTest.js ├── examples ├── README.md ├── extended │ ├── basicSentimentAnalysis.js │ ├── passwordStrengthChecker.js │ ├── toUpperCaseProcessing.js │ └── transactionRiskAnalysis.js └── simple │ ├── generateRandomNumbers.js │ └── taskRunner.js ├── package.json └── turbit.js /.editorconfig: -------------------------------------------------------------------------------- 1 | # http://editorconfig.org 2 | 3 | root = true 4 | 5 | [*] 6 | indent_style = space 7 | indent_size = 4 8 | end_of_line = lf 9 | charset = utf-8 10 | trim_trailing_whitespace = true 11 | insert_final_newline = false -------------------------------------------------------------------------------- /.gitattributes: -------------------------------------------------------------------------------- 1 | # Auto detect text files and perform LF normalization 2 | * text=auto 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | .idea 3 | node_modules 4 | npm-debug.log 5 | package-lock.json -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Jose Pino 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Turbit 2 | 3 | ![turbit](assets/images/gh-cover.png) 4 | 5 | ## High-speed Multicore Computing 6 | 7 | [![npm version](https://img.shields.io/npm/v/turbit.svg)](https://www.npmjs.com/package/turbit) 8 | 9 | Turbit is an advanced high-speed multicore computing library in Node.js, designed to optimize performance for computationally intensive operations by leveraging parallel processing across multiple CPU cores. 10 | 11 | > Create powerful applications, scripts, and automations with enhanced 12 | > performance through parallel processing. **Turbit was designed for the multi-core era.** 13 | 14 | ## Key Applications 15 | 16 | Turbit excels at optimizing resource-intensive operations, making it ideal for: 17 | 18 | | Application | Description | 19 | |-------------|-------------| 20 | | Data Processing and Analysis | Efficiently handle and analyze large volumes of data | 21 | | Scientific Computations | Accelerate complex scientific calculations and simulations | 22 | | Batch Processing | Process large datasets quickly and efficiently | 23 | | Complex Algorithmic Operations | Speed up execution of complex algorithms and mathematical operations | 24 | 25 | ## Inspiration 26 | 27 | As a researcher, I often struggled to find a truly user-friendly implementation for parallel processing in Node.js. Most existing solutions were not user-friendly, complex, and unwieldy, making it challenging to efficiently harness CPU power for high-performance tasks. This inspired me to create Turbit, a library that simplifies parallel processing in applications and processes, allowing developers to easily tap into the full potential of their hardware without getting bogged down in implementation details. With AI advancements, many people have chosen to exclusively use the GPU to process data and perform complex tasks. However, although the CPU is not always directly compared to the GPU in terms of performance for certain tasks, it also has great potential that should not be underestimated. Turbit facilitates the harnessing of this potential, enabling efficient parallel processing on the CPU. 28 | 29 | _*One of my goals when creating things is always to simplify them as much as possible for others and, of course, to make them unique.*_ 30 | 31 | > The code is art, akin to music and other creative fields. I created Turbit out of passion and love for coding. Writing code is not just a daily habit for me, it's a task of focus and discipline, driven by my lifelong love for technology. During the final stages of developing this library and its two powerful use cases, throughout the code writing process, I constantly listened to some songs that I'd like to share: 32 | > 33 | > - [Monaco](https://www.youtube.com/watch?v=_PJvpq8uOZM) - Bad Bunny: "Dime (dime), dime, ¿esto es lo que tú querías?" 34 | > - [Los Pits](https://www.youtube.com/watch?v=yaHf1FwMYA4) - Bad Bunny: "Te lo dije que me hacía millo antes de los treinta." 35 | > - [1 of 1](https://www.youtube.com/watch?v=21Z0GoTYqkE) - Maluma: "Mi carrera es una chimba 1 of 1, me voy y vuelvo y los bajo de number one." 36 | > - [Mírame](https://www.youtube.com/watch?v=zcw8NlHljF4) - Blessd: "No cualquiera puede aguantar mi ritmo." 37 | > - [Vuelve candy b](https://www.youtube.com/watch?v=UBVm40IONzw) - Bad Bunny: "¿Cómo tú vas a dudar del más cabrón que le mete?" 38 | > - [Otra noche en miami](https://www.youtube.com/watch?v=hoQmSA6MRAk) - Bad Bunny: "Todo es superficial, nada real, nada raro que el dinero no pueda comprar." 39 | > - [Decisions](https://www.youtube.com/watch?v=jQd5OEl1W-Q) - Borgore & Miley Cyrus: "Decisions, but I want it all." 40 | > - [Si tú supieras](https://www.youtube.com/watch?v=75gaEbTLqpg) - Feid: "Baby, perdón, pero el tiempo que no estoy contigo es tiempo perdido." 41 | > - [Brickell](https://www.youtube.com/watch?v=2p6O7EmroEs) - Feid: "Ojalá me hubieras dicho que era la última vez." 42 | > - [Meek Mill](https://www.youtube.com/watch?v=S1gp0m4B5p8) - Drake: "Playin' with my name, that's lethal, dawg (who you say you was?)" 43 | > - [La Jumpa](https://www.youtube.com/watch?v=ubbE6gyBf8k) - Arcangel & Bad Bunny: "Hoy tú te vas con una leyenda que no va a volver a nacer." 44 | 45 | As I reflect on this project, I can't help but wonder: Perhaps these are among the last lines of code written 100% by a human. Regardless, I hope you enjoy this code as much as I enjoyed writing it. 46 | 47 | > Always the same creative hacker, but more evolved. - [Jose Pino](https://x.com/jofpin) 48 | 49 | ## Getting Started 50 | 51 | Turbit's only dependency is the built-in `child_process` module in Node.js for managing child processes. 52 | 53 | To use Turbit, you need to have **[Node.js](https://nodejs.org/)** installed. Then, you can install Turbit using npm: 54 | 55 | ```shell 56 | npm install turbit 57 | ``` 58 | 59 | After installation, import it in your code as follows: 60 | 61 | ```js 62 | const Turbit = require("turbit"); 63 | // Create a Turbit instance for parallel processing 64 | const turbit = Turbit(); 65 | ``` 66 | 67 | ## Usage 68 | 69 | #### RUN (Main Function) 70 | 71 | Turbit provides a `run` function to execute tasks across multiple cores for parallel processing. 72 | 73 | ```js 74 | turbit.run(func, options) 75 | ``` 76 | 77 | **Parameters** 78 | 79 | 1. `func` (Function): The function to be executed in parallel. This is the task you want to distribute across multiple cores. 80 | 81 | 2. `options` (**Object**): Configuration options for the execution. 82 | - `type` (**required**, string): Specifies the type of execution. Can be either "simple" or "extended". 83 | - **simple**: Use this when you have a single task that doesn't require input data. 84 | - **extended**: Use this when you have a set of data that needs to be processed in parallel. It divides the input data into chunks and distributes them across multiple processes for simultaneous processing. 85 | 86 | - `data` (**optional**, array): The data to be processed in parallel. This is required for "extended" type executions. Each item in this array will be passed as an argument to your function. 87 | 88 | - `args` (**optional**, object): Additional arguments to pass to the function. These will be passed to your function along with the data item (for "extended" type). 89 | 90 | - `power` (**required**, number): Controls the intensity of parallel processing. Values range from 1 to 100 for normal system resource utilization. Values above 100 enable an intentional system overload mode, allowing for more processes than the system would typically handle. Default is 70. 91 | - `power: 1-100`: Normal utilization of system resources. 92 | - `power: > 100`: Overload mode. Allows generating more processes than the system would normally handle. 93 | 94 | **Note**: Using `power` > 100 can lead to increased RAM usage, significant CPU utilization, and potential system instability. Use this feature with caution and only when necessary for extremely intensive processing tasks. Suitable use cases include: 95 | - Processing large datasets in limited time 96 | - Complex simulations requiring multiple parallel iterations 97 | - High-intensity rendering or scientific calculations 98 | - Intensive cryptographic operations and brute-force processes 99 | 100 | **Returns** 101 | - A promise that resolves with the execution results and statistics. 102 | - `data` (**Array**): The results of the executed function. 103 | - `stats` (**Object**): Performance statistics of the execution. 104 | - `timeTakenSeconds`: The total execution time in seconds. 105 | - `numProcessesUsed`: The number of processes used for the execution. 106 | - `dataProcessed`: The amount of data items processed. 107 | - `memoryUsed`: The amount of memory used during execution. 108 | 109 | 110 | The `stats` object provides valuable insights into the performance of your parallel processing tasks. It allows you to monitor execution time, resource utilization, and processing efficiency, which can be crucial for optimizing your applications. 111 | 112 | #### KILL (Function) 113 | 114 | Turbit provides a `kill` function to terminate all active child processes and free system resources. 115 | 116 | ```js 117 | turbit.kill(); 118 | ``` 119 | 120 | ## Architecture 121 | 122 | Turbit utilizes a parallel processing architecture based on Node.js `child_process` module. Here's a brief overview of how it works: 123 | 124 | 1. **Initialization**: Turbit creates a pool of child processes up to the maximum number of available CPU cores when the Turbit instance is created. 125 | 126 | 2. **Task Distribution**: When `turbit.run()` is called, the library prepares the task for parallel execution. For '**simple**' type, it replicates the task across processes. For '**extended**' type, it divides the input data into chunks for distribution across processes. The number of processes used is determined by the `power` parameter. 127 | 128 | 3. **Process Management**: Turbit manages the pool of child processes, creating additional processes or terminating excess ones as needed based on the `power` parameter for each `run()` call. 129 | 130 | 4. **Parallel Execution**: Tasks are distributed across child processes, allowing for simultaneous utilization of multiple CPU cores. 131 | 132 | 5. **Inter-Process Communication**: Turbit uses Node.js built-in messaging system to send function definitions and arguments to child processes, and to receive results back. 133 | 134 | 6. **Result Collection**: As child processes complete their tasks, results are gathered and consolidated into a single output. 135 | 136 | 7. **Performance Tracking**: Turbit monitors and reports execution time, number of processes used, amount of data processed, and memory consumption. 137 | 138 | ## Examples 139 | 140 | Here are some examples of how to use Turbit: 141 | 142 | #### Simple Execution 143 | 144 | ```js 145 | // Define a simple task function 146 | const task = function() { 147 | return "Hello, humans and intelligent machines!"; 148 | } 149 | 150 | // Implement Turbit for parallel processing 151 | turbit.run(task, { type: "simple", power: 100 }) 152 | .then(result => { 153 | console.log("Simple execution result:", result.data); 154 | turbit.kill(); // Cleans up child processes after completing the task 155 | }) 156 | .catch(error => { 157 | console.error("Error in simple execution:", error); 158 | turbit.kill(); // Make sure to call kill even if there's an error 159 | }); 160 | ``` 161 | 162 | #### Extended Execution 163 | 164 | ```js 165 | // Define a calculation function to be executed in parallel 166 | function calculate(item) { 167 | return item * 12; 168 | } 169 | 170 | // Implement Turbit for parallel processing 171 | turbit.run(calculate, { 172 | type: "extended", // Use extended mode for processing an array of data 173 | data: [1, 2, 3, 4], // Input data to be processed in parallel 174 | power: 75 // Set processing intensity to 75% of available resources 175 | }) 176 | .then(result => { 177 | console.log("Extended execution results:", result.data); 178 | turbit.kill(); // Terminate all active child processes 179 | }) 180 | .catch(error => { 181 | console.error("Error in extended execution:", error); 182 | turbit.kill(); // Ensure termination of child processes in case of error 183 | }); 184 | ``` 185 | 186 | #### Turbit Showcase: Parallel Password Security Cracking 187 | 188 | This example demonstrates a more complex use of Turbit in a cybersecurity context by simulating parallel dictionary-based password cracking. Please note that this is an example code, designed and limited to show another Turbit implementation scenario for educational purposes. It's not recommended to use it for real-world password cracking: 189 | 190 | ```js 191 | /** 192 | * Script functionality: 193 | * - Simulates a user database with passwords 194 | * - Uses a dictionary to attempt password cracking 195 | * - Leverages Turbit for parallel processing to enhance cracking speed 196 | * - Demonstrates efficiency in data-intensive security tasks 197 | */ 198 | 199 | // Importing turbit (you must install with 'npm install turbit' in the terminal) 200 | const Turbit = require("turbit"); 201 | 202 | /** 203 | * Executes a simulated password cracking attempt using Turbit distributed computing. 204 | */ 205 | const basicPasswordCracker = { 206 | attack: async function({ users, dictionary }) { 207 | if (!users || !dictionary) { 208 | throw new Error("Both users and dictionary must be provided"); 209 | } 210 | 211 | // Create a Turbit instance for parallel processing 212 | const turbit = Turbit(); 213 | 214 | try { 215 | // Implement Turbit for parallel processing: Distribute the cracking workload across available resources 216 | const result = await turbit.run( 217 | (input) => { 218 | const { data, args } = input; 219 | const { dictionary } = args; 220 | 221 | // Process each user password in parallel 222 | return data.map(user => { 223 | for (const word of dictionary) { 224 | if (word === user.password) { 225 | return { ...user, crackedPassword: word, attempts: dictionary.indexOf(word) + 1 }; 226 | } 227 | } 228 | return { ...user, crackedPassword: null, attempts: dictionary.length }; 229 | }); 230 | }, 231 | { 232 | type: "extended", // Use extended type for processing an array of data 233 | data: users, // The array of user objects to process 234 | args: { dictionary }, // Additional arguments passed to the function 235 | power: 100 // Utilize full available computing power 236 | } 237 | ); 238 | 239 | // Compile and format the results 240 | const output = { 241 | results: result.data.map(user => ({ 242 | username: user.username, 243 | passwordCracked: user.crackedPassword !== null, 244 | crackedPassword: user.crackedPassword, 245 | attempts: user.attempts 246 | })), 247 | stats: result.stats // Include performance metrics 248 | }; 249 | 250 | // Output the results in a readable JSON format 251 | console.log(JSON.stringify(output, null, 2)); 252 | } catch (error) { 253 | console.error("Error during execution:", error); 254 | } finally { 255 | turbit.kill(); // Ensure proper cleanup of Turbit resources 256 | } 257 | } 258 | }; 259 | 260 | // Demonstrating the setup and execution of the password cracker 261 | (async () => { 262 | 263 | // Simulated victim user database (for demonstration purposes only) 264 | const victims = [ 265 | { username: "elonmusk", password: "Mars2028@" }, 266 | { username: "stevejobs", password: "iThink1Different" }, 267 | { username: "josepino", password: "H4ck3r2024" }, 268 | { username: "maxi", password: "theNextGeneration" }, 269 | { username: "lucian", password: "2024Created" } 270 | ]; 271 | 272 | // Sample dictionary for the cracking attempt 273 | const dictionary = [ 274 | "dreams2027", "123456", "qwerty", "admin", "letmein", 275 | "welcome", "rocket", "1234", "12345", "test", "pino", 276 | "medellin", "miami", "letme1n", "abcdef12345", "1337", 277 | "hack1ng", "trustno1", "thegoat", "master", "Mars2028@", 278 | "iThink1Different", "H4ck3r2024", "theNextGeneration", 279 | "NextComputer", "SpaceX", "Tesla", "singapur", "Neuralink", "Apple", 280 | "iPhone", "Macintosh", "SiliconValley", "Innovation", "china" 281 | "Coding", "Ovnis", "Dubai", "1234567890", "jofpin" 282 | ]; 283 | 284 | try { 285 | await basicPasswordCracker.attack({ victims, dictionary }); 286 | } catch (error) { 287 | console.error("Error running password cracker:", error); 288 | } 289 | })(); 290 | ``` 291 | 292 | Check out the [examples](examples) directory for more detailed usage examples. 293 | 294 | ## Benchmarking 295 | 296 | Turbit includes a built-in benchmarking tool to measure the performance gains of parallel processing. To run the benchmark, use the following command in your terminal: 297 | 298 | ```shell 299 | node benchmark/speedTest.js 300 | ``` 301 | 302 | This tool compares standard sequential processing with Turbit parallel processing, providing: 303 | 304 | 1. Execution times for each method 305 | 2. Average processing times 306 | 3. Performance improvements (speed increase, time saved) 307 | 4. Practical implications of using Turbit 308 | 309 | Use this benchmark to quantify Turbit benefits for your specific use cases and optimize your parallel processing implementations. 310 | 311 | #### Custom Benchmarking 312 | 313 | You can also benchmark your own scripts by modifying [`benchmark/speedTest.js`](benchmark/speedTest.js): 314 | 315 | ```javascript 316 | SpeedTest.benchmark({ 317 | standardScript: "path/standard_script.js", 318 | turbitScript: "path/turbit_script.js" 319 | }); 320 | ``` 321 | 322 | ## Tools Powered by Turbit 323 | 324 | I created these two powerful tools with unique interfaces to demonstrate the potential of Turbit through real-world applications: 325 | 326 | - [**synthBTC**](https://github.com/jofpin/synthBTC): A tool that leverages advanced Monte Carlo simulations to generate Bitcoin price prediction scenarios. 327 | - **bitbreak**: A high-performance Bitcoin private key brute-force tool. [IN PROGRESS] 328 | 329 | ------------ 330 | 331 | > #### ⚠️ **RECOMMENDATION** 332 | 333 | > For optimal performance, consider the size and complexity of your tasks. Very small tasks might not benefit from parallelization due to the overhead of creating and managing child processes. 334 | 335 | > #### ℹ️ **TIPS** 336 | 337 | > 1. Use the "extended" type for data-parallel tasks where you have a large array of items to process. 338 | > 2. Adjust the `power` option based on your system's capabilities and the nature of your tasks. 339 | 340 | ## License 341 | 342 | The content of this project itself is licensed under the [Creative Commons Attribution 3.0 license](http://creativecommons.org/licenses/by/3.0/us/deed.en_US), and the underlying source code used to format and display that content is licensed under the [MIT license](LICENSE). 343 | 344 | Copyright (c) 2024 by [**Jose Pino**](https://x.com/jofpin) 345 | -------------------------------------------------------------------------------- /assets/images/gh-cover.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jofpin/turbit/9f811944198a29eca8542573d27560c2cb0c1802/assets/images/gh-cover.png -------------------------------------------------------------------------------- /benchmark/_standardProcessing.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Standard Processing Benchmark 3 | * 4 | * Generates and sorts large arrays of random numbers using standard JavaScript. 5 | * Creates 1,000,000 random numbers, sorts them, and repeats 10 times. 6 | * Measures total execution time for performance comparison. 7 | */ 8 | function generateAndSortNumbers(size) { 9 | const numbers = Array.from({ length: size }, () => Math.random()); 10 | return numbers.sort((a, b) => a - b); 11 | } 12 | 13 | function runWithoutTurbit(size, iterations) { 14 | const start = Date.now(); 15 | 16 | for (let i = 0; i < iterations; i++) { 17 | generateAndSortNumbers(size); 18 | } 19 | 20 | const end = Date.now(); 21 | return end - start; 22 | } 23 | 24 | function main() { 25 | const size = 1000000; 26 | const iterations = 10; 27 | 28 | const standardTime = runWithoutTurbit(size, iterations); 29 | console.log(`Standard Time: ${standardTime}ms`); 30 | } 31 | 32 | main(); -------------------------------------------------------------------------------- /benchmark/_turbitProcessing.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Turbit-Enhanced Processing Benchmark 3 | * 4 | * Uses Turbit for parallel processing to generate and sort large arrays. 5 | * Creates 1,000,000 random numbers, sorts them, and repeats 10 times. 6 | * Utilizes 30% of available CPU cores for distributed processing. 7 | * Measures total execution time for comparison with Standard method. 8 | */ 9 | let Turbit; 10 | 11 | try { 12 | // Try to import the installed version of Turbit 13 | Turbit = require("turbit"); 14 | } catch (error) { 15 | // If the library is not installed, use the local version 16 | Turbit = require("../turbit"); 17 | } 18 | 19 | const turbit = Turbit(); 20 | 21 | function generateAndSortNumbers(size) { 22 | const numbers = Array.from({ length: size }, () => Math.random()); 23 | return numbers.sort((a, b) => a - b); 24 | } 25 | 26 | async function runWithTurbit(size, iterations) { 27 | const start = Date.now(); 28 | 29 | for (let i = 0; i < iterations; i++) { 30 | await turbit.run(generateAndSortNumbers, { 31 | type: "simple", 32 | args: size, 33 | power: 30 34 | }); 35 | } 36 | 37 | const end = Date.now(); 38 | return end - start; 39 | } 40 | 41 | async function main() { 42 | const size = 1000000; 43 | const iterations = 10; 44 | 45 | const turbitTime = await runWithTurbit(size, iterations); 46 | console.log(`Turbit Time: ${turbitTime}ms`); 47 | 48 | turbit.kill(); 49 | } 50 | 51 | main(); -------------------------------------------------------------------------------- /benchmark/speedTest.js: -------------------------------------------------------------------------------- 1 | /** 2 | * speedTest.js 3 | * Benchmarking Turbit performance against standard processing. 4 | */ 5 | const { execSync } = require("child_process"); 6 | 7 | const SpeedTest = { 8 | // Configuration for the benchmark 9 | CONFIG: { 10 | iterations: 5, // Number of benchmark iterations 11 | warmupRuns: 3 // Number of warmup runs before the actual benchmark 12 | }, 13 | 14 | callScript(scriptName, isWarmup = false) { 15 | if (!isWarmup) { 16 | process.stdout.write(`- Running \x1b[33m${scriptName}\x1b[0m\n`); 17 | } 18 | const start = process.hrtime.bigint(); 19 | execSync(`node ${scriptName}`); 20 | const end = process.hrtime.bigint(); 21 | if (!isWarmup) { 22 | process.stdout.write(""); 23 | } 24 | return Number(end - start) / 1e6; 25 | }, 26 | 27 | calculateTime(ms) { 28 | return ms < 1000 ? `${ms.toFixed(2)} milliseconds` : `${(ms / 1000).toFixed(2)} seconds`; 29 | }, 30 | 31 | calculateSpeedup(standardTime, turbitTime) { 32 | return ((standardTime - turbitTime) / standardTime) * 100; 33 | }, 34 | 35 | calculateAverage(times) { 36 | return times.reduce((a, b) => a + b, 0) / times.length; 37 | }, 38 | 39 | benchmark(options = {}) { 40 | const standardScript = options.standardScript || "_standardProcessing.js"; 41 | const turbitScript = options.turbitScript || "_turbitProcessing.js"; 42 | 43 | console.clear(); 44 | console.log("\x1b[36m%s\x1b[0m", "╔══════════════════════════════╗"); 45 | console.log("\x1b[36m%s\x1b[0m", "║ BENCHMARK: Turbit Speed Test ║"); 46 | console.log("\x1b[36m%s\x1b[0m", "╚══════════════════════════════╝\n"); 47 | 48 | // Warmup: Executes both standard and Turbit scripts multiple times 49 | // to stabilize performance before the actual benchmark. This helps to 50 | // reduce the impact of initial overhead and caching effects. 51 | console.log("\x1b[36m%s\x1b[0m", "Warmup Runs:"); 52 | for (let i = 0; i < this.CONFIG.warmupRuns; i++) { 53 | this.callScript(standardScript, true); 54 | this.callScript(turbitScript, true); 55 | console.log(`> Run ${i + 1} \x1b[32m[completed]\x1b[0m`); 56 | } 57 | 58 | // Benchmark: Runs both standard and Turbit scripts multiple times, 59 | // measuring the execution time for each run. These measurements are used 60 | // to calculate average performance and speedup metrics. 61 | console.log("\n\x1b[36m%s\x1b[0m", "Benchmark Runs:"); 62 | const standardTimes = []; 63 | const turbitTimes = []; 64 | 65 | for (let i = 0; i < this.CONFIG.iterations; i++) { 66 | standardTimes.push(this.callScript(standardScript)); 67 | turbitTimes.push(this.callScript(turbitScript)); 68 | console.log(`> Run ${i + 1} \x1b[32m[completed]\x1b[0m\n`); 69 | } 70 | 71 | // Calculate performance metrics 72 | const avgStandardTime = this.calculateAverage(standardTimes); 73 | const avgTurbitTime = this.calculateAverage(turbitTimes); 74 | const speedup = this.calculateSpeedup(avgStandardTime, avgTurbitTime); 75 | const timeSaved = avgStandardTime - avgTurbitTime; 76 | const timesFaster = avgStandardTime / avgTurbitTime; 77 | 78 | // Display results 79 | console.log("\n\x1b[36m%s\x1b[0m", "REPORT:\n"); 80 | 81 | console.log("\x1b[36m%s\x1b[0m", "1. Execution History"); 82 | console.log(" Standard Method:"); 83 | standardTimes.forEach((time, index) => { 84 | console.log(` Run ${index + 1}: \x1b[33m${this.calculateTime(time)}\x1b[0m`); 85 | }); 86 | console.log(" Turbit Method:"); 87 | turbitTimes.forEach((time, index) => { 88 | console.log(` Run ${index + 1}: \x1b[32m${this.calculateTime(time)}\x1b[0m`); 89 | }); 90 | 91 | console.log("\n\x1b[36m%s\x1b[0m", "2. Average Processing Times"); 92 | console.log(` • Standard method: \x1b[33m${this.calculateTime(avgStandardTime)}\x1b[0m`); 93 | console.log(` • Turbit method: \x1b[32m${this.calculateTime(avgTurbitTime)}\x1b[0m`); 94 | 95 | console.log("\n\x1b[36m%s\x1b[0m", "3. Turbit Performance"); 96 | console.log(` • Up to \x1b[32m${timesFaster.toFixed(0)}x\x1b[0m faster`); 97 | console.log(` • \x1b[32m${speedup.toFixed(0)}%\x1b[0m less processing time`); 98 | console.log(` • \x1b[32m${this.calculateTime(timeSaved)}\x1b[0m saved per task`); 99 | 100 | console.log("\n\x1b[36m%s\x1b[0m", "4. What this means for you:"); 101 | if (speedup > 0) { 102 | console.log(` • Tasks that took ${this.calculateTime(avgStandardTime)} now complete in just ${this.calculateTime(avgTurbitTime)}.`); 103 | console.log(` • Turbit processes your data ${timesFaster.toFixed(1)} times faster than standard methods.`); 104 | console.log(` • You'll save ${this.calculateTime(timeSaved)} on every run, allowing you to do more in less time.\n`); 105 | } else if (speedup < 0) { 106 | console.log(` In this case, the standard method was slightly faster on average.`); 107 | console.log(` This can happen with very small tasks or due to system variability.\n`); 108 | } else { 109 | console.log(` Both methods took approximately the same amount of time on average.\n`); 110 | } 111 | console.log(` Note: Results are based on ${this.CONFIG.iterations} iterations with ${this.CONFIG.warmupRuns} warmup run(s).\n`); 112 | } 113 | }; 114 | 115 | // Run the benchmark with default scripts 116 | // You can customize this by passing different script names as options 117 | SpeedTest.benchmark({ 118 | standardScript: "_standardProcessing.js", 119 | turbitScript: "_turbitProcessing.js" 120 | }); -------------------------------------------------------------------------------- /examples/README.md: -------------------------------------------------------------------------------- 1 | # Usage Examples 2 | 3 | This directory contains example scripts demonstrating the usage of Turbit for various tasks. The examples are divided into two types: extended and simple. 4 | 5 | ## Extended type 6 | 7 | The `/extended` directory contains examples that utilize Turbit extended functionality for processing arrays of data in parallel. 8 | 9 | | Script | Description | 10 | |--------|-------------| 11 | | [`basicSentimentAnalysis.js`](extended/basicSentimentAnalysis.js) | Demonstrates parallel sentiment analysis on multiple text samples. | 12 | | [`passwordStrengthChecker.js`](extended/passwordStrengthChecker.js) | Shows simultaneous strength evaluation of multiple passwords. | 13 | | [`toUpperCaseProcessing.js`](extended/toUpperCaseProcessing.js) | Showcases parallel text transformation from lowercase to uppercase. | 14 | | [`transactionRiskAnalysis.js`](extended/transactionRiskAnalysis.js) | Simulates large-scale financial transaction risk analysis using distributed computing. | 15 | 16 | ## Simple type 17 | 18 | The `/simple` directory contains examples that demonstrate the basic usage of Turbit for single-task parallel processing. 19 | 20 | | Script | Description | 21 | |--------|-------------| 22 | | [`generateRandomNumbers.js`](simple/generateRandomNumbers.js) | Illustrates parallel generation of a large number of random numbers. | 23 | | [`taskRunner.js`](simple/taskRunner.js) | Provides a basic template for running a simple task using Turbit. | 24 | 25 | ## Running the Examples 26 | 27 | To run these examples, ensure Turbit is installed in your project. Execute each script using Node.js: 28 | 29 | ```shell 30 | node examples/extended/passwordStrengthChecker.js 31 | node examples/simple/generateRandomNumbers.js 32 | node examples/extended/transactionRiskAnalysis.js 33 | ``` 34 | 35 | ## Further Exploration 36 | 37 | Feel free to modify and experiment with these examples to better understand how Turbit can be integrated into your own projects for improved performance through parallel processing. 38 | 39 | For more detailed information about Turbit and its capabilities, refer to the main [Turbit documentation](https://github.com/jofpin/turbit). -------------------------------------------------------------------------------- /examples/extended/basicSentimentAnalysis.js: -------------------------------------------------------------------------------- 1 | let Turbit; 2 | 3 | try { 4 | // Try to import the installed version of Turbit 5 | Turbit = require("turbit"); 6 | } catch (error) { 7 | // If the library is not installed, use the local version 8 | Turbit = require("../../turbit"); 9 | } 10 | 11 | // Create a Turbit instance for parallel processing 12 | const turbit = Turbit(); 13 | 14 | /** 15 | * analyzeSentiment: Analyze the sentiment of the reviews of the entered texts 16 | */ 17 | const analyzeSentiment = function(reviews) { 18 | // Arrays of positive and negative words 19 | const positiveWords = ["good", "excellent", "perfect", "cool"]; 20 | const negativeWords = ["bad", "terrible", "horrible"]; 21 | 22 | // Store sentiment score 23 | let score = 0; 24 | 25 | // Loop through each review 26 | reviews.forEach(review => { 27 | positiveWords.forEach(word => { if (review.includes(word)) score += 1; }); 28 | negativeWords.forEach(word => { if (review.includes(word)) score -= 1; }); 29 | }); 30 | 31 | if (score > 0) { 32 | return "Positive"; 33 | } else if (score < 0) { 34 | return "Negative"; 35 | } else { 36 | return "Neutral"; 37 | } 38 | } 39 | 40 | const main = async function() { 41 | const reviews = [ 42 | "This product is excellent and I love it", 43 | "horrible experience, would not recommend it", 44 | "Perfect for what I need, good and cheap", 45 | "Terrible, broke after the second use", 46 | "The Apple Vision Pro is a game-changer in the tech industry. Is a Good product.", 47 | "Elon Musk: visionary leadership has revolutionized space exploration", 48 | "Steve Jobs: Innovation and design philosophy continue to inspire generations", 49 | "Turbit is very cool." 50 | ]; 51 | 52 | try { 53 | const result = await turbit.run(analyzeSentiment, { 54 | type: "extended", 55 | data: reviews, 56 | power: 100 // Uses 100% of available cores 57 | }); 58 | // Return results 59 | console.log("Sentiment Analysis:", result.data); 60 | // View of stats 61 | console.log("Stats:", result.stats); 62 | 63 | } catch (error) { 64 | console.error("Error:", error.message); 65 | } finally { 66 | turbit.kill(); 67 | } 68 | } 69 | 70 | main(); -------------------------------------------------------------------------------- /examples/extended/passwordStrengthChecker.js: -------------------------------------------------------------------------------- 1 | const fs = require("fs"); 2 | const path = require("path"); 3 | 4 | let Turbit; 5 | 6 | try { 7 | // Try to import the installed version of Turbit 8 | Turbit = require("turbit"); 9 | } catch (error) { 10 | // If the library is not installed, use the local version 11 | Turbit = require("../../turbit"); 12 | } 13 | 14 | // Create a Turbit instance for parallel processing 15 | const turbit = Turbit(); 16 | 17 | /** 18 | * passwordGenerator: Generates a strong password of a predefined length using a set of characters. 19 | */ 20 | const passwordGenerator = function() { 21 | const chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*()_+-=[]{}|;':\",.<>/?"; 22 | const length = 22; 23 | let password = ""; 24 | for (let i = 0; i < length; i++) { 25 | const random = Math.floor(Math.random() * chars.length); 26 | password += chars[random]; 27 | } 28 | return password; 29 | } 30 | /** 31 | * checkStrength: Checks the strength of an array of passwords based on specific criteria. 32 | */ 33 | const checkStrength = function(passwords) { 34 | return passwords.map(password => { 35 | const uppercase = /[A-Z]/.test(password); 36 | const lowercase = /[a-z]/.test(password); 37 | const numbers = /\d/.test(password); 38 | const special = /[!@#$%^&*()_+\-=\[\]{};':"\\|,.<>\/?]+/.test(password); 39 | const isStrong = password.length >= 8 && uppercase && lowercase && numbers && special; 40 | return { password, isStrong }; 41 | }); 42 | } 43 | /** 44 | * saveDataPasswords: Saves the password strength check results to a CSV file. 45 | */ 46 | const saveDataPasswords = function(results, filename) { 47 | const csvLines = results.map(result => 48 | `"${result.password.replace(/"/g, '""')}",${result.isStrong}` 49 | ); 50 | csvLines.unshift("password,isStrong"); 51 | fs.writeFileSync(filename, csvLines.join("\n")); 52 | } 53 | /** 54 | * main: Generate passwords, check their strength, and save the results. 55 | */ 56 | const main = async function() { 57 | const numPasswords = 1000000; // At this moment it generates a million data, but you can modify it to your liking. 58 | const passwords = Array.from({ length: numPasswords }, passwordGenerator); 59 | 60 | try { 61 | const results = await turbit.run(checkStrength, { 62 | type: "extended", 63 | data: passwords, 64 | power: 100 // Uses 100% of available cores 65 | }); 66 | 67 | turbit.kill(); 68 | 69 | saveDataPasswords(results.data, path.join(__dirname, "passwords.csv")); 70 | 71 | console.log("") 72 | console.log("") 73 | console.log("Generated passwords:", numPasswords) 74 | console.log("-") 75 | console.log('Results saved in passwords.csv'); 76 | console.log("-") 77 | console.log("Stats:", results.stats) 78 | console.log("") 79 | console.log("") 80 | } catch (error) { 81 | console.error(`Error: ${error.message}`); 82 | } finally { 83 | turbit.kill(); 84 | } 85 | } 86 | 87 | main(); -------------------------------------------------------------------------------- /examples/extended/toUpperCaseProcessing.js: -------------------------------------------------------------------------------- 1 | let Turbit; 2 | 3 | try { 4 | // Try to import the installed version of Turbit 5 | Turbit = require("turbit"); 6 | } catch (error) { 7 | // If the library is not installed, use the local version 8 | Turbit = require("../../turbit"); 9 | } 10 | 11 | // Create a Turbit instance for parallel processing 12 | const turbit = Turbit(); 13 | 14 | /** 15 | * toUpperCase: Convert all text to uppercase. 16 | */ 17 | const toUpperCase = function(chunk) { 18 | return chunk.map(text => text.toUpperCase()); 19 | } 20 | 21 | const main = async function() { 22 | const texts = [ 23 | "hello world", 24 | "turbit is very fast", 25 | "parallel processing", 26 | "enhancing node.js performance", 27 | "easy multitasking", 28 | "scalable applications", 29 | "efficient computing" 30 | ]; 31 | try { 32 | const result = await turbit.run(toUpperCase, { 33 | type: "extended", 34 | data: texts, 35 | power: 70 // Uses 70% of available cores 36 | }); 37 | console.log("Uppercase Texts:", result.data); 38 | } catch (error) { 39 | console.error("Error during Turbit execution:", error.message); 40 | } finally { 41 | turbit.kill(); 42 | } 43 | } 44 | 45 | main(); -------------------------------------------------------------------------------- /examples/extended/transactionRiskAnalysis.js: -------------------------------------------------------------------------------- 1 | /** 2 | * Transaction Risk Analysis (Simulator) 3 | * 4 | * This script simulates a large-scale financial transaction risk analysis system. 5 | * It demonstrates the use of Turbit for distributed computing to process and analyze 6 | * a high volume of transactions efficiently. 7 | * 8 | * The system focuses on two primary risk factors: 9 | * 1. High-value transactions: Transactions exceeding a specified threshold amount. 10 | * 2. Unusual hour transactions: Transactions occurring during specified night hours. 11 | * 12 | * The analysis workflow includes: 13 | * - Generating a large set of simulated transaction data. 14 | * - Using Turbit to distribute the analysis workload across multiple processes. 15 | * - Analyzing each transaction for risk factors. 16 | * - Aggregating results and producing a summary report. 17 | * 18 | * This simulation is useful for: 19 | * - Demonstrating distributed computing concepts. 20 | * - Testing the performance of risk analysis algorithms at scale. 21 | * - Prototyping financial monitoring systems. 22 | */ 23 | 24 | let Turbit; 25 | try { 26 | Turbit = require("turbit"); 27 | } catch (error) { 28 | Turbit = require("../../turbit"); 29 | } 30 | 31 | /** 32 | * Analyzes transactions for risk based on amount and time 33 | * 34 | * This function is designed to be executed in parallel by Turbit across multiple processes. 35 | * It evaluates each transaction against predefined risk criteria and assigns risk scores. 36 | */ 37 | const analyzeRisk = function(input) { 38 | const { data, args } = input; 39 | const { riskThreshold, unusualHourStart, unusualHourEnd } = args; 40 | 41 | return data.map(transaction => { 42 | let risk = 0; 43 | let riskFactors = []; 44 | if (transaction.amount > riskThreshold) { 45 | risk++; 46 | riskFactors.push("High amount"); 47 | } 48 | if (transaction.hour >= unusualHourStart || transaction.hour < unusualHourEnd) { 49 | risk++; 50 | riskFactors.push("Unusual hour"); 51 | } 52 | return { ...transaction, risk, riskFactors, suspicious: risk > 0 }; 53 | }); 54 | } 55 | 56 | const transactionRiskAnalysis = { 57 | /** 58 | * Configuration settings for the analysis 59 | * 60 | * These settings control the simulation parameters and risk thresholds. 61 | * Adjust these values to experiment with different scenarios. 62 | */ 63 | CONFIG: { 64 | transactionCount: 1000000, // Number of transactions to simulate 65 | maxAmount: 1000000, // Maximum transaction amount 66 | riskThreshold: 500000, // Amount threshold for high-risk transactions 67 | unusualHourStart: 23, // Start of unusual hours (11 PM) 68 | unusualHourEnd: 5 // End of unusual hours (5 AM) 69 | }, 70 | /** 71 | * Generates a set of sample transactions for analysis 72 | * 73 | * This method creates a large dataset of simulated transactions. 74 | * Each transaction has a unique ID, a random amount, and a random hour of occurrence. 75 | */ 76 | generateTransactions(counter, maxAmount) { 77 | return Array.from({ length: counter }, (_, index) => ({ 78 | id: index + 1, 79 | amount: Math.random() * maxAmount, 80 | hour: Math.floor(Math.random() * 24) 81 | })); 82 | }, 83 | /** 84 | * Executes the transaction risk analysis using Turbit 85 | * 86 | * This method orchestrates the entire analysis process: 87 | * 1. Generates the sample transaction data. 88 | * 2. Initializes Turbit for distributed processing. 89 | * 3. Executes the risk analysis across multiple processes. 90 | * 4. Collects and reports the results. 91 | */ 92 | async run() { 93 | const transactions = this.generateTransactions(this.CONFIG.transactionCount, this.CONFIG.maxAmount); 94 | const turbit = Turbit(); 95 | 96 | try { 97 | const result = await turbit.run(analyzeRisk, { 98 | type: "extended", // Use Turbit's extended processing mode for large datasets 99 | data: transactions, 100 | args: this.CONFIG, 101 | power: 100 // Utilize full available processing power 102 | }); 103 | 104 | if (result && result.data) { 105 | this.reportResults(result); 106 | } else { 107 | console.error("No data received from Turbit analysis"); 108 | } 109 | } catch (error) { 110 | console.error("Error in transaction analysis:", error.message); 111 | } finally { 112 | turbit.kill(); // Ensure Turbit resources are properly released 113 | } 114 | }, 115 | /** 116 | * Generates and logs a report of the analysis results 117 | * 118 | * This method processes the results from the distributed analysis: 119 | * - Identifies suspicious transactions. 120 | * - Calculates total amounts and counts. 121 | * - Determines the top 10 most suspicious transactions. 122 | * - Formats and logs a comprehensive JSON report. 123 | */ 124 | reportResults(result) { 125 | const suspiciousTransactions = result.data.filter(transaction => transaction.suspicious); 126 | const topSuspiciousTransactions = suspiciousTransactions 127 | .sort((a, b) => b.risk - a.risk || b.amount - a.amount) 128 | .slice(0, 10); 129 | 130 | const totalAmount = result.data.reduce((sum, transaction) => sum + transaction.amount, 0); 131 | const suspiciousAmount = suspiciousTransactions.reduce((sum, transaction) => sum + transaction.amount, 0); 132 | 133 | console.log(JSON.stringify({ 134 | executionStats: result.stats, 135 | analysisResults: { 136 | overview: { 137 | totalTransactions: result.data.length, 138 | totalAmount: totalAmount.toFixed(2), 139 | suspiciousTransactions: suspiciousTransactions.length, 140 | suspiciousAmount: suspiciousAmount.toFixed(2) 141 | }, 142 | riskAnalysis: { 143 | riskThreshold: this.CONFIG.riskThreshold, 144 | unusualHours: `${this.CONFIG.unusualHourStart}:00 - ${this.CONFIG.unusualHourEnd}:00` 145 | }, 146 | top10SuspiciousTransactions: topSuspiciousTransactions.map(transaction => ({ 147 | id: transaction.id, 148 | amount: transaction.amount.toFixed(2), 149 | hour: transaction.hour, 150 | riskScore: transaction.risk, 151 | riskFactors: transaction.riskFactors 152 | })) 153 | } 154 | }, null, 2)); 155 | } 156 | }; 157 | 158 | // Execute the analysis 159 | transactionRiskAnalysis.run(); -------------------------------------------------------------------------------- /examples/simple/generateRandomNumbers.js: -------------------------------------------------------------------------------- 1 | let Turbit; 2 | 3 | try { 4 | // Try to import the installed version of Turbit 5 | Turbit = require("turbit"); 6 | } catch (error) { 7 | // If the library is not installed, use the local version 8 | Turbit = require("../../turbit"); 9 | } 10 | 11 | // Create a Turbit instance for parallel processing 12 | const turbit = Turbit(); 13 | 14 | const generateRandomNumber = function() { 15 | return Math.floor(Math.random() * 100); 16 | } 17 | 18 | const main = async function() { 19 | try { 20 | const result = await turbit.run(generateRandomNumber, { 21 | type: "simple", 22 | power: 50 // Uses 50% of the available cores. 23 | }); 24 | console.log("Random Numbers:", result.data); 25 | } catch (error) { 26 | console.error("Error:", error.message); 27 | } finally { 28 | turbit.kill(); 29 | } 30 | } 31 | 32 | main(); -------------------------------------------------------------------------------- /examples/simple/taskRunner.js: -------------------------------------------------------------------------------- 1 | let Turbit; 2 | 3 | try { 4 | // Try to import the installed version of Turbit 5 | Turbit = require("turbit"); 6 | } catch (error) { 7 | // If the library is not installed, use the local version 8 | Turbit = require("../../turbit"); 9 | } 10 | 11 | // Create a Turbit instance for parallel processing 12 | const turbit = Turbit(); 13 | 14 | const task = function() { 15 | return "Hello, humans and intelligent machines!"; 16 | } 17 | 18 | turbit.run(task, { type: "simple", power: 100 }) 19 | .then(result => { 20 | console.log("Simple execution result:", result.data); 21 | turbit.kill(); // Cleans up child processes after completing the task 22 | }) 23 | .catch(error => { 24 | console.error("Error in simple execution:", error); 25 | turbit.kill(); // Make sure to call kill even if there's an error 26 | }); -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "turbit", 3 | "version": "1.0.0", 4 | "description": "High-speed multicore computing library for optimizing intensive operations through parallel CPU processing.", 5 | "main": "turbit.js", 6 | "directories": { 7 | "images": "assets/images", 8 | "example": "examples" 9 | }, 10 | "repository": { 11 | "type": "git", 12 | "url": "git+https://github.com/jofpin/turbit.git" 13 | }, 14 | "keywords": [ 15 | "javascript", 16 | "nodejs", 17 | "cpu-booster", 18 | "multicore", 19 | "high-performance", 20 | "multiprocessing", 21 | "data-processing", 22 | "multithreading", 23 | "parallelism", 24 | "parallel-programming", 25 | "parallel-computing" 26 | ], 27 | "author": "Jose Pino", 28 | "license": "MIT", 29 | "bugs": { 30 | "url": "https://github.com/jofpin/turbit/issues" 31 | }, 32 | "homepage": "https://github.com/jofpin/turbit#readme" 33 | } -------------------------------------------------------------------------------- /turbit.js: -------------------------------------------------------------------------------- 1 | /** 2 | * turbit.js 3 | * High-speed multicore computing library for optimizing intensive operations through parallel CPU processing. 4 | * 5 | * @author Jose Pino 6 | * @contact jose@pino.sh (https://x.com/jofpin) 7 | * @version 1.0.0 8 | * @license MIT 9 | * 10 | * Find the project on GitHub: 11 | * https://github.com/jofpin/turbit 12 | * 13 | * =============================== 14 | * Copyright (c) 2024 by Jose Pino 15 | * =============================== 16 | * 17 | * Released on: August 2, 2024 18 | * Last update: August 2, 2024 19 | * 20 | */ 21 | if (typeof process === "undefined" || !process.versions || !process.versions.node) { 22 | console.error("Turbit is developed for Node.js and does not support browsers."); 23 | } else { 24 | const childProcess = require("child_process"); 25 | const os = require("os"); 26 | 27 | if (process.argv[2] === "child") { 28 | process.on("message", async (message) => { 29 | try { 30 | const func = new Function("return " + message.func)(); 31 | let result; 32 | if (Array.isArray(message.args)) { 33 | result = func(...message.args); 34 | } else { 35 | result = func(message.args); 36 | } 37 | process.send({ result }); 38 | } catch (error) { 39 | process.send({ error: error.message }); 40 | } 41 | }); 42 | } else { 43 | /** 44 | * Creates a Turbit instance for high-speed multicore computing. 45 | */ 46 | const Turbit = () => { 47 | const MAX_PROCESSES = os.cpus().length; 48 | let CHILD_PROCESSES = []; 49 | /** 50 | * core: Manages child processes and task distribution for parallel execution. 51 | */ 52 | const core = { 53 | startProcesses: function (numProcesses = MAX_PROCESSES) { 54 | for (let i = CHILD_PROCESSES.length; i < numProcesses; i++) { 55 | try { 56 | const newChildProcess = childProcess.fork(__filename, ["child"]); 57 | CHILD_PROCESSES.push(newChildProcess); 58 | } catch (error) { 59 | console.log(`Error: Maintaining current level of child processes due to resource limitation - ${error.message}`); 60 | break; 61 | } 62 | } 63 | }, 64 | killProcesses: function () { 65 | CHILD_PROCESSES.forEach(worker => { 66 | if (!worker.killed) { 67 | worker.kill(); 68 | worker.removeAllListeners(); 69 | } 70 | }); 71 | 72 | CHILD_PROCESSES = []; 73 | }, 74 | createWorkerPromises: function (tasks, numProcesses) { 75 | return tasks.map((task, index) => { 76 | const worker = CHILD_PROCESSES[index % numProcesses]; 77 | return new Promise((resolve, reject) => { 78 | worker.send({ 79 | func: task.func.toString(), 80 | args: task.args || [] 81 | }); 82 | worker.once("message", (message) => { 83 | if (message.error) { 84 | reject(new Error(message.error)); 85 | } else { 86 | resolve(message.result); 87 | } 88 | }); 89 | }); 90 | }); 91 | }, 92 | calculateNumProcesses: function (power) { 93 | const percentage = Math.max(power, 0) / 100; 94 | return Math.max(Math.round(MAX_PROCESSES * percentage), 1); 95 | } 96 | }; 97 | /** 98 | * helper: Provides utility functions for system metrics, data formatting, and execution statistics. 99 | */ 100 | const helper = { 101 | getCurrentCpuLoad: function () { 102 | return os.loadavg()[0]; 103 | }, 104 | getCurrentMemoryUsage: function () { 105 | const totalMemory = os.totalmem(); 106 | const freeMemory = os.freemem(); 107 | return ((totalMemory - freeMemory) / totalMemory) * 100; 108 | }, 109 | // calculateBytes: Function to calculate the weight of the bytes, extracted from the temcrypt project: https://github.com/jofpin/temcrypt/blob/main/temcrypt.js#L223 110 | calculateBytes: function (bytes) { 111 | const units = ["Bytes", "KB", "MB", "GB", "TB"]; 112 | const kbytes = 1024; 113 | 114 | if (bytes === 0) { 115 | return "0 Bytes"; 116 | } 117 | 118 | const index = Math.floor(Math.log(bytes) / Math.log(kbytes)); 119 | const convertBytes = bytes / Math.pow(kbytes, index); 120 | const calculateBytes = convertBytes % 1 === 0 ? convertBytes.toFixed(0) : convertBytes.toFixed(2); 121 | 122 | return `${calculateBytes} ${units[index]}`; 123 | }, 124 | // showStats: You can see performance statistics and the result of consumed resources. 125 | showStats: function (startTime, numProcesses, dataLength, initialMemory) { 126 | const endTime = Date.now(); 127 | const duration = (endTime - startTime) / 1000; 128 | const totalRAMAfter = os.freemem(); 129 | const memoryUsed = initialMemory - totalRAMAfter; 130 | return { 131 | timeTakenSeconds: duration, 132 | numProcessesUsed: numProcesses, 133 | dataProcessed: dataLength, 134 | memoryUsed: this.calculateBytes(memoryUsed) 135 | }; 136 | } 137 | }; 138 | /** 139 | * types: Defines execution strategies (simple and extended) for tasks, leveraging parallel processing capabilities. 140 | */ 141 | const types = { 142 | // simpleType: Executes a task using a simple execution strategy, without parallel processing. 143 | simpleType: async function (func, numProcesses) { 144 | const initialMemory = os.freemem(); 145 | const startTime = Date.now(); 146 | 147 | const tasks = Array(numProcesses).fill({ 148 | func, 149 | args: [] 150 | }); 151 | 152 | const promises = core.createWorkerPromises(tasks, numProcesses); 153 | const output = await Promise.all(promises); 154 | return { 155 | data: output, 156 | stats: helper.showStats(startTime, numProcesses, output.length, initialMemory) 157 | }; 158 | }, 159 | // extendedType: Executes tasks using an extended execution strategy, with parallel processing across multiple CPU cores. 160 | extendedType: async function (func, data, numProcesses, args) { 161 | const initialMemory = os.freemem(); 162 | const startTime = Date.now(); 163 | 164 | const chunkSize = Math.ceil(data.length / numProcesses); 165 | const dataChunks = []; 166 | for (let i = 0; i < data.length; i += chunkSize) { 167 | dataChunks.push(data.slice(i, i + chunkSize)); 168 | } 169 | 170 | const tasks = dataChunks.map(chunk => ({ 171 | func, 172 | args: Object.keys(args).length ? { data: chunk, args } : [chunk] 173 | })); 174 | 175 | const promises = core.createWorkerPromises(tasks, numProcesses); 176 | const output = await Promise.all(promises); 177 | return { 178 | data: output.flat(), 179 | stats: helper.showStats(startTime, numProcesses, data.length, initialMemory) 180 | }; 181 | } 182 | }; 183 | /** 184 | * Initializes child processes up to the maximum available CPU cores. This step is essential for setting up the parallel processing environment before any tasks are submitted for execution. Invoking this method at the start ensures that the system is ready to distribute tasks across multiple processes, optimizing performance and resource utilization from the outset. 185 | */ 186 | core.startProcesses(); 187 | 188 | return { 189 | /** 190 | * Executes a given function across multiple cores for parallel processing, with the level of parallelism based on the specified execution type and power percentage. 191 | * 192 | * Before using `run`, you must import and initialize `Turbit` as shown below: 193 | * @example 194 | * const Turbit = require('turbit'); // Adjust the path as needed if `turbit` is located in a different directory 195 | * const turbit = Turbit(); // Initialize Turbit to use its methods 196 | * 197 | * @param {Function} func - The function to be executed. This function should be capable of operating on the provided data, if applicable. 198 | * @param {Object} options - The options for execution. 199 | * @param {string} [options.type="simple"] - The type of execution: "simple" for execution with a default level of parallel processing, or "extended" for customized parallel processing across multiple CPU cores. 200 | * @param {Array} [options.data=[]] - The data to be processed. Required for the "extended" type execution to distribute data across processes. 201 | * @param {Array} [options.args={}] - The arguments to be passed to the function. Optional for the "extended" type execution to provide extra arguments to the function. 202 | * @param {number} [options.power=50] - The processing power to use, as a percentage of total available CPU cores. Determines the number of child processes spawned for both "simple" and "extended" type execution. 203 | * @returns {Promise} - The result of the execution, including any data processed and statistics about the execution, such as time taken and memory used. 204 | * 205 | * - For "simple" type: 206 | * - Executes the function across multiple processes, using a default level of parallel processing determined by the `power` option. 207 | * - The `data` option is ignored for this type, assuming the function does not require input data or processes static data. 208 | * 209 | * - For "extended" type: 210 | * - Distributes the provided `data` across multiple processes for parallel processing, with the degree of parallelism customized through the `power` option. 211 | * - The `data` option must be an array of items, which will be processed in chunks across the spawned processes. 212 | * 213 | * 214 | * @example 215 | * // Example usage for simple execution with parallel processing 216 | * function simpleTask() { 217 | * // This example function performs a simple operation 218 | * return "Simple task completed"; 219 | * } 220 | * 221 | * 222 | * turbit.run(simpleTask, { type: "simple", power: 100 }) 223 | * .then(result => console.log("Simple execution result:", result)) 224 | * .catch(error => console.error("Error in simple execution:", error)); 225 | * 226 | * // Example usage for extended execution with customized parallel processing 227 | * function exampleFunction(item) { 228 | * // This example function doubles the input 229 | * return item * 2; 230 | * } 231 | * 232 | * 233 | * turbit.run(exampleFunction, { 234 | * type: "extended", 235 | * data: [1, 2, 3, 4], // Data to be processed in parallel 236 | * power: 75 // Use 75% of available CPU cores for enhanced parallel processing 237 | * }) 238 | * .then(result => console.log("Extended execution results:", result.data)) 239 | * .catch(error => console.error("Error in extended execution:", error)); 240 | * 241 | * @throws {Error} If the `func` is not a function or if required parameters for the chosen execution type are not provided or are invalid. 242 | */ 243 | run: async (func, options = {}) => { 244 | let { data = [], args = {}, type = "simple", power = 70 } = options; 245 | let numProcesses = core.calculateNumProcesses(power); 246 | 247 | if (numProcesses > CHILD_PROCESSES.length) { 248 | core.killProcesses(); 249 | core.startProcesses(numProcesses); 250 | } 251 | 252 | const handlers = { 253 | "simple": async () => { 254 | if (data.length > 0) { 255 | throw new Error("Simple execution type should not include 'data'. Please ensure 'data' is empty or not provided for simple tasks."); 256 | } 257 | if (typeof func !== "function") { 258 | throw new Error("For 'simple' execution type, 'func' must be a valid function. Please check that 'func' is defined correctly as a function."); 259 | } 260 | try { 261 | return await types.simpleType(func, numProcesses); 262 | } catch (e) { 263 | throw new Error(`Error executing 'simple' type with function ${func.name || 'anonymous'}: ${e.message}`); 264 | } 265 | }, 266 | "extended": async () => { 267 | if (data.length === 0) { 268 | throw new Error("Extended execution type requires a non-empty 'data' array. Ensure 'data' is provided and contains elements."); 269 | } 270 | if (typeof func !== "function") { 271 | throw new Error("For 'extended' execution type, 'func' must be a valid function. Verify that 'func' is correctly defined as a function."); 272 | } 273 | try { 274 | return await types.extendedType(func, data, numProcesses, args); 275 | } catch (e) { 276 | throw new Error(`Error executing 'extended' type with function ${func.name || 'anonymous'}: ${e.message}`); 277 | } 278 | } 279 | }; 280 | 281 | if (!handlers[type]) { 282 | throw new Error(`Invalid execution type specified: '${type}'. Valid types are 'simple' and 'extended'.`); 283 | } 284 | 285 | try { 286 | return await handlers[type](); 287 | } catch (error) { 288 | console.error("Error during Turbit execution:", error.message); 289 | } 290 | }, 291 | /** 292 | * kill: Terminates all active child processes to ensure a clean shutdown and free system resources. This method is crucial for preventing resource leaks and ensuring that the system remains stable and responsive after the completion of parallel tasks. It should be invoked when all parallel processing tasks are completed, or when the Turbit instance is no longer needed. 293 | */ 294 | kill: core.killProcesses 295 | }; 296 | }; 297 | 298 | module.exports = Turbit; 299 | } 300 | } --------------------------------------------------------------------------------