├── .gitignore ├── LICENSE ├── README.md ├── archive └── README.md ├── designs ├── admin-api-authentication.md ├── adminAuth-user-management.md ├── atomic-context-api │ └── atomic-context-api.md ├── dashboard-layout-tool │ ├── Layout_tool_overview.png │ ├── README.md │ └── dashboard_sidebar.png ├── delay-node-enhancements │ ├── delay-node-enhancements.md │ └── nodepropertyui.png ├── dynamic-mqtt-node.md ├── env-vars │ ├── EnvVar4TabAndGroup.md │ ├── README.md │ ├── env-var-lookup.png │ ├── env-var-ui-for-group.png │ ├── nr-envvar-projsettings.png │ ├── nr-envvar-subflows.png │ └── nr-envvar-typedinput.png ├── exportable-subflow │ ├── README.md │ ├── Subflow-delete-UI.png │ ├── Subflow-export-UI.png │ ├── Subflow-import-UI.png │ └── Unknown-node.png ├── flow-linter │ ├── README.md │ ├── code-generation.svg │ ├── external-cli.svg │ ├── fmapi.svg │ ├── func-diagram.svg │ ├── on-the-fly.png │ └── sidebar.png ├── flow-manipulation-api │ ├── README.md │ ├── api-overview.png │ ├── case-debugnode.png │ ├── case-generation.png │ ├── case-multiruntime.png │ ├── case-otherruntime.png │ └── fmapi-model.png ├── flow-outliner │ ├── README.md │ └── images │ │ ├── blender-outliner.png │ │ ├── sidebar-mock1.png │ │ └── sidebar-mock2.png ├── flow-testing │ ├── README.md │ ├── images │ │ ├── 20200131whiteboard.png │ │ ├── FlowTestingModel.png │ │ ├── flowtesting_sidebar.png │ │ ├── production_flow.png │ │ ├── seup-cleanup.png │ │ ├── test-case.png │ │ ├── test-in_node_properties.png │ │ ├── test-out_node_properties.png │ │ ├── test-settings.png │ │ ├── test-sidebar.png │ │ └── testing_flow.png │ └── previous_idea.md ├── function-library-node.md ├── function-node-lifecycle │ ├── README.md │ ├── config.png │ ├── fn-lifecycle.png │ ├── init-fin.png │ ├── lifecycle.pptx │ ├── modules-tab.png │ └── require-interaction.png ├── groups │ ├── README.md │ └── images │ │ ├── groups-uc1.png │ │ └── groups-uc2.png ├── node-installation │ ├── README.md │ ├── add-nodes.png │ ├── install-from-url.md │ └── upload-tgz.md ├── node-messaging-api.md ├── node-property-typing.md ├── overwrite-settings.md ├── pluggable-library.md ├── pluggable-message-routing │ ├── README.md │ └── message-router-events.png ├── plugins.md ├── pouchdb-context-plugin.md ├── runnable-projects.md ├── subflow-node-modules.md ├── subflow-property-ui │ ├── README.md │ └── images │ │ ├── Subflow-dev-process.png │ │ ├── UI-app-any.png │ │ ├── UI-app-checkbox.png │ │ ├── UI-app-label.png │ │ ├── UI-app-menu.png │ │ ├── UI-app-node.png │ │ ├── UI-app-simple.png │ │ ├── UI-app-spinner.png │ │ ├── UI-def-any0.png │ │ ├── UI-def-checkbox.png │ │ ├── UI-def-label.png │ │ ├── UI-def-menu.png │ │ ├── UI-def-node.png │ │ ├── UI-def-simple.png │ │ ├── UI-def-spinner.png │ │ ├── UI-edit-panel.png │ │ ├── UI-element01.png │ │ ├── field-label.png │ │ ├── field-target-envvar.png │ │ ├── field-target-target.png │ │ ├── field-tgt.png │ │ └── field-type.png └── timeout-api.md └── proposal-template.md /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Node-RED Project Designs 2 | 3 | This repository is used to track proposals for new features within Node-RED. It 4 | covers both the core Node-RED and any of the assets the project maintains. 5 | 6 | The full contribution process for the project is documented [here](https://nodered.org/about/contribute/). 7 | 8 | All feature requests or ideas **must** be discussed on the [Node-RED forum](https://discourse.nodered.org) 9 | or [Slack](https://nodered.org/slack) to check it fits with the project's plans. 10 | 11 | If the discussion results in an agreed plan to create a design note, a pull 12 | request should be opened in this repository to add a new design note to the `designs` 13 | folder. 14 | 15 | If the design requires images, create a subfolder for the proposal. 16 | 17 | The design note *must* use the [proposal-template](proposal-template.md) provided. 18 | 19 | ### Design proposals 20 | 21 | #### Draft 22 | 23 | All design proposals start in the `draft` state. This means they are still under 24 | discussion and review. 25 | 26 | At this stage, the proposal should set out the high-level goals of the feature 27 | with enough detail to review the intent and direction of the feature. 28 | 29 | - [adminAuth User Management](designs/adminAuth-user-management.md) 30 | - [Function Library Node](designs/function-library-node.md) 31 | - [Subflow Node Modules](designs/subflow-node-modules.md) 32 | - [Runnable Projects](designs/runnable-projects.md) 33 | - [Dynamic MQTT Node](designs/dynamic-mqtt-node.md) 34 | - [Exportable Subflow](designs/exportable-subflow/README.md) 35 | - [Node Timeout API](designs/timeout-api.md) 36 | - [Overwrite Values in settings.js](designs/overwrite-settings.md) 37 | - [PouchDB Context store plugin](designs/pouchdb-context-plugin.md) 38 | 39 | #### In-progress 40 | 41 | Through consensus, the design can moved to the `in-progress` state. This is 42 | for designs that are actively being worked on beyond the high-level detail. 43 | 44 | - [Subflow Property UI](designs/subflow-property-ui) 45 | - [Dashboard Layout Tool](designs/dashboard-layout-tool) 46 | - [Flow Manipulation API](designs/flow-manipulation-api) 47 | - [Function Node Lifecycle Model](designs/function-node-lifecycle/README.md) 48 | 49 | #### Complete 50 | 51 | Once a design has been implemented it should be updated to the `complete` state. 52 | The design should be updated to include references to where it has been implemented - 53 | such as the release it is included in. 54 | 55 | - [Node Messaging API](designs/node-messaging-api.md) 56 | - [Environment Variables](designs/env-vars) 57 | - [Admin API Authentication](designs/admin-api-authentication.md) 58 | -------------------------------------------------------------------------------- /archive/README.md: -------------------------------------------------------------------------------- 1 | # Archived Designs 2 | 3 | Designs in this folder have been archived without having been implemented. This 4 | could be because it has been decided to not proceed with the idea, or an alternative 5 | design has superseded it. 6 | -------------------------------------------------------------------------------- /designs/admin-api-authentication.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Admin API Authentication function 6 | 7 | ## Summary 8 | The authentication function of Node-RED supports the 'credentials' scheme and the 'strategy' scheme, but there is a problem that the Admin API can not be used in the 'strategy' scheme . 9 | 10 | This design note proposes a feature that enables the execution of the Admin API even in the 'strategy' scheme by adding a customizable token-based authentication function. 11 | 12 | ## Authors 13 | 14 | - @KazuhiroIto 15 | 16 | ## Details 17 | 18 | ### Current behavior 19 | - `adminAuth` defines the authentication scheme used by the Admin API. 20 | - The authentication scheme results in a token being generated. 21 | - This token is then included in future http requests to authenticate the request. 22 | - This authentication is done by comparing the token to the list of known tokens that Node-RED maintains. 23 | - For the 'credentials' scheme, tokens can be generated by doing an HTTP POST to `/auth/token`. 24 | - For the "strategy" scheme, this is not possible. Because the OAuth strategy requires interaction with the browser, the Admin API cannot be executed. 25 | 26 | ### Adding a customizable token-based authentication function 27 | Add a new option `tokens` and `tokenHeader` to `adminAuth`. 28 | 29 | The `tokens` options is a user-defined function process that has a 'token' as an argument, and can verify the provided token. 30 | This function is called if the token is not recognized from Node-RED's internal token store. 31 | 32 | The token is obtained from the existing Authorization header. 33 | If you want to get the token from any custom header, use the `tokenHeader` option. 34 | You can obtain a token from the specified header name instead of the authorization header by specifying the header name in `tokenHeader` option. 35 | 36 | #### setting.js example: 37 | ```javascript 38 | adminAuth: { 39 | type: 'strategy', 40 | strategy : { ... }, 41 | users: function(username) { ... }, 42 | tokens: function(token) { 43 | return new Promise(function(resolve, reject) { 44 | // Do whatever work is needed to check token is valid 45 | if (valid) { 46 | // Resolve with the user object. It must contain 47 | // properties 'username' and 'permissions' 48 | var user = { username: 'admin', permissions: '*' }; 49 | resolve(user); 50 | } else { 51 | // Resolve with null as this user does not exist 52 | resolve(null); 53 | } 54 | }); 55 | }, 56 | // Change the value to be set to the argument 'token' to 57 | // the specified header name. 58 | // If not specified, get from Authorization header. 59 | tokenHeader: '' 60 | } 61 | ``` 62 | #### curl example: 63 | ```console 64 | curl -H ': ' http://localhost:1880/settings 65 | ``` 66 | 67 | ## History 68 | 69 | - 2020-02-21 - Initial proposal submitted 70 | -------------------------------------------------------------------------------- /designs/adminAuth-user-management.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # adminAuth User Management 6 | 7 | ## Summary 8 | 9 | We need to make it much easier to secure Node-RED for first-time users. Today, 10 | a user must hand-edit their settings file to manage the users. 11 | 12 | This design note considers proposals for how to achieve this. 13 | 14 | This is a very early stage proposal - it needs a lot more work to turn into a 15 | proper proposal. 16 | 17 | 18 | ## Authors 19 | 20 | - @knolleary 21 | 22 | ## Details 23 | 24 | We could provide an `adminAuth` implementation that is trivial to enable, which 25 | uses an external file to maintain user information in. Once it is in a known 26 | external file, it could become writable by the runtime - allowing for some 27 | level of user-management UX in the editor. 28 | 29 | This would be a feature than can be turned on/off for the OEM users who don't 30 | want this feature. 31 | 32 | It could also be possible to manage the users from the command-line. There are a 33 | couple possible approaches: 34 | 35 | - What if the `node-red` command did more than just run node-red. With the right 36 | set of arguments to could be used as a cli tool to manage users. 37 | 38 | - `node-red-admin` already exists as a remote client for the admin api. If we 39 | were planning to add elements in the UI, they must come with additional admin 40 | api end points - so `node-red-admin` could also be used here. However, no-one 41 | installs `node-red-admin`. What if `node-red-admin` was installed as a dependency 42 | of node-red? 43 | 44 | ## History 45 | 46 | - 2019-02-26 - migrated from Design note wiki 47 | -------------------------------------------------------------------------------- /designs/atomic-context-api/atomic-context-api.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Atomic Context API 6 | 7 | ## Summary 8 | 9 | To avoid concurrency headaches, we should add some utility functions to the Context api to allow atomic updates to properties. 10 | This is important for when you want to have multiple instances running and interacting with the same context properties. 11 | 12 | ## Authors 13 | 14 | - @HirokiUchikawa 15 | 16 | ## Details 17 | 18 | ### Use Cases 19 | 20 | 1. Multiple Node-RED instances share context without causing data inconsistency or conflicts. 21 | When multiple instances share a context and try to update it with the current API, data conflicts may occur. 22 | For example, two instances share a context and use it as a counter. When they increment it at the same time, operations may interleave as following. 23 | 24 | | Time | value of `count` in store | Instance A | Instance B | 25 | |:----:|:-------------------------:|-----------------------------|-----------------------------| 26 | | 1 | 0 | var c = global.get("count") | | 27 | | 2 | 0 | c = c + 1 | var c = global.get("count") | 28 | | 3 | 1 | global.set("count", c) | c = c + 1 | 29 | | 4 | 1 | | global.set("count", c) | 30 | | 5 | 1 | | | 31 | 32 | Each instance increments the counter, so the expected value of 'count' is 2, but the result is 1. 33 | 34 | ### Atomic Context Store API 35 | 36 | These proposed API will be added to the [Context Store API](https://nodered.org/docs/api/context/methods/). 37 | The context plugin (e.g. memory, localfilesysytem) should create that implements these API. 38 | 39 | #### incr 40 | 41 | - ContextStore.incr(scope, key, [amount], [callback]) 42 | 43 | | Name | Type | Description | 44 | | --------- | :--------: | ---------------------------------------------------------------------- | 45 | | scope | `string` | the scope of the key | 46 | | key | `string` | the key to increment the value for. | 47 | | amount | `number` | _optional_ the amount to increment by. Default : `1` | 48 | | callback | `function` | _optional_ a callback function to invoke when the value is incremented | 49 | 50 | ##### Description 51 | 52 | This API increments the value by one or passed `amount`. Pass a negative value to `amount` to decrement the value of the key. 53 | If the key does not exist, it is set to 0 before performing the operation. 54 | 55 | If no callback is provided, and the store supports synchronous access, the `incr` function should return the value after the increment. If the store does not support synchronous access it should throw an error. 56 | ```javascript 57 | incr(scope, key, amount) // return the value after the increment. 58 | ``` 59 | 60 | If callback argument is provided, it must be a function that takes two arguments: 61 | ```javascript 62 | incr(scope, key, amount, (error, value) => { 63 | // If an error occurred in process, pass the error object as first argument. 64 | // If the process was succeed, pass null as first argument and the incremented value as second argument. 65 | }); 66 | ``` 67 | 68 | if the value of the key or `amount` argument is not a number, throw an error. 69 | 70 | ##### API of the Function Node 71 | 72 | Not only `incr` but also `decr` will be added to the API of Function node. 73 | This makes it easy for the user to see what the code is doing. 74 | Internally, `context.decr` calls `ContextStore.incr` and passes negated `amount`. (i.e. `-amount`) 75 | 76 | 77 |
Uasge in the Function Node 78 | 79 | ```javascript 80 | global.incr("count"); 81 | // Increase the value of `count` by 1 in `default` store. 82 | // And return the value after increment. 83 | 84 | global.decr("count"); 85 | // Decrease the value of `count` by 1 in `default` store. 86 | // And return the value after decrement. 87 | 88 | global.incr("count", 5); 89 | // Increase the value of `count` by 5 in `default` store. 90 | // And return the value after increment. 91 | 92 | global.incr("count", "file"); 93 | // Increase the value of `count` by 1 in `file` store. 94 | // And return the value after increment. 95 | 96 | global.incr("count", 5, "file"); 97 | // Increase the value of `count` by 5 in `file` store. 98 | // And return the value after increment. 99 | 100 | global.incr("count", (err, value) => { 101 | // Increase the value of `count` by 1 in `default` store. 102 | // Then invoke callback and pass an error object or the value after increment. 103 | }); 104 | 105 | global.incr("count", 5, (err, value) => { 106 | // Increase the value of `count` by 5 in `default` store. 107 | // Then invoke callback and pass an error object or the value after increment. 108 | }); 109 | 110 | global.incr("count", "file", (err, value) => { 111 | // Increase the value of `count` by 1 in `file` store. 112 | // Then invoke callback and pass an error object or the value after increment. 113 | }); 114 | 115 | global.incr("count", 5, "file", (err, value) => { 116 | // Increase the value of `count` by 5 in `file` store. 117 | // Then invoke callback and pass an error object or the value after increment. 118 | }); 119 | ``` 120 | 121 |
122 | 123 | ##### Uasge in Other Nodes 124 | 125 | TBD 126 | 127 | It would be useful that some nodes (e.g. Change Node) can call atomic api. 128 | Currently change node can incr/decr with JSONata but it is not atomic. 129 | 130 | For example: 131 | ```json 132 | [{"id":"cd1ed2c0.4f3e7","type":"change","z":"92e079a4.8d9f78","name":"Increment global.count","rules":[{"t":"set","p":"count","pt":"global","to":"$globalContext(\"count\") ? $globalContext(\"count\") + 1 : 1","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":420,"y":60,"wires":[[]]},{"id":"af7472b9.69e3f","type":"inject","z":"92e079a4.8d9f78","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":220,"y":60,"wires":[["cd1ed2c0.4f3e7"]]}] 133 | ``` 134 | 135 | ### Next Steps 136 | 137 | * Add atomic API for Array 138 | This is useful for representing queue and stack with array. 139 | * Add atomic API for String 140 | 141 | ### Atomic API provided by NoSQL databases 142 | 143 | This proposal refers to atomic api provided by popular NoSQL databases. 144 | 145 | 146 | 147 | 148 | 149 | 150 | 151 | 152 | 153 | 154 | 155 | 156 | 157 | 158 | 159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | 170 | 176 | 181 | 182 | 186 | 187 | 188 | 194 | 200 | 205 | 206 | 207 | 208 | 212 | 216 | 220 | 221 | 225 | 229 | 230 | 231 | 232 | 233 | 234 | 235 | 236 | 240 | 244 | 245 | 249 | 250 | 251 | 252 | 256 | 262 | 263 | 264 | 265 | 266 | 270 | 274 | 275 | 276 | 280 | 284 | 285 | 290 | 294 | 295 | 296 | 297 | 298 | 299 | 303 | 307 | 311 | 312 | 313 | 314 | 315 | 320 | 324 | 325 | 326 | 327 | 328 | 329 | 333 | 337 | 338 | 339 | 343 | 344 | 345 | 346 | 347 | 348 | 349 |
#NameNumber OperationsString OperationsArray Operations
Increment the numberDecrement the numberOthersAppend data to the stringPrepend data to the stringOthersAdd element to the arrayRemove element from the arrayOthers
1 167 | 168 | [Redis](https://redis.io/) 169 | 171 | 172 | [INCR](https://redis.io/commands/incr) 173 | [INCRBY](https://redis.io/commands/incrby) 174 | [INCRBYFLOAT](https://redis.io/commands/incrbyfloat) 175 | 177 | 178 | [DECR](https://redis.io/commands/decr) 179 | [DECRBY](https://redis.io/commands/decrby) 180 | - 183 | 184 | [APPEND](https://redis.io/commands/append) 185 | -- 189 | 190 | [LPUSH](https://redis.io/commands/lpush) 191 | [RPUSH](https://redis.io/commands/rpush) 192 | [LINSERT](https://redis.io/commands/linsert) 193 | 195 | 196 | [LPOP](https://redis.io/commands/lpop) 197 | [RPOP](https://redis.io/commands/rpop) 198 | [LREM](https://redis.io/commands/lrem) 199 | 201 | 202 | [RPOPLPUSH](https://redis.io/commands/rpoplpush) 203 | [All list commnads...](https://redis.io/commands#list) 204 |
2 209 | 210 | [memcached](https://memcached.org/) 211 | 213 | 214 | [incr](https://github.com/memcached/memcached/wiki/Commands#incrdecr) 215 | 217 | 218 | [decr](https://github.com/memcached/memcached/wiki/Commands#incrdecr) 219 | - 222 | 223 | [append](https://github.com/memcached/memcached/wiki/Commands#append) 224 | 226 | 227 | [prepend](https://github.com/memcached/memcached/wiki/Commands#prepend) 228 | ----
3 237 | 238 | [MongoDB](https://www.mongodb.com/) 239 | 241 | 242 | [$inc](https://docs.mongodb.com/manual/reference/operator/update/inc/) 243 | Pass negative value to `$inc` 246 | 247 | [$mul](https://docs.mongodb.com/manual/reference/operator/update/mul/) 248 | --- 253 | 254 | [$push](https://docs.mongodb.com/manual/reference/operator/update/push/) 255 | 257 | 258 | [$pop](https://docs.mongodb.com/manual/reference/operator/update/pop/) 259 | [$pull](https://docs.mongodb.com/manual/reference/operator/update/pull/) 260 | [$pullall](https://docs.mongodb.com/manual/reference/operator/update/pullall/) 261 | -
4 267 | 268 | [Couchbase](https://www.couchbase.com/) 269 | 271 | 272 | [Bucket#counter](http://docs.couchbase.com/sdk-api/couchbase-node-client-2.6.1/Bucket.html#counter__anchor) 273 | Pass negative value to `Bucket#counter`- 277 | 278 | [Bucket#append](http://docs.couchbase.com/sdk-api/couchbase-node-client-2.6.1/Bucket.html#append__anchor) 279 | 281 | 282 | [Bucket#prepend](http://docs.couchbase.com/sdk-api/couchbase-node-client-2.6.1/Bucket.html#prepend__anchor) 283 | - 286 | 287 | [Bucket#listAppend](http://docs.couchbase.com/sdk-api/couchbase-node-client-2.6.1/Bucket.html#listAppend__anchor) 288 | [Bucket#listPrepend](http://docs.couchbase.com/sdk-api/couchbase-node-client-2.6.1/Bucket.html#listPrepend__anchor) 289 | 291 | 292 | [Bucket#listRemove](http://docs.couchbase.com/sdk-api/couchbase-node-client-2.6.1/Bucket.html#listRemove__anchor) 293 | -
5 300 | 301 | [Cassandra](https://cassandra.apache.org/) 302 | 304 | 305 | [UPDATE table_name SET column_name = column_name + value](https://cassandra.apache.org/doc/latest/cql/dml.html#update-statement) 306 | 308 | 309 | [UPDATE table_name SET column_name = column_name - value](https://cassandra.apache.org/doc/latest/cql/dml.html#update-statement) 310 | ---- 316 | 317 | [UPDATE table_name SET column_name = \[ value (, value)* \] + column_name](https://cassandra.apache.org/doc/latest/cql/types.html#lists) 318 | [UPDATE table_name SET column_name = column_name + \[ value (, value)* \]](https://cassandra.apache.org/doc/latest/cql/types.html#lists) 319 | 321 | 322 | [DELETE column_name\[index\] FROM table_name WHERE where_clause](https://cassandra.apache.org/doc/latest/cql/types.html#lists) 323 | -
6 330 | 331 | [HBase](https://hbase.apache.org/) 332 | 334 | 335 | [Increment](https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Increment.html) 336 | Pass negative value to `Increment`- 340 | 341 | [Append](https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Append.html) 342 | -----
350 | 351 | - Some of the above databases support transaction. So they can perform above operations by set of operations. 352 | However, To focus on only single operation here, transaction operations are excluded. 353 | - memcached and HBase do not support Array(or List) data type. 354 | 355 | ## Related Links 356 | 357 | - https://trello.com/c/lwMQX5kn/91-add-atomic-actions-to-context-api 358 | - https://github.com/node-red/node-red/wiki/Design%3A-Persistable-Context 359 | 360 | ## History 361 | 362 | - 2019-03-27 - Initial proposal submitted -------------------------------------------------------------------------------- /designs/dashboard-layout-tool/Layout_tool_overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/dashboard-layout-tool/Layout_tool_overview.png -------------------------------------------------------------------------------- /designs/dashboard-layout-tool/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: in-progress 3 | --- 4 | 5 | # Dashboard Layout Tool 6 | 7 | ## Summary 8 | User cannot visually design a layout of dashboard widgets. This layout tool 9 | will allow a user to deploy dashboard widgets by drag & drop. 10 | 11 | ## Authors 12 | 13 | - tbd 14 | 15 | ## Details 16 | 17 | There are several requirements that this layout tool should satisfy. 18 | 19 | 1. Change the order of widgets in a group by drag & drop. 20 | 2. Change the size of a widget. 21 | 3. Pad `ui_spacer` widgets. 22 | 4. Move a widget from one group to the another. 23 | 5. (optional) Change the size of a group. 24 | 25 | ### User interface 26 | To show the layout tool of dashboard, Add `layout` button on the tab menu. 27 | 28 | ![dashboard sidebar](dashboard_sidebar.png) 29 | 30 | When a user clicks the `layout` button, the window of a layout tool will appear. 31 | It should look like the maximized window of function node. 32 | 33 | After a user adjusts the layout on the window: 34 | - If the user clicks `Done` button: 35 | - inserts `ui_spacer` widget if necessary. 36 | - updates the layout of sidebar and workspace. 37 | - If the user clicks `Cancel` button, the change will be discarded. 38 | 39 | 40 | ![Layout tool](Layout_tool_overview.png) 41 | 42 | 43 | ### Future discussion 44 | 45 | - Add a node on the layout tool. 46 | - Add/remove a group. 47 | 48 | 49 | ## History 50 | 51 | - 2019-02-27 - migrated from Design note wiki 52 | -------------------------------------------------------------------------------- /designs/dashboard-layout-tool/dashboard_sidebar.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/dashboard-layout-tool/dashboard_sidebar.png -------------------------------------------------------------------------------- /designs/delay-node-enhancements/delay-node-enhancements.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # delay node enhancements 6 | ## Summary 7 | The enhancements for delay node change message queue handling and provide logging function. 8 | 9 | ## Author 10 | @kazuhitoyokoi 11 | 12 | ## Details 13 | ### Use cases 14 | #### 1. Incoming message control 15 | Because Node-RED accepts all incoming message from input nodes like mqtt-in node, Node.js process sometimes reaches the upper limit of memory usage. 16 | To avoid the risk of message queue overflow, rate limit functionality in delay node will be useful but Node-RED users cannot customize queue handling. 17 | For example, the current delay node cannot simultaneously use the drop option and queue to process burst incoming messages. 18 | 19 | #### 2. Logging and error handling 20 | Because current drop option in rate limit mode doesn't record dropped messages, Node-RED users aren't aware of the lost messages. 21 | Therefore, it will be useful for Node-RED users to track background behaviors if the delay node outputs the number of dropped messages to the log stream. 22 | Additionally, Node-RED users can add their error handling in their flow if the delay node throws the dropped messages to the catch node when the queue reaches the defined limit. 23 | 24 | #### 3. Changing properties on demand 25 | Currently, the delay node supports msg.delay to change property value on demand. 26 | If the delay node additionally supports msg.rate to set rate value and msg.queueLength to change queue size, users can create custom delay node using subflow to wrap delay node. 27 | 28 | ### Options in setting.js which the delay node uses 29 | - nodeMessageBufferMaxLength *[Existing option]* 30 | 31 | The enhanced delay node supports nodeMessageBufferMaxLength as the default buffer length. 32 | The buffer length needs to be greater than queue length defined in msg.queueLength or node property UI. 33 | 34 | ![nodepropertyui.png](nodepropertyui.png) 35 | 36 | - throwDroppedMessages (default: false) *[Newly added]* 37 | 38 | If the throwDroppedMessages in setting.js is true, the enhanced delay node will throw dropped messages to the catch node. 39 | After the Node-RED supports this handling, Node-RED users are able to write their error handling in the flow. 40 | If the throwDroppedMessages in setting.js is false, the behavior of enhanced delay node will be the same as existing other nodes (Other nodes which support nodeMessageBufferMaxLength setting throws an empty message to the catch node when they reach the limit of the buffer length). 41 | 42 | ### Logging 43 | - The number of dropped messages 44 | - debug level: Node-RED outputs the node ID and the number of dropped messages to the log stream every 15 seconds. 45 | - trace level: In addition to debug level information, Node-RED writes message ID of dropped messages to the log stream. 46 | - Queue usage rate 47 | - debug level: Node-RED outputs the node ID and the size of the message queue every 15 seconds. 48 | 49 | ### Properties to change a property on demand 50 | - msg.rate: property in message to change rate value *[Discussed]* 51 | 52 | The enhanced delay node overwrites the existing rate value defined in the node property UI when it receives the message which contains msg.rate value. 53 | To be aware of changing the value when tracing, the messages about changed rate value should be outputted to log stream after changing the rate using msg.rate. 54 | 55 | ### Properties 56 | The enhanced delay node newly has queueLength as a node property. 57 | The values in the following table are default values. 58 | 59 | | # | Action | Mode 1 | Mode 2 | pauseType | timeout | timeoutUnits | rate | nbRateUnits | rateUnits | randomFirst | randomLast | randomUnits | drop | queueLength (newly added) | 60 | |---|----------------------|----------------------------------|----------------------------|-----------|---------|--------------|------|-------------|-----------|-------------|------------|-------------|-------|--------------------------| 61 | | 1 | Delay　each　Message | Fixed　delay | - | delay | 5 | seconds | | | | | | | | | 62 | | 2 | | Random　delay | - | random | | | | | | 1 | 5 | seconds | | | 63 | | 3 | | Override　delay　with　msg.delay | - | delayv | 5 | seconds | | | | | | | | | 64 | | 4 | Rate　limit | All　messages | - | rate | | | 1 | 1 | second | | | | false | 0 | 65 | | 5 | | For each　msg.topic | Send　each　topic　in turn | queue | | | 1 | 1 | second | | | | | | 66 | | 6 | | | Send　all topics | timed | | | 1 | 1 | second | | | | | | 67 | 68 | ## Reference 69 | - https://github.com/node-red-hitachi/node-red-notes/blob/master/Teleconference/20190604/20190604nodered_teleconference_yokoi.pptx 70 | - https://github.com/node-red-hitachi/node-red-notes/blob/master/Meeting/20190702/08-controlling_flows_execution.pdf 71 | 72 | ## History 73 | - 2019-08-08 - The first proposal submitted 74 | 75 | -------------------------------------------------------------------------------- /designs/delay-node-enhancements/nodepropertyui.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/delay-node-enhancements/nodepropertyui.png -------------------------------------------------------------------------------- /designs/dynamic-mqtt-node.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Dynamic MQTT Node 6 | 7 | ## Summary 8 | 9 | There is a requirement for the MQTT nodes to allow for more dynamic configuration. 10 | There are two uses: 11 | 12 | - determine a subscription topic at runtime 13 | - modify broker connection details at runtime 14 | 15 | This design note proposes a new MQTT node that can be used to satisfy these requirements. 16 | 17 | ## Authors 18 | 19 | - @knolleary 20 | 21 | ## Details 22 | 23 | Here's the basic outline: 24 | 25 | - A new MQTT node is proposed called `mqtt-control` (name, as ever, subject to change). 26 | - The node has an input and an output so sits in the middle of a flow 27 | - The node is configured with an `mqtt-broker` node - this broker node provides the connection used by the control node. 28 | - The node can be passed messages with a well-defined property (eg `msg.mqttControl`) that causes it to take an action. 29 | - The actions the node can take are: 30 | - Connect to its broker (with any of the `mqtt-broker` node's properties overridden by properties in the message) 31 | - Disconnect from the broker 32 | - Subscribe to a topic 33 | - Unsubscribe from a topic 34 | - Publish a message to a topic 35 | - The node will send any message it receives due to its subscriptions. 36 | - It will also pass through messages it receives when the action is complete. **TODO**: identify how to distinguish these from subscription messages 37 | 38 | 39 | Some observations from this design: 40 | 41 | - the control node is controlling the `mqtt-broker` node. Other regular nodes can also use that `mqtt-broker` node so will also be effected by changes. 42 | - the `mqtt-broker` config node will have a new option to not auto-connect. This would be used where the decision to connect is left to the application's use of the control node. 43 | 44 | --- 45 | 46 | ### `msg.mqttControl` 47 | 48 | This message property is used to control the `mqtt-control` node. It's content determines the action the control node will take. 49 | 50 | #### Connect 51 | 52 | The connect action will connect the mqtt-control node if it is not currently connected. 53 | 54 | If the broker is already connected, the action will be ignored. (**Q**: or should it cause a disconnect and reconnect?) 55 | 56 | The `broker` object can contain any property to override the mqtt-broker node. 57 | 58 | ~~~ 59 | msg.mqttControl = { 60 | "action": "connect", 61 | "broker": { 62 | "url": "mqtt://localhost:1883", 63 | "clientid": "my-client-id", 64 | "keepalive": 60, 65 | "clean": true 66 | "username": "", 67 | "password": "", 68 | "birth": { 69 | "topic":"", 70 | "qos": "", 71 | "retain": false, 72 | "payload": "" 73 | }, 74 | "close": { 75 | "topic":"", 76 | "qos": "", 77 | "retain": false, 78 | "payload": "" 79 | }, 80 | "will": { 81 | "topic":"", 82 | "qos": "", 83 | "retain": false, 84 | "payload": "" 85 | } 86 | } 87 | } 88 | ~~~ 89 | 90 | #### Disconnect 91 | 92 | ~~~ 93 | msg.mqttControl = { 94 | "action": "disconnect" 95 | } 96 | ~~~ 97 | 98 | #### Subscribe 99 | 100 | ~~~ 101 | msg.mqttControl = { 102 | "action": "subscribe", 103 | "topic": "a/b/c" 104 | } 105 | ~~~ 106 | 107 | #### Unsubscribe 108 | 109 | ~~~ 110 | msg.mqttControl = { 111 | "action": "unsubscribe", 112 | "topic": "a/b/c" 113 | } 114 | ~~~ 115 | 116 | #### Publish 117 | 118 | The publish action will use the same standard properties as the existing `mqtt out` node. But the `msg.mqttControl` property *must* be present to tell the control node to publish. 119 | ~~~ 120 | msg.mqttControl = { 121 | "action": "publish" 122 | } 123 | ~~~ 124 | 125 | 126 | 127 | ## History 128 | 129 | - 2019-02-25 - migrated from Design note wiki 130 | -------------------------------------------------------------------------------- /designs/env-vars/EnvVar4TabAndGroup.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: in-progress 3 | --- 4 | 5 | # Environment Variables for Group and Tab 6 | 7 | ## Summary 8 | 9 | Environment variable is useful for customizing SUBFLOW’s behavior. It supports hierarchical propagation from enclosing SUBFLOW to enclosed SUBFLOW. This means that setting the same value for multiple SUBFLOW instances is difficult without nesting. 10 | 11 | Also, there is no direct means for specifying top-level values except OS environment variable. 12 | 13 | Thus, this design note proposes ability to define environment variable in groups or tabs. 14 | 15 | ## Authors 16 | 17 | - Hiroyasu Nishiyama 18 | 19 | ## UI Extension for Groups and Tabs 20 | 21 | We add a new environment variable tab for group and tab settings panel. 22 | 23 | It is same as environment variable setting tab for SUBFLOW instance but do not provide UI definition feature for variables. 24 | 25 | ![env-var-ui-for-group.png](env-var-ui-for-group.png) 26 | 27 | Environment variable look up from a node is performed in the following order: 28 | 29 | 1. apply following lookup from inner nodes to outer nodes, 30 | 31 | 1. if the node is included in a group and the group defines the variable, its value is used, otherwise, 32 | 33 | 2. if the node is included in a subflow template and the subflow instance defines the variable, its value is used, 34 | 35 | 2. if lookup of #1 reaches top-level flow, 36 | 37 | 1. if the tab defines the variable, its value is used, otherwise, 38 | 39 | 2. if OS defines the variable, its value is used. 40 | 41 | ![env-var-lookup.png](env-var-lookup.png) 42 | 43 | Currently, environment variable input interface do not support editing value, this design note also propose adding JSONata expression to typedInput interface to make environment variable editable. 44 | 45 | ## History 46 | 47 | - 2021-07-07 - Initial Design 48 | -------------------------------------------------------------------------------- /designs/env-vars/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: in-progress 3 | --- 4 | 5 | # Environment Variables 6 | 7 | ## Summary 8 | 9 | We currently allow any node configuration property to be set from an environment variable. The node property should be set to a value of `$(ENV_VAR)` and the runtime will insert the value of `ENV_VAR` before passing the configuration to the node. 10 | 11 | There are a few other places where we want to enable more flexible configuration of nodes, so it makes sense to pull these threads together. 12 | 13 | The basic usability of the `$(ENV_VAR)` approach has issues. It can only be used for node properties that are String types. For example, you can't use it within the Change or Switch node rules as the details of each rule are wrapped up into a single JavaScript Object. It's also not an obvious thing - you need to know about it in order to use it. 14 | 15 | - [x] [Update existing env var syntax](#0-update-existing-env-var-syntax) (_in 0.19_) 16 | - [x] [Add envvar as option to TypedInput](#1-add-envvar-as-option-to-typedinput) (_in 0.19_) 17 | - [ ] [Project Settings](#2-project-settings) 18 | - [x] [Subflow Instance Properties](#3-subflow-instance-properties) (_in dev branch for 0.20_) 19 | - [x] [Compound Env Vars](#4-compound-env-vars) (_in 0.19_) 20 | - [x] [JSONata support for env vars](#5-jsonata-support-for-env-vars) (_in 0.19_) 21 | - [x] [Function node access](#6-function-node-access) (_in dev branch for 0.20_) 22 | 23 | ## Authors 24 | 25 | - @knolleary 26 | 27 | ## Details 28 | 29 | ### 0. Update existing env var syntax 30 | 31 | _In 0.19 release_ 32 | 33 | The current syntax of `$(ENV_VAR)` was loosely inspired by the bash syntax for command substitution. However, JavaScript now supports [Template Literals](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals) that use a subtly different syntax: `${ENV_VAR}` 34 | 35 | The difference between `$()` and `${}` is a potential source of confusion to new users. 36 | 37 | The proposal here is to deprecate (but still support where it's already used) the `$()` syntax in favour of `${}`. 38 | 39 | ### 1. Add envvar as option to TypedInput 40 | 41 | _In 0.19 release_ 42 | 43 | The first step here is to add Environment Variable as one of the pre-defined types in the TypedInput widget. That will make it a first-class configuration option and can be used with any node property. The `RED.utils.setMessageProperty` function can be updated to support env-var lookup - meaning Switch/Change node rules will be able to use them. 44 | 45 | ![](nr-envvar-typedinput.png) 46 | 47 | 48 | ### 2. Project Settings 49 | 50 | It should be possible to declare the env-vars a project uses within its settings, along with default values for them. It would then be possible to override the default values using normal env-vars. These would be stored in a project's `settings.json` file. As the values may be credentials, they would need to be encrypted. 51 | 52 | ![](nr-envvar-projsettings.png) 53 | 54 | #### Project Profiles 55 | 56 | Extending the Project Settings idea, we could consider adding the ability to have named profiles for the project. For example, you can define a 'test' profile and associated env-var values, and a 'production' profile with its own values. It should then be possible, through some mechanism (env var, command-line option etc) to tell the runtime which profile to use. 57 | 58 | ### 3. Subflow Instance Properties 59 | 60 | _In dev branch for 0.20_ 61 | 62 | We've had an item on the backlog for a long time to allow subflow instances to be customised. This could be achieved using the env-var model - where a user is able to set env-var values within a Subflow Instance node's properties that apply only to the nodes within that instance. 63 | 64 | ![](nr-envvar-subflows.png) 65 | 66 | ### 4. Compound env-vars 67 | 68 | _In 0.19 release_ 69 | 70 | We currently only support using `$(ENV_VAR)` syntax (changing to also support `${ENV_VAR}`) if it is the complete node property - you cannot use `$()` within a longer string. But that would be quite useful to do. 71 | 72 | For example, an MQTT node that subscribes to `$(HOSTNAME)/events`. To achieve that today, env vars have to be created for each and every property that needs to be set. It's an okay workaround but can be cumbersome. 73 | 74 | The question is whether we can support this sort of compound syntax within the editor. 75 | 76 | Within a normal ``, it would just be case of looking for any and all `$(XYZ)` and substituting the corresponding value. The major issue with this approach is that longer properties, such as a Function or Template, are more likely to legitimately want to have `$(FOO)` appear in them without intending to be substituted. This is a real concern and prevents us from adding blanket support for it to all node properties. 77 | 78 | With the proposed TypedInput addition we can be more flexible. The question is how to do compound env-vars here - there's no markup to indicate name of env var versus text to concatenate it with. An option would be to require the use of `${FOO}` if needed to delimit the env var. 79 | 80 | - `FOO` - looks up `process.env.FOO` - as this is in context of a declared env-var type, the `${}` is superfluous, except... 81 | - `${FOO}` - looks up `process.env.FOO` 82 | - `FOO${BAR}` - looks up `process.env.BAR` and concatenates it to the literal `FOO` 83 | 84 | Both the Project Settings and Subflow Instance properties list would also support this format. 85 | 86 | This does mean compound env vars would only be possible using these new features - not in any arbitrary node property ``. But a compound env var could be declared in Project Settings of Subflow Instance Properties which is then available as a simple `${NEW_ENV_VAR}` to use anywhere in the flow. 87 | 88 | ### 5. JSONata support for env vars 89 | 90 | _In 0.19 release_ 91 | 92 | We'll add a new function to our JSONata setup to allow env vars to be retrieved: 93 | 94 | $env('ENV_VAR') 95 | 96 | ### 6. Function node access 97 | 98 | _In dev branch for 0.20 release_ 99 | 100 | Currently the only way to access env vars in a Function node is to either: 101 | 102 | - import them into global context via `functionGlobalContext` 103 | - use a preceding Change node to set a message property to `$(FOO)` so the value can be passed in with each message. 104 | 105 | These are both unnecessarily onerous on the developer. The `fGC` approach doesn't fit with project/subflow settings concept. 106 | 107 | The proposal here is to add a new top-level object in the Function node to provides *read-only* access to env vars. 108 | 109 | let foo = env.get("FOO"); 110 | 111 | This will work for env vars, project settings and subflow instance properties. 112 | 113 | 114 | ## History 115 | 116 | - 2019-02-26 - migrated from Design note wiki 117 | -------------------------------------------------------------------------------- /designs/env-vars/env-var-lookup.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/env-vars/env-var-lookup.png -------------------------------------------------------------------------------- /designs/env-vars/env-var-ui-for-group.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/env-vars/env-var-ui-for-group.png -------------------------------------------------------------------------------- /designs/env-vars/nr-envvar-projsettings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/env-vars/nr-envvar-projsettings.png -------------------------------------------------------------------------------- /designs/env-vars/nr-envvar-subflows.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/env-vars/nr-envvar-subflows.png -------------------------------------------------------------------------------- /designs/env-vars/nr-envvar-typedinput.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/env-vars/nr-envvar-typedinput.png -------------------------------------------------------------------------------- /designs/exportable-subflow/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Exportable SUBFLOW 6 | 7 | This proposal covers how a SUBFLOW can be exported from the editor for reuse elsewhere. 8 | 9 | They key requirement is that once it is imported, the end user is not aware the node 10 | was implemented as a SUBFLOW - it appears as a regular node in their palette and 11 | they cannot edit the internals. 12 | 13 | This design will cover how this could be handled in the editor UI. 14 | 15 | The mechanism for packaging a SUBFLOW as a redistributable npm node module will be 16 | covered by [SUBFLOW Node Modules](../subflow-node-modules.md). 17 | 18 | This feature *must* also consider the wider enhancements to the Library UX that 19 | are in the roadmap. (*TODO: add a link to the design doc once added*). 20 | 21 | ### Authors 22 | 23 | - @HiroyasuNishiyama 24 | 25 | ### Details 26 | 27 | Node can be exported as JSON code (*Exported SUBFLOW Template*) or converted to a Node (*SUBFLOW Node*). In this document, design for *Exported SUBFLOW Template* is described. 28 | 29 | #### Exporting SUBFLOW Template 30 | 31 | ![Subflow-export-UI](Subflow-export-UI.png) 32 | 33 | By clicking **Export** button on SUBFLOW Template edit tab, SUBFLOW export panel is shown. In this panel, JSON representation of the SUBFLOW can be downloaded, copied to clipboard, or stored to library. 34 | 35 | #### Importing SUBFLOW Template 36 | 37 | SUBFLOW Template represented as JSON data can be imported to other Node-RED environment from import new node panel. 38 | 39 | ![Subflow-import-UI](Subflow-import-UI.png) 40 | 41 | #### Setting Properties for Exproted SUBFLOW Template 42 | 43 | Clicking **Edit** button on SUBFLOW Template edit tab or clicking ***Edit** button on SUBFLOW Export panel shows SUBFLOW Template properties edit tab. 44 | 45 | In this tab, new node information such as node name, version, etc. can be specified. There is **Include All SUBFLOW** check box on this panel. If checked, SUBFLOWs used within the exported SUBFLOW are recursively included in exported SUBFLOW Template (default: checked). 46 | 47 | If **Editable** check box on this panel is checked, SUBFLOW template can be edited on imported Node-RED editor (default: unchecked). Otherwise, the imported SUBLOW template is sealed. Thus, its internal flow or UI definition can not be edited. 48 | 49 | If imported subflow is sealed and its **Include all SUBFLOW** is checked, SUBFLOWs used within the imported SUBFLOW are not listed on editor pallette and hidden from users. 50 | 51 | #### Uninstalling SUBFLOW Template 52 | 53 | Imported SUBFLOW Template are listed in *Manage Palette > Palette > Nodes* tab. Similar to normal nodes, *SUBFLOW Template*s listed in this tab will have **remove** button. If a imported SUBFLOW Template includes other SUBFLOW Templates and is sealed, contained SUBFLOW Templates are also deleted. 54 | 55 | If imported SUBFLOW template is not sealed, the SUBFLOW template can also be deleted by clicking **delete subflow** button. 56 | 57 | ![Subflow-delete-UI](Subflow-delete-UI.png) 58 | 59 | ### Unknown node in imported SUBFLOW Template 60 | 61 | When a sealed SUBFLOW Template is imported, if nodes contained in the template are not installed, imported SUBFLOW template appear with shaded color and dotted line similar to `unknown` node on Node-RED Editor and are disabled. 62 | 63 | List of nodes contained in SUBFLOW but not installed can be found in Manage palette/Palette/Nodes list. 64 | 65 | ![Unknown Node](Unknown-node.png) 66 | 67 | 68 | #### SUBFLOW Template Export Format 69 | 70 | Exported SUBFLOW Template is represented by JSON array format which is similar to current flow representation. 71 | 72 | Following is a list of properties exported SUBFLOW node in SUBFLOW Template may contain (+ means that property is optional): 73 | 74 | | | name | type | description | 75 | | ---- | ------- | ------ | ---------------------------------- | 76 | | 1 | flow | array | array of contained nodes in SUBFLOW | 77 | | 2 | sealed+ | bool | SUBFLOW template is not editable | 78 | | 3 | hidden+ | bool | SUBFLOW template is hidden | 79 | | 4 | name | string | name of SUBFLOW Template | 80 | | 5 | color+ | string | color of SUBFLOW | 81 | | 6 | version | string | version of SUBFLOW (format: x.y.z) | 82 | | 7 | author+ | string | author of SUBFLOW | 83 | | 8 | license+ | string | license of SUBFLOW | 84 | | 9 | keywords+ | array | array of keywords of SUBFLOW | 85 | | 10 | description+ | string | simple description of SUBFLOW | 86 | 87 | Exported SUBFLOW Template can be **sealed** which means that internal of the SUBFLOW Template is hidden from users. 88 | 89 | If imported **seald** SUBFLOW Template includes other SUBFLOW Templates, **hidden** property of these templates are set to true and these SUBFLOW Templates are not shown in editor pallette. 90 | 91 | `flow` property is an array of nodes that are used in its SUBFLOW template. *id* of nodes in SUBFLOW template must be *name*-*ID* where *ID* is a original *id* of the node. 92 | 93 | Extended properties are ignored if they are imported to older version of Node-RED. 94 | 95 | ## Implementation Phases 96 | 97 | 1. Implement basic import & export features including: 98 | - SUBFLOW Export Panel, 99 | - Changes to import mechanisms 100 | 1. Add support of SUBFLOW Template properties tab, 101 | 1. Implement sealing related features including: 102 | - sealed SUBFLOW import/export, 103 | - uninstallation of sealed SUBFLOW, 104 | - SUBFLOW with unknown node 105 | 106 | ## History 107 | 108 | - 2019-02-27 - migrated from Design note wiki 109 | - 2019-07-26 - updated according to design discussion on early July 110 | - 2019-09-06 - updated accodring to design discussion on Sep. 4th -------------------------------------------------------------------------------- /designs/exportable-subflow/Subflow-delete-UI.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/exportable-subflow/Subflow-delete-UI.png -------------------------------------------------------------------------------- /designs/exportable-subflow/Subflow-export-UI.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/exportable-subflow/Subflow-export-UI.png -------------------------------------------------------------------------------- /designs/exportable-subflow/Subflow-import-UI.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/exportable-subflow/Subflow-import-UI.png -------------------------------------------------------------------------------- /designs/exportable-subflow/Unknown-node.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/exportable-subflow/Unknown-node.png -------------------------------------------------------------------------------- /designs/flow-linter/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Flow Linter 6 | 7 | ## Summary 8 | Node-RED makes it easy to program by wiring nodes. But sometimes, because of its less restrictive nature, programmer may write flows that are hard to understand. For example: 9 | - Function node that has no name or description. The reader need to read internal JavaScript code to understand what the node do, 10 | - Excessive amount of nodes in a single flow. 11 | 12 | And, some subtle differences in programming style cause readability low. 13 | - Size of flow on Flow Editor 14 | - Grid size 15 | - JavaScript coding style in function nodes. 16 | 17 | The flow linter provides a framework that automatically checks whether a flow is compliant with rules and conventions. 18 | 19 | ## Authors 20 | - [@k-toumura](https://github.com/k-toumura) 21 | 22 | ## Details 23 | 24 | ### Use cases 25 | - To use the consistent coding rules in a team, the tool checks a flow before submit to a repository. 26 | - To prevent bugs, the tool checks a flow when the programmer requests it, or continuously checks as a background task of Flow Editor. 27 | - Editor displays suggestions to user: 'This HTTP-in node doesn't have corresponding HTTP-response node.', 'These nodes in this flow must preserve message properties set by HTTP-in', etc. 28 | - To prevent move/edit/delete node accidentally in Editor, changing position or property of node can be restricted by rules. 29 | 30 | ### Requirements / ideas 31 | - There are various rules/conventions, and each organization/community/etc. has different policies on it. Because of this, the rules has to be pluggable and customizable. 32 | - Linter is used in both batch-style (e.g. command-line interface) and on-the-fly-style (e.g. integrated in Editor). The core service of linter can be used in both usages. 33 | - Rule configuration can be exported as JSON format and can be included in or packaged with `flows.json` so that developers can distribute flow templates with their own restrictions. 34 | - Use compatible format with other linter/tester tools for result output. 35 | 36 | ### User interface 37 | 38 | #### Command-line interface (batch-style) 39 | 40 | Stand-alone command, which reads a flow.json file and generate a verification result to standard output 41 | (like a ordinal `lint` command). 42 | ![External CLI](external-cli.svg) 43 | 44 | #### Integrated in Editor 45 | Linter can be integrated in the Editor. 46 | 47 | We are considering following two pattern for user interface: batch and on-the-fly. After we have the core design of the linter settled, we 48 | will consider more detailed design work, such as 49 | how the linter will be exposed in the editor. 50 | 51 | ##### Batch 52 | In batch style, an user pushes a button in linter sidebar 53 | to start linting. This pattern requires processing power 54 | only when the user want to check the flows. But the 55 | user explicitly push the button to check the flows. 56 | ![Sidebar](sidebar.png) 57 | 58 | ##### On-the-fly 59 | In on-the-fly style, the Editor automatically checks 60 | the flows when user updates some part of flows. 61 | This pattern need no intervention of a user, but 62 | each update of a flow causes validation processes. 63 | ![On the fly](on-the-fly.png) 64 | 65 | ### Architecture 66 | 67 | ![Functional diagram](func-diagram.svg) 68 | 69 | Outline of main routine of flow linter: 70 | 71 | 1. Read a configuration from a designated config file or `nrlint` property in setting.js 72 | 2. Determine which rule plug-in should be loaded based on the configuration. 73 | 3. create an internal representation (FlowSet object) from flow.json file or flow object in editor. 74 | 4. Loading rule plug-in modules. 75 | 5. for all plug-ins, call check() function with FlowSet object, configuration, and context. 76 | Each check() function is called in the order of appearance in configuration file. 77 | Returned context from a rule plug-in are passed to argument of next rule plug-in. 78 | 6. Consolidate results and a context, and display the result to console, side-bar, etc. 79 | 80 | #### Rule module 81 | We use a npm module mechanism to extend validation rules. 82 | 83 | - The 'core' plug-in module implements rules which are commonly used in any users. 84 | - The optional plug-in modules implement rules which are not commonly used or complex rules which require larger memory/computational footprints. 85 | 86 | #### Flow manipulation API 87 | The flow manipulation API provides a high-level interface to handle a flow. 88 | Programmers need not to know about the format of flow.json or structure of flow object in editor. 89 | 90 | Note that the Flow Manipulation API doesn't replace current flow operation functions in Node-RED 91 | editor or runtime. The API provides a abstract view to a flow, regardless of whether a target flow 92 | is in editor, or a file on the filesystem. 93 | ![Flow Manipulation API](./fmapi.svg) 94 | 95 | Flow Manipulation API is composed of following three categories. 96 | 97 | - Creation of FlowSet object: A FlowSet object is created from a flow.json file/string or flow objects which are managed in Flow Editor. 98 | - `FlowSet.parseFlow()` 99 | - `FlowSet.copyFlow()` 100 | 101 | - Manipulation of a flow via FlowSet object: All of manipulation functions can be used through a FlowSet object. 102 | - `FlowSet.prototype.getAllNodesArray()` 103 | - `FlowSet.prototype.get{Node/Flow/Config/Subflow}()` 104 | - `FlowSet.prototype.{next/prev}()` 105 | - `FlowSet.prototype.{downstream/upstream/connected}()` 106 | - `FlowSet.prototype.{insert/remove}()` 107 | - (update operations are under consideration) 108 | 109 | - Exporting a FlowSet object: FlowSet object can be exported to a file, or merging into editor's flow information. 110 | - (under consideration) 111 | 112 | For linter, we are now focusing on improving read and search interfaces. 113 | 114 | #### pluggable rules 115 | Developer can create their own rules by writing plug-in module. 116 | 117 | Plug-in module includes single function `check`, 118 | and it takes following three arguments: 119 | - complete flow set (instance of FlowSet, which implements Flow Manipulation API) 120 | - configuration object for linter 121 | - context: any object to control rule validation process 122 | 123 | The function returns an array of results and a (modified) context. 124 | Each result object contains following information 125 | - result: array of verification results of this rule 126 | - rule: (String) rule plugin name. 127 | - ids: array of node IDs which violate the rule. 128 | - name: (String) rule name. 129 | - severity: "error" or "warn" 130 | - message: (String) description of a violated rule. 131 | - context: updated context 132 | 133 | Following code shows example of plug-in, that checks existence of name of function node. 134 | 135 | ```javascript 136 | function check (afs, conf, cxt) { 137 | var funcs = afs.getAllNodesArray(); // extract all nodes 138 | .filter(function(e) {return e.type==='function';}) // filter out unrelated nodes 139 | .map(function(e) { 140 | return {id:e.id, name:e.name}; // extract their node id and name 141 | }); 142 | var verified = funcs 143 | .filter(function(e) {return e.name === undefined || e.name === "";})// check existence of name 144 | .map(function(e) { 145 | return {rule:"no-func-name", ids: [e.id], severity: "warn", 146 | name: "no-func-name", message: "function node has no name"}; 147 | // generate result 148 | }); 149 | 150 | return {result: verified, context: cxt}; // context is not modified, just pass through 151 | } 152 | 153 | module.exports = { 154 | check: check 155 | }; 156 | ``` 157 | 158 | #### Configuration 159 | Linter reads configuration files in following order: 160 | 161 | CLI version: 162 | 1. File designated in command-line argument: `nrlint --config nrlintrc.js` 163 | 2. Per-user configuration: `$HOME/.nrlintrc.js` 164 | 165 | Editor-integrated version: 166 | 1. Project-specific configuration: `$HOME/.node-red/project/projectname/nrlintrc.js` 167 | 2. Per-user configuration: `$HOME/.node-red/settings.js` 168 | 169 | - in settings.js 170 | ```json 171 | ... 172 | "nrlint": { 173 | "rules": [ 174 | { 175 | "name": "no-func-name", 176 | "mode": "warn" 177 | }, 178 | { 179 | "name": "func-style-eslint", 180 | "parserOptions": { 181 | "ecmaVersion": 6 182 | }, 183 | "rules": { 184 | "semi": 2 185 | } 186 | }, 187 | { 188 | "name": "http-in-resp", 189 | } 190 | ] 191 | }, 192 | ... 193 | ``` 194 | - in separate nrlintrc.js file 195 | ```javascript 196 | module.exports = { 197 | "rules": [ 198 | { 199 | "name": "no-func-name", 200 | "mode": "warn" 201 | }, 202 | // ... 203 | ] 204 | } 205 | ``` 206 | 207 | #### Command line options 208 | ``` 209 | Lint tool for Node-RED flow 210 | Usage: nrlint [-h] [-c configfile] filename 211 | 212 | Options: 213 | -h, --help show this help 214 | -c, --config FILE use specified configuration 215 | otherwise, the command uses ~/.nrlintrc.js 216 | ``` 217 | 218 | #### Report format 219 | 220 | ``` 221 | % nrlint /path/to/flows.json 222 | parsing /path/to/flows.json... 223 | 49e0eb77.01e8a4 warn 'too large flow size' flowsize 224 | 4579ab75.fb1b24 warn 'function node has no name' no-func-name 225 | 288e10.1e7061f warn 'function node has no name' no-func-name 226 | 4c107e00.c2394 warn 'function node has no name' no-func-name 227 | 9063e883.ecebe8 warn 'dangling http-in node' dangling-http-in 228 | c493f498.3df928 warn 'dangling http-response node' dangling-http-resp 229 | 6841634.807f99c... warn 'possible infinite loop detected' loop 230 | 4579ab75.fb1b24 warn 'Missing semicolon.' func-style-eslint 231 | 288e10.1e7061f warn 'Missing semicolon.' func-style-eslint 232 | 288e10.1e7061f warn 'Missing semicolon.' func-style-eslint 233 | e5cba426.b81768 warn 'Missing semicolon.' func-style-eslint 234 | 7b2eb3e0.dd37cc warn 'Missing semicolon.' func-style-eslint 235 | ✖ 12 problems (0 errors, 12 warnings) 236 | ``` 237 | - First column: Node ID(s) 238 | - Second column: severity (error, warn) 239 | - Third column: description of the error/warning 240 | - Fourth column: a name of rule 241 | 242 | Concerns: 243 | - Difficult to find a correspondent node by node ID. 244 | - For current workaround, use Node-RED editor's search window. 245 | 246 | 247 | 248 | ### Implementation plan 249 | 250 | #### First step 251 | - Implement CLI version 252 | - stabilize APIs for flow manipulation. 253 | - collect validation use cases, and implement rule set for them. 254 | 255 | #### Second step 256 | - Implement Editor-integrated version (rule validation in Server, batch style) 257 | - Implement a mock-up prototype of plug-in architecture 258 | - Add UI element for linter side-bar. 259 | - To check, push a button on the side-bar. Results is also displayed in it. 260 | - Internally, this calls a server-side private API endpoint for linter (`POST /linter`), 261 | and the endpoint calls CLI version of linter, then linter returns 262 | result in a JSON format. 263 | 264 | #### Third steps 265 | - Implement Editor-integrated version (rule validation in Editor, on-the-fly) 266 | - rule codes in Editor are generated automatically, or are written by hand. 267 | - Rule configuration UI 268 | - Make rules be exported as JSON format and can be included in flow.json so that developers can distribute flow templates with their own restrictions. 269 | 270 | ### Related works 271 | - [Design: Flow Manipulation API](https://github.com/node-red/designs/tree/master/designs/flow-manipulation-api) 272 | 273 | ## Appendix: Editor plug-in mechanism 274 | 275 | 276 | 277 | ### Loader 278 | 279 | In order to load a linter function and rules to the editor, the editor need to employ a plug-in mechanism. 280 | 281 | #### Utilize a node loading mechanism 282 | 283 | To implement a rule npm module for both of CLI and editor, developer can use a code 284 | generation mechanism based on Webpack. 285 | ![Generating Code](code-generation.svg) 286 | 287 | Rule developer writes their own rule in single JavaScript file, and put it on 288 | rule plug-in framework (which will be provided in flow linter project repository). 289 | These files composes an npm package, and it can distributed among users. 290 | When an user installs the package, an installation script generates 291 | an HTML file which contains a converted JavaScript code and UI elements for Editor. 292 | 293 | If the user installs the package globally (i.e. with `-g` option), the original 294 | rule code is used for CLI linter command. If the user install the package into 295 | Node-RED's user directory (`~/.node-red` or another directory which is designated by 296 | `-u` option of node-red command), Node-RED runtime automatically loads the 297 | package as a Node-RED node module, and generated code in HTML file is invoked in Editor. 298 | 299 | #### New (generic) editor plugin mechanism 300 | Above mechanism is somewhat tricky and hacky, so we are planning to design a more generic plug-in mechanism of Node-RED. 301 | 302 | ### Automatic generation of browser-side code from server-side code 303 | When validation is invoked in Editor, validation codes is executed in editor. 304 | API call from editor to runtime is expensive, and it is need to add extra adminAPI in the core. 305 | However, managing two versions (for editor and for runtime/CLI) of validation codes is cumbersome process, even if we can use some automatic generation mechanism. 306 | 307 | - Flow-check code in plug-in is used on both CLI (runs on node.js) and Editor (runs on browser). To use same code in both node.js and browser, there are tools for generate codes for both: 308 | - [Browserify](http://browserify.org/): get all dependent npm modules and put in one script file. 309 | - [webpack](https://webpack.js.org/): more general module bundler. 310 | - Currently plug-in installer uses this to generate Editor-side code, but we should use only on plug-in development phase, and make plug-in user need not to depend the tool. 311 | - [Babel](https://babeljs.io/): convert modern JavaScript (ES2015+) code to (traditional) JavaScript code that can be executed on various browsers. 312 | - It is useful but it is very large project, so we should avoid to make dependency with it. 313 | 314 | ### Other approach for linting service 315 | - [Language Server Protocol](https://microsoft.github.io/language-server-protocol/) defines the protocol used between an editor and a language server that provides language features like lint. It may be alternative option to implement lint function as a language server embedded in a Node-RED server. 316 | - Language Server Protocol itself is aimed for line-oriented text programming languages. It is not suitable for visual programming language like Node-RED. If we adopt the LSP, We might incorporate only their 'Client-Server' architecture, and not incorporate their protocol or data model. 317 | - If we adopt this architecture, the linter need not to generate code for server and browser. But we have to estimate an overhead to send flow object from browser to server. 318 | 319 | ## History 320 | - 2018-12-21 - Initial proposal submitted on [Design note wiki](https://github.com/node-red/node-red/wiki/Design:-Flow-Linter) 321 | - 2019-03-06 - migrated from Design note wiki 322 | - 2019-12-19 - update document structure, and update description of plug-in and API. 323 | - 2020-04-27 - Plug-in mechanism 324 | - 2020-06-01 - Update implementation plan 325 | - 2020-07-07 - Move description of editor plug-in mechanism to Appendix 326 | -------------------------------------------------------------------------------- /designs/flow-linter/on-the-fly.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-linter/on-the-fly.png -------------------------------------------------------------------------------- /designs/flow-linter/sidebar.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-linter/sidebar.png -------------------------------------------------------------------------------- /designs/flow-manipulation-api/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: in-progress 3 | --- 4 | 5 | # Flow Manipulation API 6 | 7 | ## Summary 8 | 9 | Purpose of Flow Manipulation API is to extend functionality of Node-RED flow 10 | editor / runtime via unified interface for translating or modifying a flow definition. 11 | 12 | ## Authors 13 | 14 | - @k-toumura 15 | 16 | ## Details 17 | 18 | ### Use Cases and Requirement 19 | 20 | #### Inactivate all debug node in one click 21 |
22 | 23 | ##### Case description 24 | In a flow development, a developer frequently uses debug nodes to check 25 | messages emitted from nodes. 26 | When development is finished, we have to disable or eliminate debug output. 27 | However, it take time to effort to do it by hand. 28 | 29 | ![case-debugnode.png](case-debugnode.png) 30 | 31 | 32 | ##### Requirements 33 | - In Editor: 34 | - Interface for search nodes that satisfy some condition. 35 | - Interface for retrieving node's properties and modifying it. 36 | - Interface for adding button or menu item to execute bulk operation. 37 |
38 | 39 | #### Deploy to multiple Node-RED runtime 40 |
41 | 42 | ##### Case description 43 | To increase capacity of flow processing, we could use multiple runtime for 44 | single flow execution. When a flow is deployed from editor, 45 | deployment handler modifies the flow and distribute it appropriately 46 | among runtimes. 47 | 48 | ![case-multiruntime.png](case-multiruntime.png) 49 | 50 | ##### Requirements 51 | - In Runtime: 52 | - Interface for insert a hook in deployment handler to call additional function. 53 | - Interface for create new flow instance for distribution. 54 | - Interface for insert new nodes to the flow for communicate between runtimes. 55 |
56 | 57 | #### Use Editor in Other Systems 58 |
59 | 60 | ##### Case description 61 | 62 | Node-RED's flow editing facility makes programming easier. 63 | If we can use Node-RED editor to write a program runs on another platform, 64 | we could increase productivity. 65 | 66 | ![case-otherruntime.png](case-otherruntime.png) 67 | 68 | ##### Requirements 69 | - In Editor or runtime 70 | - Interface for CRUD operation for flow and contained nodes to create an abstruct syntax tree from flow definition. 71 | - Transpile a flow definition to another configuration file or a program written in another language. 72 |
73 | 74 | #### Generate flow from Other Configration/Definition Language 75 |
76 | 77 | ##### Case description 78 | For integrate IoT things to Node-RED, we can use interface description Languages to 79 | automatically generate stub to interact things. 80 | For example, W3C Web of Things uses "Thing Description", and OCF uses "Swagger/OpenAPI" 81 | for describe interface of Things. 82 | 83 | ![case-generation.png](case-generation.png) 84 | 85 | ##### Requirements 86 | - In external systems 87 | - create Node-RED flow definition object in a program. 88 | - write out flow.json file that contains skelton flows for interact with things. 89 |
90 | 91 | ### API Overview 92 | 93 | ![api-overview.png](api-overview.png) 94 | 95 | Flow Manipulation API is used in Editor, Runtime, and custom code module between Editor and Runtime. A custom code module is used to modify flow definition, distribute multiple runtime, convert flow definition to configuration file of other execution environment, and so on. 96 | The API can be used in stand-alone CLI application (4th use case). 97 | 98 | - JavaScript API for browser-side 99 | - Create/Read/Update/Delete nodes/flows in editor 100 | - Serialize flow object 101 | - Javascript API for server-side, stand-alone CLI application 102 | - Serialize/deserialize flow object 103 | - Create/Read/Update/Delete nodes/flows 104 | - Export/Import a flow in well-defined format 105 | 106 | In first phase, server-side module will be provided and will mainly used in custom code modules. 107 | Similar Flow Manipulation API in browser-side is future work due to its complexity. 108 | 109 | #### Flow Model 110 | 111 | Based on [Design: Flow file format v2](https://github.com/node-red/node-red/wiki/Design%3A-Flow-file-format-v2). 112 | 113 | ![fmapi-model](fmapi-model.png) 114 | 115 | #### Operations 116 | 117 | ##### FlowSet Class 118 | - constructor() 119 | - create empty flowset 120 | - static copyFlow(flowset, flowId) 121 | - create flowset from a flow of other FlowSet 122 | - static parseFlow(string or array) 123 | - create Flowset from flow.json string or parsed array 124 | - returns: FlowSet object 125 | - generator() 126 | - iterate over all nodes in this flowset. 127 | - getNode(nodeId) 128 | - retrieve a node in this FlowSet by its node Id 129 | - returns: Node object 130 | - getFlow/Config/Subflow(nodeId) 131 | - retrieve a flow/config/subflow in this FlowSet by its node Id 132 | - returns: Flow/Config/Subflow object 133 | - prev(nodeId) 134 | - retrieve nodes which output port is connected to the node. 135 | - returns: Array of Node objects. 136 | - next(nodeId) 137 | - retrieve nodes which the node's output port is connected to. 138 | - returns: Array of Node objects. 139 | - insert(nodeId, port, prevNodeId, prevNodePort, nextNode) 140 | - insert the node between prevNode and nextNode 141 | - port is output port number of inserting node 142 | - prevNodePort id output port number of node in front of inserting node 143 | - if there is a link from prev node to nextNode, the link is unconnected. 144 | - returns: none. 145 | - remove(nodeId) 146 | - remove the node from Nodeset 147 | - returns: removed node object. 148 | - serialize(format-string) 149 | - seriazlize FlowSet to designated format 150 | - format-string: "JSON-1", "JSON-2", "YAML-1", etc. 151 | - returns: serialized flowset 152 | - connected(nodeId) 153 | - get connected nodes of the designated node. 154 | - returns: array of nodeIds 155 | - diff(flowset) 156 | - analyse difference of two flow set 157 | - returns: node ids of added, deleted, changed nodes 158 | 159 | 160 | ##### Flow Class 161 | - constructor(nodeid, name) 162 | - create empty flow. 163 | - generator() 164 | - iterate over nodes in this flow 165 | - addNode(node) 166 | - add Node to this Flow 167 | - getNode(nodeId) 168 | - retrieve a node by its id 169 | - prev(nodeId) 170 | - retrieve nodes which output port is connected to the node. 171 | - returns: Array of Node objects. 172 | - next(nodeId) 173 | - retrieve nodes which the node's output port is connected to. 174 | - returns: Array of Node objects. 175 | - insert(nodeId, port, prevNodeId, prevNodePort, nextNode) 176 | - insert the node between prevNode and nextNode 177 | - port is output port number of inserting node 178 | - prevNodePort id output port number of node in front of inserting node 179 | - if there is a link from prev node to nextNode, the link is unconnected. 180 | - returns: none. 181 | - remove(nodeId) 182 | - remove the node from Nodeset 183 | - returns: removed node object. 184 | - diff(flow) 185 | - analyse difference of two flows 186 | - returns: node ids of added, deleted, changed nodes 187 | 188 | 189 | ##### Subflow Class 190 | - constructor(nodeid, name) 191 | - create empty subflow. 192 | - generator() 193 | - iterate over nodes in this subflow 194 | - addNode(node) 195 | - add Node to this Flow 196 | - getNode(nodeId) 197 | - retrieve a node by its id 198 | - prev(nodeId) 199 | - retrieve nodes which output port is connected to the node. 200 | - returns: Array of Node objects. 201 | - next(nodeId) 202 | - retrieve nodes which the node's output port is connected to. 203 | - returns: Array of Node objects. 204 | - setInOutNode(numIn, numOut) 205 | - create input/output ports as special node. 206 | - insert(nodeId, port, prevNodeId, prevNodePort, nextNode) 207 | - insert the node between prevNode and nextNode 208 | - port is output port number of inserting node 209 | - prevNodePort id output port number of node in front of inserting node 210 | - if there is a link from prev node to nextNode, the link is unconnected. 211 | - returns: none. 212 | - remove(nodeId) 213 | - remove the node from Nodeset 214 | - returns: removed node object. 215 | - diff(flow) 216 | - analyse difference of two flows 217 | - returns: node ids of added, deleted, changed nodes 218 | 219 | ##### Node Class 220 | - constructor(nodeid) 221 | - create empty node. 222 | - connectTo(fromport, nextNodeId) 223 | - connect the node to designated node. 224 | - unconnectTo(fromport, nextNodeId) 225 | - unconnect the node from designated node. 226 | 227 | ##### Config Class 228 | - constructor(nodeid, name) 229 | - create empty config node. 230 | 231 | ### Related works 232 | 233 | - [Design: Flow file format v2](https://github.com/node-red/node-red/wiki/Design%3A-Flow-file-format-v2) 234 | 235 | - [Design: Runtime Editor Split](https://github.com/node-red/node-red/wiki/Design%3A-Runtime-Editor-Split) 236 | 237 | 238 | ## History 239 | 240 | - 2019-02-27 - migrated from Design note wiki 241 | -------------------------------------------------------------------------------- /designs/flow-manipulation-api/api-overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-manipulation-api/api-overview.png -------------------------------------------------------------------------------- /designs/flow-manipulation-api/case-debugnode.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-manipulation-api/case-debugnode.png -------------------------------------------------------------------------------- /designs/flow-manipulation-api/case-generation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-manipulation-api/case-generation.png -------------------------------------------------------------------------------- /designs/flow-manipulation-api/case-multiruntime.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-manipulation-api/case-multiruntime.png -------------------------------------------------------------------------------- /designs/flow-manipulation-api/case-otherruntime.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-manipulation-api/case-otherruntime.png -------------------------------------------------------------------------------- /designs/flow-manipulation-api/fmapi-model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-manipulation-api/fmapi-model.png -------------------------------------------------------------------------------- /designs/flow-outliner/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Flow Outliner 6 | 7 | ## Summary 8 | 9 | A common piece of feedback in the 2019 Developer survey was better ways to navigate 10 | and manage tabs within the editor. There are related issues with the general 11 | visibility of the entities in a flow - config nodes, subflow definitions etc. 12 | 13 | This design is for a new Sidebar component that provides a tree-view of the entire 14 | flow configuration. 15 | 16 | ### Summary of changes 17 | 18 | - New tree-view of flows (`Outline`) is added to the top of the Info sidebar 19 | - The Outline tree view provides: 20 | - easy navigation - clicking on an entry will reveal it in the editor 21 | - double-clicking on an entry will open its edit dialog 22 | - search and filtering of the tree view 23 | - The `Node Help` section of the Info sidebar moves to a new `Help` tab 24 | - A button is added to all node edit dialogs to open the help for that node 25 | 26 | ## Authors 27 | 28 | - Nick O'Leary 29 | 30 | ## Details 31 | 32 | ### Use Cases 33 | 34 | 1. A user has a large number of tabs - more than fit in the visible tab bar. Scrolling 35 | the tab bar is slow, particular when they want to jump from either end of the 36 | tab list. The tab-list search button helps, but requires multiple clicks. 37 | 38 | 2. A user wants to search for something repeatedly. The current search box forgets 39 | the latest search as soon as a result is clicked on or the search box closed. 40 | 41 | 3. A Subflow cannot be exported unless an instance is first added to the workspace 42 | and that is used to export. 43 | 44 | 45 | ### UI Design 46 | 47 | The design for this sidebar is inspired by the Outliner view in Blender: 48 | 49 | ![Blender outliner UI reference](images/blender-outliner.png) 50 | 51 | It provides the structure tree view of all objects in the file. Along with: 52 | 53 | 1. a search bar 54 | 2. an option to filter the view 55 | 3. options to toggle visibility 56 | 4. for a collapsed item in the tree, info on what the item contains 57 | 58 | The Outliner in Node-RED will follow a similar approach. 59 | 60 | This new component will be added to the Information sidebar. 61 | 62 | To make room for it, the current Node Help section will move to a new "Help" tab. 63 | To ease concerns over the Help being moved off the default sidebar tab, a button 64 | will be added to all edit dialogs to open the help for the node being edited. 65 | 66 | ![](images/sidebar-mock1.png) 67 | ![](images/sidebar-mock2.png) 68 | 69 | 70 | 71 | ## History 72 | 73 | - 2020-01-20 - Initial proposal 74 | -------------------------------------------------------------------------------- /designs/flow-outliner/images/blender-outliner.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-outliner/images/blender-outliner.png -------------------------------------------------------------------------------- /designs/flow-outliner/images/sidebar-mock1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-outliner/images/sidebar-mock1.png -------------------------------------------------------------------------------- /designs/flow-outliner/images/sidebar-mock2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-outliner/images/sidebar-mock2.png -------------------------------------------------------------------------------- /designs/flow-testing/images/20200131whiteboard.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-testing/images/20200131whiteboard.png -------------------------------------------------------------------------------- /designs/flow-testing/images/FlowTestingModel.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-testing/images/FlowTestingModel.png -------------------------------------------------------------------------------- /designs/flow-testing/images/flowtesting_sidebar.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-testing/images/flowtesting_sidebar.png -------------------------------------------------------------------------------- /designs/flow-testing/images/production_flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-testing/images/production_flow.png -------------------------------------------------------------------------------- /designs/flow-testing/images/seup-cleanup.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-testing/images/seup-cleanup.png -------------------------------------------------------------------------------- /designs/flow-testing/images/test-case.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-testing/images/test-case.png -------------------------------------------------------------------------------- /designs/flow-testing/images/test-in_node_properties.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-testing/images/test-in_node_properties.png -------------------------------------------------------------------------------- /designs/flow-testing/images/test-out_node_properties.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-testing/images/test-out_node_properties.png -------------------------------------------------------------------------------- /designs/flow-testing/images/test-settings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-testing/images/test-settings.png -------------------------------------------------------------------------------- /designs/flow-testing/images/test-sidebar.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-testing/images/test-sidebar.png -------------------------------------------------------------------------------- /designs/flow-testing/images/testing_flow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/flow-testing/images/testing_flow.png -------------------------------------------------------------------------------- /designs/flow-testing/previous_idea.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: in-progress 3 | --- 4 | 5 | # Flow Testing 6 | 7 | _Currently, we are considering another idea for testing flow as we wrote the following document._ 8 | 9 | Document: [another_idea.md](another_idea.md) 10 | 11 | ## Summary 12 | 13 | Node-RED project already provides the tools for testing: 14 | - [Runtime](https://github.com/node-red/node-red/wiki/Testing) 15 | - [Node](https://github.com/node-red/node-red-node-test-helper) 16 | - [Editor](https://github.com/node-red/node-red/wiki/UI-testing) (under development) 17 | 18 | This design note focuses on realizing flow testing. 19 | 20 | ## Autohrs 21 | 22 | - @k-nakanishi 23 | - @YumaMatuura 24 | 25 | ## Details 26 | 27 | ### Requirements 28 | Basic requirements of this tool are as follows. 29 | - Node-RED user can test flows without programming. 30 | - Node-RED uesr can see the test results on the editor. 31 | - Node-RED developer can test flows with CLI. 32 | - It should test the original flow as is. If it tests a copied flow of the original one, sync problem will happen. 33 | 34 | ### Usecase 35 | 1. Node-RED user does regression testing for the existing flows when updating Node-RED or services that connect with Node-RED. 36 | 1. Node-RED automatically runs a test on Travis CI when pull request was posted. 37 | 38 | #### User interface 39 | ##### Test node 40 | There are two new nodes called test-in node and test-out node. Test-in node sends a mock message on behalf of the actual input node such as http-in node. Test-out node receives a message on behalf of the actual output node such as http-response node. Test-in node and test-out node can handle multiple test cases. 41 | 42 | ![Flow testing model](images/FlowTestingModel.png) 43 | 44 | The requirements of test node are as follows. 45 | - Do nothing when it is NOT in a testing mode. 46 | - Appear only in a testing mode. 47 | - Substitute input node and output node in a testing mode. (named as `shadowing`) 48 | 49 | ##### test-in node 50 | Test-in node specifies one or more test cases so that one test-in node can run multiple test cases. 51 | 52 | ![Testing in node properties](images/test-in_node_properties.png) 53 | 54 | - Target 55 | - Specifies a target node to be substituted by a test-in node. 56 | - When you click the link on the right, all nodes except for test nodes will be shown. 57 | - Cases 58 | - Specifies test cases. You can add a test case by clicking `add` button. 59 | - Each test case has `Label` and `Message` properties. 60 | - Label 61 | - Describes the name of the test case. 62 | - Message 63 | - Describes the properties of a message. 64 | - Although it would be nice to specify each message property on GUI, simply specifying JSON data is a good starting point for the first version. 65 | - Some properties such as `msg.req` of http-in node cannot be completely expressed with JSON. This is a fundamental restriction of flow testing. 66 | 67 | ##### test-out node 68 | Test-out node verifies whether the received message is exactly the expected value or not. 69 | 70 | ![Testing out node properties](images/test-out_node_properties.png) 71 | 72 | - Target 73 | - Specifies a target node to be substituted by a test-out node. 74 | - When you click the link on the right, all nodes except for test nodes will be shown. 75 | - Cases 76 | - Specifies one or more test-in nodes. In each test-in node, one or more test cases can be specified. 77 | - Test in 78 | - Specifies a test-in node corresponding to this test-out node. 79 | - When you click the link on the right, all test-in nodes will be shown. 80 | - Label 81 | - Selects a label that the specific test-out node has. 82 | - When you click the list, show all labels in the test-in node. 83 | - Expected 84 | - Describes the expected property value of a message. 85 | - In the case that the test-out node receives a message without specifying the case here, the test will result in failure. 86 | 87 | #### Testing mode 88 | As described in test node section, Node-RED needs to change the mode from production to testing, and vice versa. 89 | 90 | ##### Difference of mode 91 | Assume that the following flow is a testing target. 92 | 93 | ![Testing target flow](images/production_flow.png) 94 | 95 | To test the above flow, change a mode to testing mode, and add test-in node and test-out node. 96 | 97 | ![Testing flow](images/testing_flow.png) 98 | 99 | To express the substitution with testing nodes, there are several design options. 100 | - Surround substituted nodes by a dotted line. 101 | - Connect a wire between substituted node and test in/out node (when clicked). 102 | 103 | ##### Switch the mode 104 | A user can see only the original flow without test nodes if Node-RED provides the functionality switching a mode. This may not be necessary for the first version. 105 | To realize the switch, the following implementation is necessary. 106 | - Add a `test` tab on the sidebar 107 | - Add a button to switch the mode on the `test` tab. 108 | - Although changing the mode right after opening a `test` tab is an option, that cannot notify to runtime server. 109 | 110 | #### Sidebar 111 | User can do the following actions on test sidebar. 112 | - Click `test mode` button to enable testing mode. 113 | - Choose test cases to run. 114 | - Click `Run` button to start testing. 115 | - See the result of each test case. 116 | 117 | ![Sidebar of flow testing](images/flowtesting_sidebar.png) 118 | 119 | 120 | ### Development 121 | #### Flow data 122 | Need to consider how to store the information of test nodes. There are two options. 123 | Option 1 is to store into flow.json. Option 2 is to create a new file test.json. 124 | 125 | - Option 1 (flow.json) 126 | - Easy to share flow data including test data. 127 | - Data size will get larger since the test data contains JSON data. 128 | - Option 2 (test.json) 129 | - Can divide production code and test code. 130 | - Need a function for storage API to load a new file. 131 | 132 | #### Test driver 133 | Needs additional discussion about which test driver should be used. If it is possible to show test results on the sidebar using Mocha, using Mocha would be good. Otherwise, we may need to implement test driver by ourselves. 134 | 135 | #### Examination 136 | When comparing the actual data and expected data, we need to consider the following cases. 137 | - If the property in `msg` is specified in expected object, the property will be compared. 138 | - If the property in `msg` is not specified in expected object, the property will be ignored. 139 | - This case always happens because `msg` object contains a random ID. 140 | - If the property in expected object does not exist in `msg`, the test case will be failed. 141 | 142 | #### Settings 143 | This flow testing feature should be turned on/off. Add a flag in settings.js. The default value is false (turn off). 144 | 145 | ### CLI 146 | This flow testing needs to be run on the CLI for targeting to run automatically on Travis CI when receiving a pull request on GitHub. 147 | When running a command like `grunt test-flow`, Node-RED runs flow testing then outputs each result such as the existing Mocha tests. 148 | - Run flow test. 149 | ``` 150 | grunt test-flow --testItem="./item.json" 151 | ``` 152 | 153 | - Command line argument. 154 | - --testItem : Specify the json file for which the test target "test-in node ID" and "Label" are set. 155 | 156 | - Example: item.json 157 | ``` 158 | [ 159 | { 160 | "id": "a9890736.34f478", // Test in node ID. 161 | "labels": [ // Label specified for test in node. 162 | "Test Case 01", 163 | "Test Case 02" 164 | ] 165 | 166 | } 167 | ] 168 | ``` 169 | - Target: 170 | - Flow to be tested is `flow.json` in user directory. 171 | 172 | ## History 173 | 174 | - 2019-04-02 - migrated from Design note wiki 175 | -------------------------------------------------------------------------------- /designs/function-library-node.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Function Library node 6 | 7 | ## Summary 8 | 9 | We have seen a growing number of users who are inserting JavaScript into global 10 | context so it can be readily reused amongst the Function nodes in a flow. This 11 | is not a great developer experience and something we could improve. 12 | 13 | The basic concept of this feature is to introduce a new Function Library 14 | configuration node (from here on referred to as `func-lib` to save typing... 15 | not necessarily the final name). This would be a node that can hold common code 16 | accessible to regular Function nodes. 17 | 18 | ## Authors 19 | 20 | - @knolleary 21 | 22 | ## Details 23 | 24 | Some initial thoughts: 25 | 26 | - each func-lib node contains some code. It should be structured as a node 27 | module - so it declares the functions/members it exports. It has a mandatory 28 | name field. 29 | - a function node can then `require` the module by its name - just as you would 30 | in node.js code. Whether this is done by introducing the `require` keyword 31 | needs thinking about. Maybe something like `node.require()` to reduce confusion 32 | over the top-level `require`. 33 | - this will mean the function has an *implicit* dependency on the func-lib config 34 | node - not an *explicit* dependency (unless we try to code-parse to spot the 35 | require statements). All of the func-lib config nodes will then appear as 36 | unused and trigger the associated warnings on deploy. This feature could 37 | introduce a new flag on config nodes that identifies them as 'userless' - that 38 | it is okay to not have any nodes explicitly depend on it. 39 | - however, not knowing the explicit dependency will mean exporting the function 40 | node won't include the func-lib nodes it needs... so maybe the function node 41 | does need to have someone to declare its dependencies... 42 | - the expanded JS editor introduced in 0.19 could be extended to a more useful 43 | IDE; with a list of all Function and func-lib nodes that can be used to 44 | quickly switch between them whilst editing. This would remove the need to 45 | close the dialog of one Function node and open another - reducing the 46 | disruption when trying edit multiple function nodes together. 47 | 48 | ## History 49 | 50 | - 2019-02-26 - migrated from Design note wiki 51 | -------------------------------------------------------------------------------- /designs/function-node-lifecycle/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Lifecycle Model of Function Node 6 | 7 | This proposal covers enhanced lifecycle model of function node. 8 | 9 | ### Authors 10 | 11 | - @HiroyasuNishiyama 12 | 13 | ### Details 14 | 15 | Function node can describe algorithms in JavaScript code. It plays a central role for expressing complex algorithms by Node-RED flows. However, current implementation of function node has following problems: 16 | 17 | 1. Can't use external libraries without modifying settings.js, 18 | 2. Function body is executed each time message is received. 19 | Describing common initialization or shutdown code is difficult. 20 | 21 | This design note addresses these problems. 22 | 23 | ### Lifecycle Model 24 | 25 | Following figure shows new lifecycle model of function node. While creating a flow using function node, a user can specify following items in settings panel: 26 | 27 | - required npm modules 28 | 29 | module to be loaded and variable name for accessing loaded module object, 30 | 31 | - initialization code 32 | code fragment executed at startup of the flow, 33 | 34 | - function body code 35 | code fragment exected when a message is received (same one in current function node), 36 | 37 | - finalization code 38 | code fragment executed when the flow is stopped. 39 | 40 | ![fn-lifecycle](./fn-lifecycle.png) 41 | 42 | ### Use of External JavaScript Libraries 43 | 44 | If required modules are not installed, we need a means to install NPM modules from inside Node-RED editor. This requires considerations on security, restriction of environments, user interaction, and other topics. In the above lifecycle model, we propose to make automatic (e.g. for headless Node-RED) or interactive module installation selectable by changing a value in `settings.js`. 45 | 46 | Function node declares its required NPM module list in its settings panel. For interactive mode, users can install, uninstall, or update NPM module from this settings panel. 47 | 48 | ![config](./config.png) 49 | 50 | A user can use required NPM module through a variable defined in this setting panel. 51 | 52 | For example, with above setting, `fs` variable in function node code can access `fs-ext` module without `require` call. 53 | 54 | If a user specifies different versions of NPM module, version conflicts may occur. The Node-RED runtime allows installing different versions of NPM modules. 55 | 56 | If Projects feature is enabled, the Node-RED runtime updates project's `package.json` on deploy if specified in `settings.js`. 57 | 58 | #### Additional settings for external npm module in `settings.js` 59 | 60 | Add a top-level property named `externalModule` to `settings.js`. It points to a object with following properties. 61 | 62 | | name | type | required | description | 63 | | :--- | :----- | :------- | :----------- | 64 | | mode | string | yes | one of `"auto"`, `"auto-update"`, `"manual"`, or `"none"` (default: `"manual"`)
- `auto`: automatically install requested NPM module if not installed.
- `auto-update`: same as `auto` but with automatic update if newer version is available.
- `manual`: users can manually install requested NPM module from user settings panel.
- `none`: installation of external NPM modules is not allowd | 65 | | allowList | array of string | no | List of regular expressions that matches module's NPM registry specification. External libraries that matches this list **can** be used in `Function` nodes. | 66 | | denyList | array of string | no | List of regular expressions that matches module's NPM registry specification. External libraries that matches this list **can not** be used in `Function` nodes. | 67 | 68 | `allowList` precedes over `denyList`. 69 | 70 | ``` 71 | // Example of settings.js 72 | externalModules: { 73 | mode: "manual", 74 | allowList: [ 75 | "^fs-ext$", // allow fs-ext 76 | "^qrcode@1.4.4$" // allow qrcode version 1.4.4 77 | ], 78 | denyList: [ 79 | ".*" // disallow others 80 | ] 81 | }, 82 | ``` 83 | If the `externalModule` property not exists, `none` mode and denyList of `".*"`, meaning not allow installation and use of external module, is expected as a default. 84 | 85 | #### New API for installing NPM module 86 | 87 | In order to allow NPM modules to be installed automatically before node execution, `RED.nodes.registerType` will be extended to accept new `dynamicModuleList` property in an optional third parameter. It may have following value: 88 | 89 | 1. a `string` value that represents a property name of a node instance. The property of the node instance must have a list of modules to install. 90 | 91 | ``` 92 | // Node definition 93 | RED.nodes.registerType("function", FunctionNode, { 94 | dynamicModuleList: "modules" 95 | }); 96 | ``` 97 | 98 | ``` 99 | // Node instance definition 100 | { 101 | ... 102 | "modules": [ "fs-ext@1.2.3", "qrcode@1.4.4" ] 103 | ... 104 | } 105 | ``` 106 | 107 | 2. or an array of objects that must have a `name` property and may have a `scope` propety (but can have any other properties) 108 | 109 | ``` 110 | // Node definition 111 | RED.nodes.registerType("function", FunctionNode, { 112 | dynamicModuleList: "modules" 113 | }); 114 | ``` 115 | 116 | ``` 117 | // Node instance definition 118 | { 119 | ... 120 | "modules": [ 121 | { 122 | "name": "fs-ext@1.2.3", 123 | "scope": "function:xyz", 124 | "var": "fs", 125 | ... 126 | }, 127 | { 128 | "name": "qrcode@1.4.4", 129 | "scope": "function:xyz", 130 | "var": "qrcode", 131 | ... 132 | } 133 | ], 134 | ... 135 | } 136 | ``` 137 | 138 | The Node-RED runtime will scan the list of modules to install in nodes deploy process before starting nodes execution. A module in the list will be installed, if auto mode is specified in `settings.js` . If `scope` is specified, NPM module is installed locally to specified scope. On removal of a node with locally installed NPM modules, the node must uninstall the installed modules. 139 | 140 | In order to support installation of NPM modules, the Node-RED runtime provides following API: 141 | 142 | - Runtime API for Nodes 143 | 144 | - `RED.nodes.addModule(opts)` 145 | - `RED.nodes.removeModule(opts)` 146 | - `RED.nodes.getModuleInfo(opts)` 147 | 148 | Node API for nodes is extended to include these functions. These new APIs accept the same parameter object `opts` to corresponding runtime APIs but works on generic NPM modules. 149 | 150 | - Editor API 151 | 152 | Currently no editor APIs and endpoints are publicly provided for installing NPM module. But, following endpoints will be used for interaction between editor and runtime for fnction node: 153 | 154 | - `PUT /function/modules/` - install or update NPM module 155 | - `GET /function/modules/` - get a list of installed NPM modules 156 | - `GET /function/modules/:module` - get a NPM module's information 157 | - `DELETE /function/modules/:module` - uninstall a NPM module 158 | 159 | ### Initialization and Finalization 160 | 161 | Add new tabs (`Initialize`/`Finalize` [tab names must be reconsidered]) for specifying code for common initialization or shouttown code. Code executed for each received messages (same as code in current Function node) can be specified in `Function` tab. This `Function` tab is selected by default. 162 | 163 | ![init-final](./init-fin.png) 164 | 165 | Button for expanding code editor should be moved to bottom of settings panel in order to make editing area larger. 166 | 167 | #### Available APIs for initialization/finalization code 168 | 169 | The `node` variable can not be accessed from initialization/finalization code. Therefore, it is not possible to use APIs depending on `node` such as `node.send`. Contex APIs for `flow` or `global` context can also be used. 170 | 171 | #### Asynchronous processing 172 | 173 | If the initialization code needs to start an asynchronous work that needs to be resolved before the start of the function body, it is possible to return the promise from initialization code. Initialization code is wrapped by async function, thus can use `await` withing the code. 174 | 175 | During the initialization process, other nodes may become active and may send messages to the function node. In such a case, the accepted messages are stored until the initialization process is completed, and they are processed in the order in which they were received when the initialization process is completed. 176 | 177 | #### Error handling 178 | 179 | Exceptions to the initialization code are logged to console. If error handling is necessary, it should be handled in the initialization/termination code appropriately. 180 | 181 | ### Library Enhancements 182 | 183 | Export format of function node to Node-RED library currently uses comments to encode properties. This must be extended to be able to include initialization/finilization code and required npm modules information. 184 | 185 | ### Additional Properties to Function node 186 | 187 | In order to support NPM module installation and initialization/finalization code, following properties are added the Function node. 188 | 189 | | ame | Type | Description | 190 | | ------------ | --------------- | ---------------------------------- | 191 | | `initialize` | `string` | JavaScript code for initialization | 192 | | `finalize` | `string` | JavaScript code for finalization | 193 | | `modules` | array of object | Array of NPM module specifications | 194 | 195 | - `initialize`/`finalize` property 196 | 197 | String representation of JavaScript code executed on initialization/finalization of function node. 198 | 199 | - `modules` property 200 | 201 | Specify NPM modules that should be installed before execution of `Function` node. It is an array of objects containing following properties: 202 | 203 | | Name | Type | Description | 204 | | ------ | -------- | ------------------------- | 205 | | `spec` | `string` | NPM install specification | 206 | | `var` | `string` | JavaScript variable name | 207 | 208 | `spec` specifies module specification (e.g. "fs-extra@1.2.3"). `var` defines variable name that can be used to access module object in `Function` node code. 209 | 210 | 211 | ## History 212 | 213 | - 2020-02-09 - Initial Note 214 | - 2020-04-02 - Update API access and error handling in init/final code 215 | - 2020-04-10 - Update async processing in initialization code 216 | - 2020-04-16 - Update message handling received while async initialization 217 | - 2020-06-01 - Update NPM installation details 218 | - 2020-07-23 - Update NPM installation details 219 | - 2020-07-25 - More update on details of NPM installation 220 | - 2020-10-30 - Update internal APIs -------------------------------------------------------------------------------- /designs/function-node-lifecycle/config.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/function-node-lifecycle/config.png -------------------------------------------------------------------------------- /designs/function-node-lifecycle/fn-lifecycle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/function-node-lifecycle/fn-lifecycle.png -------------------------------------------------------------------------------- /designs/function-node-lifecycle/init-fin.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/function-node-lifecycle/init-fin.png -------------------------------------------------------------------------------- /designs/function-node-lifecycle/lifecycle.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/function-node-lifecycle/lifecycle.pptx -------------------------------------------------------------------------------- /designs/function-node-lifecycle/modules-tab.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/function-node-lifecycle/modules-tab.png -------------------------------------------------------------------------------- /designs/function-node-lifecycle/require-interaction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/function-node-lifecycle/require-interaction.png -------------------------------------------------------------------------------- /designs/groups/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: complete 3 | --- 4 | 5 | # Grouping Nodes 6 | 7 | ## Summary 8 | 9 | This design note covers the ability to group nodes in the editor in a way that 10 | is familiar to many drawing applications such as PowerPoint. 11 | 12 | A group of nodes can be moved, copy/pasted as a single entity within the editor. 13 | 14 | The group can have some visual properties applied, including a border width/color/ 15 | style and a background color/style. 16 | 17 | The group can also have additional key/value pairs applied to it. These properties 18 | are then available to the nodes within the group. 19 | 20 | ## Authors 21 | 22 | - Nick O'Leary 23 | 24 | ## Use Cases 25 | 26 | ### Documenting Flows 27 | 28 | The visual appearance of the group can be used to help document the flows directly 29 | in the workspace. 30 | 31 | We already provide multiple ways to document flows - adding descriptions to flows 32 | and nodes, and adding Comment nodes to the workspace. But none of these approaches 33 | help to indicate what nodes a Comment node is related to. 34 | 35 | By creating a group with a Comment node in the corner as a label, it is clearer 36 | what the Comment node relates to. 37 | 38 | ![](images/groups-uc1.png) 39 | 40 | ### Adding metadata to parts of the flow 41 | 42 | There is a long-term concept for Node-RED where a flow as drawn in the editor 43 | is actually deployed across multiple runtimes. 44 | 45 | By using groups, and then applying suitable metadata to each group, the runtime 46 | would then be able to take appropriate action to deploy the different parts to the 47 | require location. 48 | 49 | ![](images/groups-uc2.png) 50 | 51 | ### User-defined scalability of flows 52 | 53 | Related to the previous use case, the metadata provided by a group could be used 54 | by the runtime to scale different parts of the flow in different ways. The user 55 | would be able to identify which parts of the flow is suitable for scaling and 56 | which should not be scaled. 57 | 58 | 59 | ## Details 60 | 61 | This design needs to cover: 62 | 63 | - The definition of a group 64 | - How is a group represented in the flow file? 65 | - Can a group contain groups? (hierarchy of groups) 66 | - Can a node exist in more than one group? (overlapping groups) 67 | - What properties does a group have? 68 | - The user interaction with groups in the editor: 69 | - How does a user create a group? 70 | - How are a group’s properties edited? 71 | 72 | ### Definition of a group 73 | 74 | A group is defined as: 75 | 76 | ``` 77 | { 78 | "id": 123, 79 | "type": "group", 80 | "z": "container-id", 81 | "nodes": [ list of node/group ids] 82 | ...other metadata... 83 | } 84 | ``` 85 | #### Group properties 86 | 87 | - **id** - unique identifier for the group 88 | - **type** - ``"group"`` 89 | - **name** - a user-friendly name for the group 90 | - **z** - identify the container of the group. This will be the id of the `tab` 91 | or `subflow` it is in. 92 | - **g** - if the group is nested inside another group, this will be the id of 93 | the parent group. 94 | - **nodes** - an array of node/group ids that are in the group. 95 | - **style** - an object containing properties related to the appearance of the group. 96 | These will be roughly consistent with SVG/CSS styles. The exact list of properties 97 | may change. 98 | - **stroke** 99 | - **stroke-opacity** 100 | - **fill** 101 | - **fill-opacity** 102 | - **label** - `boolean` - whether to display the name as a label 103 | - **label-position** - `string` - where to place the label. This is expressed 104 | as compass positions: `nw`,`n`,`ne`,`sw`,`s`,`se`. 105 | - **color** - `string` - label text color 106 | 107 | - **meta** - (name tbd) - an object of user-defined metadata for the group 108 | 109 | 110 | ### Flow file representation 111 | 112 | Groups are added to the flow as a new `group` node type. 113 | 114 | For backwards compatibility, `node-red-node-group` has been published that can 115 | be installed in older versions of NR to provide a `group` node type. This just 116 | allows the flow to run - it does not add any group functionality. 117 | 118 | 119 | When a node is added to a group, it gains a `g` property that identifies the id 120 | of the group it is in. 121 | 122 | ### User Interaction 123 | 124 | This is the most important part to get right and not all of it can be designed 125 | on paper. Some of the fine detail will come from experimentation. 126 | 127 | The intention is the grouping function is familiar to users who have used grouping 128 | functions in other applications, for example, PowerPoint, Keynote and Inkscape. 129 | 130 | More research and testing of those applications is needed to identify the standard 131 | patterns of interaction with groups and their contents. They are all slightly 132 | different, so we need to identify what feels right in the Node-RED context. 133 | 134 | #### Creating a group 135 | 136 | 1. The user selects one or more nodes in the workspace 137 | 2. They then create the group by either: 138 | - Invoke the new `core:group-selection` action 139 | - Use the keyboard shortcut assigned to that action 140 | - Select the `Create Group` option from the drop-down menu 141 | 142 | #### Ungrouping 143 | 144 | 1. The user selects a group in the workspace 145 | 2. They then remove the group by either: 146 | - Invoke the new `core:ungroup-selection` action 147 | - Use the keyboard shortcut assigned to that action 148 | - Select the `Remove Group` option from the drop-down menu 149 | 150 | This will remove the group but leave its contents in place. 151 | 152 | #### Deleting a group 153 | 154 | 1. The user selects the group. 155 | 2. They delete it by pressing the delete key 156 | - Invoke the existing `delete-section` action 157 | - Use the keyboard shortcut assigned to that action 158 | 159 | #### Editing a group 160 | 161 | 1. The user double-clicks on the group to open its edit dialog 162 | 2. The edit dialog provides options to set its appearance: 163 | - border color, thickness and style 164 | - background color, opacity and style 165 | - size and padding 166 | 167 | As with nodes, it will have a Description tab in the edit dialog. 168 | 169 | It will also provide a properties table where user-defined key/value pairs can 170 | be set. (This could be done in a later iteration - it isn't required for the 171 | initial implementation) 172 | 173 | 174 | #### Adding a node to an existing group 175 | 176 | We don't want the groups to get in the way of creating flows. Once a group has 177 | been created, it should be easy to add/remote nodes from the group without having 178 | to ungroup, select the nodes, recreate group. 179 | 180 | ##### Dragging a node into a group 181 | 182 | If the node is dragged over a group, and the user pauses the drag for a couple of 183 | seconds, it will 'enter' the group. This will reduce the chance of a node being 184 | accidentally added to a group when moving them around. 185 | 186 | ##### Merging selection 187 | 188 | Another action will be provided called `core:merge-selection-to-group`. This 189 | will merge the current selection into a group. 190 | 191 | If all of the selected 'things' are nodes, this is equivalent to the `core:group-selection` 192 | option. 193 | 194 | If there are one or more groups in the selection, they will be merged to be a single 195 | group. The new group will adopt the appearance of the first group in the selection. 196 | This is different to the `core:group-selection` action which would create a new group 197 | containing the groups. 198 | 199 | #### Removing a node from a group 200 | 201 | There's not an obvious way to drag a node out of a group. Maybe adding a combined 202 | keyboard meta-key + mouse option (We already use `Shift-Drag` to toggle `snap-to-grid` 203 | whilst dragging). Not sure how intuitive that would be. 204 | 205 | As of 1.1.0, no such mouse action has been implement for removing a node from a group. 206 | 207 | An action will be provide called `core:remove-selection-from-group` that 208 | can be used to move the selection out of the group, with a keyboard shortcut 209 | and menu item. 210 | 211 | 212 | ## History 213 | 214 | - 2020-07-09 - Updated to reflect what shipped in 1.1.0 215 | - 2020-01-15 - Initial proposal 216 | -------------------------------------------------------------------------------- /designs/groups/images/groups-uc1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/groups/images/groups-uc1.png -------------------------------------------------------------------------------- /designs/groups/images/groups-uc2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/groups/images/groups-uc2.png -------------------------------------------------------------------------------- /designs/node-installation/README.md: -------------------------------------------------------------------------------- 1 | # Node installation 2 | 3 | There are two separate designs under the title of Node installation: 4 | 5 | - [Install from url](install-from-url) - adding in NR 1.1.0 6 | - [Uploading tgz files directly](upload-tgz) - planned for 1.2 7 | -------------------------------------------------------------------------------- /designs/node-installation/add-nodes.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/node-installation/add-nodes.png -------------------------------------------------------------------------------- /designs/node-installation/install-from-url.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: complete 3 | --- 4 | 5 | # Node installation from other than public site 6 | 7 | **Node-RED 1.1.0** 8 | 9 | ## Summary 10 | Using the function for adding nodes of "Palette", you can install your own node on Node-RED from other than public site, such as local environment. 11 | 12 | You can install your own node from other than the public site by Node-RED settings and preparing your own 'npm' registry. This time I am considering providing more convenient and easier install function. I propose ideas below. 13 | 14 | ### Idea 15 | Specifying URLs for node information catalogues 16 | 17 | ### Explanation 18 | Allowing to add Git and tar file URLs in the catalogue.json 19 | 20 | ### Delivery Site 21 | GitHab(GitLab) or Web Server 22 | 23 | Supports external node modules(Tar or Git projects) installation by specifying its URL at the node installation tab on the palette management screen. 24 | 25 | ## Authors 26 | 27 | - @KazuhiroIto 28 | 29 | ## Details 30 | 31 | ### Adding following functions 32 | 33 | 34 | #### Add an option key to catalogue.json 35 | 36 | Add an option key which indicates the distribution URLs of node modules to nodes information(catalogue.json). 37 | 38 | #### Add POST /nodes method 39 | 40 | An API for installation external node modules from the palette management screen. By adding distribution URL to Admin API POST /nodes, install external node modules (Tar or Git projects) . 41 | 42 | With this additional function, support not only "npm install <pkg>@<version>" but also “npm install <git://url>” or “npm install <tarball url>” as a command executed when adding a node to Node-RED. 43 | 44 | ![bitmaps](./add-nodes.png) 45 | 46 | ### Adding an option key to catalogue.json 47 | 48 | Adding an option key **‘pkg_url’ (optional)** to catalogue.json. Specify the distribution URL to the key in the following format. 49 | 50 | - Tar file case: 51 | "http[s]://<Tar file distribution server path>/<node module file.tgz>" 52 | 53 | - GitURL case: 54 | "git+https://<Git Server>/<node module project.git>" 55 | 56 | #### Example: catalogue.json 57 | ```json 58 | { 59 | "name": "Node additional test catalogue", 60 | "updated_at": "2019-08-09T00:00:00.000Z", 61 | "modules": [ 62 | { 63 | "description":"Node-RED Dashboard UI widget node for simple list", 64 | "keywords": [ "list", "item" ], 65 | "types": [ "ui_list" ], 66 | "updated_at": "2019-08-08T00:00:00.000Z", 67 | "id": "node-red-node-ui_list", 68 | "version": "0.1.4", 69 | "url": "https://pkg-srv /ui_list/README.md", 70 | "pkg_url": "https://pkg-srv/ui_list/node-red-node-ui_list-0.1.4.tgz" 71 | } 72 | ] 73 | } 74 | ``` 75 | 76 | ### Adding a function to POST /nodes method 77 | 78 | #### POST /nodes 79 | Request parameters for installation of a new node module. 80 | 81 | ##### The request parameters: 82 | JSON string which has the following field. 83 | 84 | - module: 85 | The name of node module to install or the full path of a directory including a node module. 86 | - version (optional): 87 | The version of the node module. (The default is latest) 88 | This parameter is ignored when specified the full path of a directory for ‘modules’ key. 89 | This parameter is ignored when specified an url for **‘url’ (optional)** key. 90 | 91 | - **url (optional)** – **NEW PARAMETER** : 92 | An url string for a tar file or a git project. 93 | When specified this parameter, the node module will be installed from the url. 94 | You must define the name of a node module for ‘modules’ key. 95 | If not or wrong name, its upgrade results are not reported correctly. 96 | 97 | #### Example: POST /nodes method 98 | ```json 99 | POST /nodes 100 | { 101 | "modules": "node-red-node-ui_list", 102 | "version": "0.1.4", 103 | "url": "https://pkg-srv/ui_list/node-red-node-ui_list-0.1.4.tgz" 104 | } 105 | ``` 106 | ## History 107 | 108 | - 2020-06-30 - Shipped in 1.1.0 109 | - 2019-09-24 - Initial proposal submitted 110 | -------------------------------------------------------------------------------- /designs/node-installation/upload-tgz.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Installing nodes via tgz upload in editor 6 | 7 | ## Summary 8 | 9 | The `/nodes` api provides a number of ways to specify a module to be installed: 10 | 11 | - by name 12 | - by path to folder containing the code 13 | - by url to an installable thing (which requires the name of the module to be provided as part of the api call as well) 14 | 15 | This design note proposes allowing the user to install a node by uploading its tgz file via the editor. 16 | 17 | ## Authors 18 | 19 | - @knolleary 20 | 21 | ## Details 22 | 23 | This feature will *only* be available if the default LocalFileSystemStorage plugin is 24 | being used. This is because it requires having persistent file storage that cannot be 25 | solved with the `autoInstallModules` option. 26 | 27 |
Why not `autoInstallModules` 28 | When the `autoInstallModules` flag is set, the runtime will automatically `npm install` 29 | any modules it knows it had previously (from `.config.json`) but fails to find on 30 | startup. This assumes the modules can be installed based just on their module name. 31 | 32 | Once we support uploading tgz files, without persistent file storage, the tgz file 33 | may no longer be available locally when node-red restarts - so a reinstall will 34 | not be possible. 35 |
36 | 37 | ### `/nodes` api updates 38 | 39 | The `/nodes` end-point will now accept a POST request with a content-type of `multipart/form-data`. 40 | 41 | It will accept a single file uploaded with the field name `tarball`. If this field 42 | is set, none of the other fields (`name`,`version`,`url`) should be provided. 43 | 44 | The following `curl` command would be a valid way to call the endpoint: 45 | 46 | ``` 47 | curl -v -F tarball=@/tmp/node-red-node-random-0.2.0.tgz http://localhost:1880/nodes 48 | ``` 49 | 50 | ### Install behaviour 51 | 52 | - The `tgz` file will first be unpacked to a temporary directory so the runtime can 53 | examine its contents. 54 | - The `tgz` file will then be saved to `/nodes/`, where 55 | `filename.tgz` is normalised to `name-of-module-x.y.z.tgz` (the standard format 56 | generated by `npm pack`). This normalisation step is needed because we cannot 57 | rely on the original `tgz` filename being accurate. 58 | - Node-RED will run `npm install /nodes/` 59 | and then load the module as it does any other module. 60 | - If we detect this is an upgrade of an existing module that has a local `tgz` 61 | file, we will remove the old versions tgz file 62 | 63 | The user's `package.json` file will be updated by npm with an entry: 64 | 65 | ``` 66 | ... 67 | "module-name": "file:./nodes/" 68 | ... 69 | ``` 70 | 71 | By having the `tgz` files in the `/nodes` directory they can be easily 72 | backed up alongside the flow files and package.json. Reinstalling the modules can 73 | be done by running `npm install` in ``. 74 | 75 | #### Projects 76 | 77 | Some extra thought is needed about how projects can include the tgz files. They are 78 | installed into the runtime, just like other modules, so projects will be able to 79 | use the nodes. But should the tgz be added to the project's files so they get version 80 | controlled? Without that, it would mean more manual work is needed to clone and 81 | run the project elsewhere - a simple `npm install` wouldn't be sufficient. 82 | 83 | The UI tries to help the user keep their project's package.json file up to date, but could do a much better job of it. Maybe the UI could allow the user to optionally 84 | copy the tgz into their project's directory (still under a `nodes` directory). 85 | 86 | 87 | ### Palette Editor enhancements 88 | 89 | The `install` tab of the palette editor will have an upload button added to the 90 | toolbar. Clicking on the button will prompt the user to select a file to upload. 91 | 92 | It will then show a progress bar as the file is being uploaded and then a similar 93 | view to when clicking on install for a node in the catalog; a spinner and a button 94 | to view the event log. 95 | 96 | ### Configuration options 97 | 98 | The Palette Editor can already be disabled by setting `editorTheme.palette.editable` 99 | to `false`. 100 | 101 | As some embedded uses of Node-RED may want to have more control over what can be 102 | installed into the runtime, the ability to upload a tgz can be disabled by setting 103 | `editorTheme.palette.upload` to `false`. 104 | 105 | ### Runtime API changes 106 | 107 | The `runtime.nodes.addModule` API will be updated to accept a `tarball` property 108 | in its options argument. 109 | 110 | This will be an object of the form: 111 | ```json 112 | { 113 | "name": "", 114 | "size": "", 115 | "buffer": "" 116 | } 117 | ``` 118 | 119 | If this property is set, then none of `module`, `version` or `url` may also be set. If 120 | any of them are, the call with reject with an error with the code `invalid_request` and 121 | a status `400`. 122 | 123 | 124 | ## History 125 | 126 | - 2020-08-12 - Initial proposal submitted 127 | -------------------------------------------------------------------------------- /designs/node-messaging-api.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: in-progress 3 | --- 4 | 5 | # Node Messaging API 6 | 7 | ## Summary 8 | 9 | The current node api doesn't allow the runtime to know when a message has 10 | been fully dealt with. This proposal looks at a new API for nodes that will 11 | address this issue. 12 | 13 | ## Authors 14 | 15 | - @knolleary 16 | 17 | ## Details 18 | 19 | The current mechanism for a node to handle messages is as follows: 20 | 21 | - a handler is registered for the `input` event 22 | - within that handler a node can call `this.send()` as many times as it wants 23 | - if an error is hit, it can call `this.error(err,msg)` 24 | 25 | ``` 26 | var node = this; 27 | this.on('input', function(msg) { 28 | // do something with 'msg' 29 | if (!err) { 30 | node.send(msg); 31 | } else { 32 | node.error(err,msg); 33 | } 34 | }); 35 | ``` 36 | 37 | This simple system has a few limitations that we want to address: 38 | 39 | 1. we cannot correlate a message being received with one being sent - particularly when there is async code involved in the handler 40 | 2. we do not know if a node has finished processing a received message 41 | 3. due to 1 & 2, we cannot build any timeout feature into the node 42 | 4. we know of a use case where an embedder of node-red requires nodes to be strictly one-in, one-out, and that should be policed by the runtime. Putting aside the specifics, we cannot provide any such mode of operation with the current model 43 | 44 | This design note explores how this mechanism could be updated to satisfy these limitations. 45 | 46 | ### Use Cases 47 | 48 | - Allow a user to create a Flow that is notified when a message is successfully 49 | processed at a critical point of a flow. For example, the `Email Out` node 50 | has sent its message. 51 | - Allow the runtime to track messages through a flow. This can be used to: 52 | - build a timeout mechanism into nodes 53 | - allow a flow to be gracefully shutdown - allowing in-progress messages to complete 54 | 55 | ### `node.on("input", function(msg, send, done) {})` 56 | 57 | The callback handler for the `input` event is updated to include `send` and `done` 58 | arguments. 59 | 60 | The `send` argument is a function that is equivalent to `node.send`. Using this 61 | function will allow the runtime to correlate the call with the message that 62 | triggered the callback. 63 | 64 | The `done` argument is a function that must be called when the node has finished 65 | processing a message. 66 | 67 | The `done` function takes one argument - an optional `error`. If this is provided, 68 | the node will log the error as if `node.error(error,msg)` was called. A node should 69 | *not* do both `node.error` and `done(error)` as this will cause the error to be 70 | reported twice. 71 | 72 | When `done()` is called, any in-scope `Complete` nodes will be triggered. This is 73 | a new node type added under this proposal. 74 | 75 | To ensure backwards compatibility, a node should check if `send` and `done` exist 76 | before using them. That will allow the node to be installed in older versions of Node-RED. 77 | 78 | ``` 79 | var node = this; 80 | this.on('input', function(msg, send, done) { 81 | send = send || node.send; 82 | // do something with 'msg' 83 | if (!err) { 84 | node.send(msg); 85 | } else { 86 | node.error(err,msg); 87 | } 88 | if (done) { 89 | done(msg); 90 | } 91 | }); 92 | ``` 93 | 94 | ### Function node 95 | 96 | If a Function node returns a message, or array of messages, then `done` will be 97 | called implicitly by the node. 98 | 99 | If the Function node does not return any messages then the Function must call `node.done()`. 100 | This allows a Function node that does asynchronous work (optionally using `node.send()`) 101 | to call `node.done()` at the right time. 102 | 103 | ### `Complete` node 104 | 105 | This design has changed quite a few times. It has bounced between adding a new 106 | node to handle the 'done' events, and reusing the `Status` node. 107 | 108 | Having modelled the `Status` node approach, a number of issues were identified that 109 | made it less ideal. 110 | 111 | - Existing flows using Status nodes will suddenly start receiving the 'done' status 112 | events. If the flows are not expecting them, that could have bad side-effects. 113 | - A workaround to that would be to add an option to the Status node to opt into 114 | receiving 'done' events. But that gets messy. 115 | - Overloading the Status event with the Done event also gets messy when there is 116 | also an error to report. 117 | 118 | Having a new node type to handle the 'done' events is the cleanest way for a flow 119 | author to create a flow that can react to the different types of event - status, 120 | error and done. 121 | 122 | This node will be very similar in design to the Catch/Status node. However, it will 123 | *not* offer the default 'Handle all' type of behaviour the others do. The user 124 | *must* target it at specific nodes in the flow. They could chose to select all, 125 | but it would not be a top-level menu option as it is with the Catch/Status nodes. 126 | 127 | #### Timeout handling 128 | 129 | This is moving to a separate Design note and not part of the delivery of this 130 | design. 131 | 132 | [https://github.com/node-red/designs/blob/master/designs/timeout-api.md]() 133 | 134 | 135 | ## History 136 | 137 | - 2019-07-09 - going back to the callback approach 138 | - 2019-06-19 - another iteration of the design 139 | - 2019-03-26 - rewritten to cover new design proposal 140 | - 2019-02-27 - migrated from Design note wiki 141 | -------------------------------------------------------------------------------- /designs/node-property-typing.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Node Property Typing 6 | 7 | ## Summary 8 | 9 | A node's `defaults` object in its HTML definition provides a list of its properties 10 | with some metadata associated with them. 11 | 12 | If a property is intended to be a reference to a configuration node, its entry 13 | in the defaults object will include the `type` property that identifies the type 14 | of config node it should point to. 15 | 16 | This allows the editor to automatically generate the Config Node select box UI 17 | and manage the relationship between the nodes. 18 | 19 | This design note explores extending this property to allow for a richer set of 20 | relationships between nodes. 21 | 22 | ## Authors 23 | 24 | - @knolleary 25 | 26 | ## Details 27 | 28 | 29 | ### Examples uses 30 | 31 | #### `mqtt in` nodes references a `mqtt-broker` node 32 | 33 | This is the existing use of the `type` property. 34 | ``` 35 | broker: { type: "mqtt-broker"} 36 | ``` 37 | 38 | #### `link out` node references multiple `link in` nodes 39 | 40 | In the current code, the editor is aware of the `link` nodes and their `links` 41 | property that is an array of ids to partner link nodes. 42 | 43 | This design proposes the following syntax could be used to tell the editor 44 | this property is an array of references to `link in` nodes: 45 | 46 | ``` 47 | links: { type: "link in[]"} 48 | ``` 49 | 50 | 51 | #### `catch` node references multiple nodes of any type 52 | 53 | As with the `link` nodes, the editor knows it has to handle the `scope` properties 54 | of the `catch`, `status` and `complete` nodes as special cases. 55 | 56 | Unlike the link nodes, these nodes can reference any node type. This design proposes 57 | this could be expressed as follows. 58 | 59 | ``` 60 | scope: { type: "*[]"} 61 | ``` 62 | 63 | *Note: an earlier iteration of this design proposed `:any[]`. That was inspired 64 | by TypeScript's `any` syntax, but the `:` was added to make it look like a keyword - 65 | as `any` would be a valid node-type. On second though, this didn't look right, so 66 | has been changed to the above syntax.* 67 | 68 | #### Node references a single node of multiple possible types 69 | 70 | The current model requires one node property per type of config node the node 71 | may have a relationship to. We have had cases where a node needs to reference a 72 | config node that could be of different types (for example, the WebSocket nodes). 73 | 74 | The following sytanx could used to list the candidate types: 75 | 76 | ``` 77 | ui_container: { type: "ui_group | ui_widget"} 78 | ``` 79 | 80 | #### Node references multiple nodes of multiple possible types 81 | 82 | ``` 83 | ui_container: { type: "(ui_group | ui_widget)[]"} 84 | ``` 85 | 86 | 87 | ## History 88 | 89 | - 2021-01-07 Updated syntax 90 | - 2020-09-29 Initial design drafted 91 | -------------------------------------------------------------------------------- /designs/overwrite-settings.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Overwrite Settings 6 | 7 | ## Summary 8 | 9 | In some cases, users want to change default values in `settings.js` file without modifying the file contents. This design note proposed new command line option for changing values in `settings.js`. 10 | 11 | The basic concept of this feature is to introduce a new Function Library 12 | configuration node (from here on referred to as `func-lib` to save typing... 13 | not necessarily the final name). This would be a node that can hold common code 14 | accessible to regular Function nodes. 15 | 16 | ## Authors 17 | 18 | - @HiroyasuNishiyama 19 | 20 | ## Details 21 | 22 | Add startup option to overwrite values in `settings.js`. 23 | 24 | `-D `*\*`=`*\* 25 | 26 | `-D @`*\* 27 | 28 | First option type overwrites specified property in the `settings.js` by the specified JSON value. 29 | 30 | Second option type overwrites properties in the `settings.js` by contents of the specified JSON file. Other properties are untouched. 31 | 32 | The `settings.js` file is not modified with these options. 33 | 34 | `--define` can be used instead of `-D`. 35 | 36 | ### Examples 37 | 38 | ``` 39 | // enable project feature 40 | -D editor.theme.projects.enables=true 41 | 42 | // set console log level to "debug" 43 | -D logging.console.level="debug" 44 | 45 | // enable context storage 46 | // [context-setting.json] 47 | // "contextStorage": { 48 | // "default": { 49 | // "module": "localfilesystem" 50 | // }, 51 | // } 52 | -D @context-setting.json 53 | ``` 54 | 55 | ### Restrictions 56 | 57 | Values that can be specified are limited to JSON value. Thus, settings that require JavaScript code execution (e.g. module require) can not be expressed. 58 | 59 | ## History 60 | 61 | - 2020-02-10 - Initial Design Note 62 | -------------------------------------------------------------------------------- /designs/pluggable-library.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Library Plugins 6 | 7 | ## Summary 8 | 9 | Building on the Node-RED Plugins design, we can now look at supporting Library Plugins. 10 | 11 | A library plugin acts as an alternative source of content for the libraries shown 12 | in the editor. 13 | 14 | For example, a library hosted on a remote URL, or an additional library on the local 15 | filesystem. 16 | 17 | An example use case is where a team of developers want a shared library of reusable 18 | assets. They have a shared drive mounted on each of their computers (or could be 19 | via a service like Dropbox). They want to be able to access the content of that 20 | shared drive when loading/saving flows and other assets within the editor. 21 | 22 | ## Authors 23 | 24 | - Nick O'Leary 25 | 26 | ## Details 27 | 28 | A library plugin consists of two parts: 29 | 30 | 1. a runtime plugin that implements a defined API to act as a library source 31 | 2. an optional editor plugin that allows the user to dynamically configure new instances of the plugin 32 | 33 | This design is currently focussed on statically adding new library sources via 34 | the settings file. More work is needed to design how a user may add/config new 35 | sources via the editor. 36 | 37 | 38 | ### Runtime plugin 39 | 40 | The following is the skeleton of an example runtime plugin 41 | 42 | ``` 43 | module.exports = function(RED) { 44 | 45 | class FileStorePlugin { 46 | constructor(config) { 47 | this.config = config; 48 | 49 | } 50 | async init() { 51 | console.log("FileStorePlugin.init") 52 | 53 | } 54 | async getEntry(type,path) { 55 | console.log("FileStorePlugin.getLibraryEntry",type,path) 56 | return [] 57 | } 58 | async saveEntry(type,path,meta,body) { 59 | console.log("FileStorePlugin.saveLibraryEntry",type,path) 60 | 61 | } 62 | } 63 | 64 | RED.plugins.registerPlugin("node-red-library-filestore", { 65 | type: "node-red-library-source", 66 | class: FileStorePlugin 67 | }) 68 | } 69 | ``` 70 | 1. The plugin defines a class (`FileStorePlugin`) that implements a standard api: 71 | - `constructor(config)` : passed in the configuration for the plugin instance 72 | - `init()` : optional async function to perform any initialisation of the plugin. 73 | - `getEntry(type,path)` : get an entry from the library 74 | - `saveEntry(type,path,meta,body)` : save an entry to the library 75 | - `deleteEntry(type,path)` : remove an entry from the library 76 | 2. It registers itself as a plugin with a type of `node-red-library-source` and 77 | provides a reference to its class 78 | 79 | *TODO* add more details on `get/save/deleteEntry` functions. Their behaviour should 80 | match the existing file system store we already have... although `deleteEntry` is 81 | a new option that hasn't existed before. 82 | 83 | ### Editor plugin 84 | 85 | *This needs more work to fill out the details* 86 | 87 | To allow the user to add and configure new instances of the plugin via the editor, 88 | it needs to provide an editor plugin. 89 | 90 | In a similar way to nodes, it provides a list of configuration properties. I don't 91 | know yet if we need to provide more custom edit functionality - or whether auto-generating 92 | a config screen of key/values pairs based on these defaults is sufficient. 93 | 94 | This is an area where the design may evolve over future versions. 95 | 96 | ``` 97 | 108 | ``` 109 | 110 | ### HTTP Admin API 111 | 112 | #### `/settings` 113 | 114 | The list of configured libraries is added to the `/settings` end point. The default 115 | value is: 116 | 117 | ``` 118 | { 119 | "libraries": [ 120 | { 121 | "id": "local", 122 | "label": "editor:library.types.local", 123 | "user": false, 124 | }, 125 | { 126 | "id": "examples", 127 | "label": "editor:library.types.examples", 128 | "user": false, 129 | "readOnly": true, 130 | "types": [ 131 | "flows" 132 | ] 133 | } 134 | ] 135 | } 136 | ``` 137 | 138 | This default shows the definition of the built-in libraries - the local lib and 139 | examples lib. 140 | 141 | - `id` : unique identifier for this library. 142 | - `label`: the label to show for the library. It gets passed through `RED._()` so can be a message catalog reference 143 | - `user` : whether this is a user-configurable library 144 | - `readOnly`: whether this is a read-only library. 145 | - `types`: an array of asset types the library supports. 146 | - (not shown) `icon`: an optional FA-icon identifier. 147 | 148 | 149 | A 'user-configurable' library is one the user has added via the editor UI. The default 150 | libraries and any configured via the settings file are *not* user-configurable. 151 | 152 | 153 | #### `/library/:id` 154 | 155 | ***This is not in the plan for 1.3 - we'll revisit it in the future*** 156 | 157 | This new admin endpoint will return information about a specific library. 158 | 159 | The object returned will match those returned by the `/settings` endpoint under 160 | the `libraries` object. 161 | 162 | In the case of user-configurable libraries, it will include an additional property: 163 | 164 | - `config`: a key/value object of configurable properties for the library. 165 | 166 | If any of the configuration properties are flagged as `password` type properties, 167 | rather than return the value, it will include a property with the name `has_` 168 | as a boolean to indicate whether the runtime has a value stored or not. This is 169 | inline with how flow credentials are handled and the general principle of never 170 | returning passwords back to the editor. 171 | 172 | To update the configuration of a user-configurable library, a `PUT` HTTP Request 173 | can be sent to the same endpoint with the updated object. 174 | 175 | #### `/library/:id/:type` 176 | 177 | This is the existing endpoint used to browse the contents of the library for a given 178 | type of object. 179 | 180 | 181 | ### `@node-red/runtime` API 182 | 183 | ***This is not in the plan for 1.3 - we'll revisit it in the future*** 184 | 185 | *TODO: define the APIs to support adding/configuring new library sources in the editor* 186 | 187 | ### Settings file 188 | 189 | A user can define new library sources via their settings file using the new `editorTheme.library` 190 | setting: 191 | 192 | ``` 193 | editorTheme: { 194 | library: { 195 | sources: [ 196 | { 197 | id: "team-dir", 198 | type: "node-red-library-filestore", 199 | label: "my team-dir", 200 | path: "/tmp/nr-lib/" 201 | } 202 | ] 203 | } 204 | } 205 | ``` 206 | 207 | For each source: 208 | - `id` : unique identifier for the library source instance 209 | - `type`: the type of library plugin this is an instance of 210 | - `label`: how the library is labelled in the UI 211 | - ... anything else is then plugin-specific configuration. 212 | 213 | Any source defined in the settings file will get its `user` property set to `false` 214 | as it cannot be reconfigured in the editor. 215 | 216 | 217 | 218 | ## History 219 | 220 | - 2020-12-xx - Initial proposal submitted 221 | -------------------------------------------------------------------------------- /designs/pluggable-message-routing/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: complete 3 | --- 4 | 5 | # Pluggable Message Routing 6 | 7 | ## Summary 8 | 9 | The mechanism by which messages are passed from one node to the next should be 10 | a pluggable component of the runtime. This would enable, for example, a flow that 11 | spans multiple runtime instances. Other use cases: 12 | 13 | - flow debugger 14 | - adding custom low-level logging of node send/receive events including the 15 | full message data 16 | 17 | ## Authors 18 | 19 | - @knolleary 20 | 21 | ## Details 22 | 23 | ### Use Cases 24 | 25 | #### 1. Flow debugger 26 | 27 | - the ability to set breakpoints in a flow that will halt the passing of messages 28 | and allow a user to inspect the state of the system 29 | - provide more detailed information about messages passing through the flows 30 | without having to instrument it with multiple Debug nodes. 31 | 32 | #### 2. Flow Testing 33 | 34 | - inserting hooks through a flow to verify their behaviour 35 | - be able to stub a node in the flow with test-specific behaviour 36 | 37 | #### 3. Flows that spanning multiple runtimes 38 | 39 | - instances running on separate cores routed manually or based on a policy or algorithm. 40 | - runtimes on separate machines in a cluster 41 | - runtimes in a fog deployment, e.g. on devices, gateways, cloud routed dynamically 42 | depending on the mobility of a device, associated connectivity, location, etc. 43 | 44 | Note: The method for managing and distributing instances running in different cores 45 | or machines is outside the scope of this design. 46 | 47 | 48 | ### High-level design 49 | 50 | There are two possible models to use for the pluggable message routing feature. 51 | 52 | 1. a configurable stack of router layers 53 | 2. a set of life-cycle hooks that can have handlers attached to them 54 | 55 | **At this stage, the life-cycle hooks approach is the preferred model.** 56 | 57 | #### Stack of layers 58 | 59 | The working assumption for this feature has always been to take an approach similar 60 | to that of Express middleware: 61 | 62 | Each message is passed to the top layer of the stack. That layer can do whatever 63 | it wants with the message before either disposing of it or passing it on to the 64 | next layer in the stack. 65 | 66 | Node-RED would provide a core `LocalRouter` layer that knows how to route messages 67 | to nodes in the local runtime. 68 | 69 | The stack layers could be customised via either the settings file or via runtime API. 70 | 71 | The problem with this model comes when you have unrelated things wanting to customise 72 | the stack. 73 | 74 | For example, if a user provided a custom layer to add additional logging, they may 75 | specify a stack that looks like: 76 | ``` 77 | 1. MyCustomLogger 78 | 2. RED.router.LocalRouter 79 | ``` 80 | 81 | They then enable the Flow Debugger that has to insert itself into this stack. That would 82 | mean the Flow Debugger would have to keep track of the old stack configuration and be 83 | able to restore it when the Debugger is disabled. 84 | 85 | This could get very complicated to manage. 86 | 87 | 88 | #### Life-Cycle Hooks 89 | 90 | This is similar to the Fastify model of registering hooks at certain key points 91 | in the life-cycle of a message. 92 | 93 | Rather than leave the stack as being completely customisable, we recognise there 94 | is a core set of steps that every message has to go through. Within those steps 95 | are points where some custom code *might* want to run. 96 | 97 | The following diagram shows the set of hook points in the messaging path. The orange 98 | lines show the span of synchronous calls. This has important implications in terms 99 | of whether a hook handler can be synchronous or asynchronous and what responsibilities 100 | it has. 101 | 102 | ![](message-router-events.png) 103 | 104 | 1. `onSend` - passed an array of `SendEvent` objects. The messages inside these objects 105 | are exactly what the node has passed to `node.send` - meaning there could be duplicate 106 | references to the same message object. *See below regarding [async vs sync handlers](#async-vs-sync-handlers)* 107 | 108 | 2. `preRoute` - passed a `SendEvent`. *See below regarding [async vs sync handlers](#async-vs-sync-handlers)* 109 | 110 | 3. `preDeliver` - passed a `SendEvent`. The local router has identified 111 | the node it is going to send to. At this point, the message has been cloned if needed. 112 | 113 | 4. `postDeliver` - passed a `SendEvent`. The message has been dispatched 114 | to be delivered asynchronously (unless the sync delivery flag is set, in 115 | which case it would be continue as synchronous delivery) 116 | 117 | 5. `onReceive` - passed a `ReceiveEvent` when a node is about to receive a message 118 | 119 | 6. `postReceive` - passed a `ReceiveEvent` when the message has been 120 | given to the node's `input` handler(s) 121 | 122 | 7. `onComplete` - passed a `CompleteEvent` when the node has completed with a message or logged an error 123 | 124 | 125 | ##### `SendEvent` object 126 | 127 | ```json 128 | { 129 | "msg": "", 130 | "source": { 131 | "id": "", 132 | "node": "", 133 | "port": "", 134 | }, 135 | "destination": { 136 | "id": "", 137 | "node": undefined, 138 | }, 139 | "cloneMessage": "true|false" 140 | } 141 | ``` 142 | ##### `ReceiveEvent` object 143 | 144 | ```json 145 | { 146 | "msg": "", 147 | "destination": { 148 | "id": "", 149 | "node": "", 150 | } 151 | } 152 | ``` 153 | 154 | ##### `CompleteEvent` object 155 | 156 | ```json 157 | { 158 | "msg": "", 159 | "node": { 160 | "id": "", 161 | "node": "" 162 | }, 163 | "error": "" 164 | } 165 | ``` 166 | 167 | 168 | #### Async vs Sync handlers 169 | 170 | Due to the requirements of the messaging path, some hook handlers must complete 171 | their work synchronously. 172 | 173 | The `onSend` and `preRoute` hooks are triggered *before* any message cloning has 174 | happened. If they complete asynchronously, execution will return to the calling 175 | node *without* the message having been cloned. That will lead to unexpected issues 176 | as the node may modify the message object before the original version has been delivered. 177 | 178 | If `onSend` and `preRoute` hooks need to do asynchronous work before the event passes 179 | on, and the `cloneMessage` property is set to `true`, then they *must* clone and 180 | replace the message object in the event object. They *must* also set the `cloneMessage` 181 | property to false to ensure no subsequent cloning happens for the message. 182 | 183 | Subsequent hooks are called *after* the platform has done any required cloning, so 184 | are free to act asynchronously or synchronously. 185 | 186 | ##### Scenario: custom cloning behaviour 187 | 188 | The initial logic around what messages get cloned does not change - the node 189 | identifies if cloning needs to occur based on how many nodes it is wired to and 190 | how many messages it is asked to send in a single go. 191 | 192 | Rather than do the cloning, the node will set the `cloneMessage` property to indicated 193 | whether cloning should occur or not. 194 | 195 | The actual cloning would happen between the `preRoute` and `preDeliver` steps. 196 | 197 | This would allow a `preRoute` handler to do its own cloning behaviour and then set 198 | `cloneMessage` to `false` so that no further cloning would happen for that message. 199 | 200 | One very important aspect to highlight is that messages will not be cloned until 201 | after the `preRoute` stage - so any modifications of the message will potentially 202 | mutate multiple messages. If earlier handlers want to modify the message in any 203 | way, they will have to honour the `cloneMessage` flag and clone the message themselves. 204 | 205 | 206 | ##### Scenario: remote message routing 207 | 208 | The `preRoute` step can be used to do remote message routing. As no message cloning 209 | has happened by this point, it avoids that overhead when serialising the message 210 | to send over the network serves the same purpose. 211 | 212 | All steps, up to and including `onReceive` will have a way to tell the runtime to 213 | stop processing that message and to not pass it on to any later steps. This would 214 | allow a `preRoute` handler to decide a message must be sent to a remote runtime 215 | and that no further local processes was needed. 216 | 217 | ##### Scenario: flow debugger - breakpoints 218 | 219 | A breakpoint could be triggered when a node sends a message or receives one. 220 | 221 | To break on node send, it would either register a handler on the `onSend` step 222 | (if it was necessary to break before *any* of the messages sent in that one call 223 | are processed) or `preRoute` if it works on a single message. That's a question 224 | for the Flow Debugger design and doesn't need addressing now. However, the options 225 | are there. 226 | 227 | To break on node receive, it would register on `preReceive`. 228 | 229 | In either case, if the breakpoint is triggered, the flow debugger needs to be able 230 | to indefinitely halt the passing of messages. 231 | 232 | This informs us that all of the handlers need to be able to complete asynchronously - 233 | so the flow debugger can defer sending on any messages until it is resumed. It would 234 | also allow the flow debugger to do step-debugging, by registering handlers at every 235 | step. 236 | 237 | ##### Scenario: flow testing 238 | 239 | The flow testing design introduces the ability to: 240 | - verify the contents/structure of a message being passed to a node 241 | - verify the contents/structure of a message send by a node 242 | - stub out an entire node with custom test-case-specific behaviour 243 | 244 | That maps to the steps: 245 | - `onReceive` - verify message passed to a node 246 | - `onSend` - verify the message(s) sent by a node 247 | - `preRoute` - pass the message to a Test Stub node instead of the real node 248 | 249 | 250 | ##### Scenario: custom logging 251 | 252 | Custom handlers added at any of the steps will be able to log whatever information 253 | is needed. 254 | 255 | #### Configuring Hooks 256 | 257 | Hooks will be registered via a runtime API initially. 258 | 259 | In the future, we may add support for adding them via the settings file, however the 260 | focus for this feature is not for end-users to manually configure it. Exposing it 261 | via the settings file at this point in time would set the wrong expectation for 262 | who uses this API. 263 | 264 | 265 | ##### Runtime API 266 | 267 | - `RED.hooks.add('', function(...) { })` 268 | - `RED.hooks.remove('')` 269 | 270 | The `` will be the name of the step to register the hook on - `onSend`, etc. 271 | 272 | It can optionally be suffixed with a label for the hook - `onSend.flow-debugger`. 273 | That label can then be used with `RED.hooks.remove` to remove the handler later on. 274 | 275 | To remove *all* hooks with a given label, `*.flow-debugger` can be used. 276 | 277 | When invoked, the hook handler will be called with a single payload object - the details 278 | of which will be specific to the hook. 279 | 280 | The handler can take an optional second argument - a callback function to call 281 | when the handler has finished its work. 282 | 283 | When the handler finishes its work it must either: 284 | - return normally 285 | - call the callback function with no arguments 286 | - return a promise that resolves 287 | 288 | Any modifications it has made to the payload object will be passed on. 289 | 290 | If the handler wants to stop further processing of the event (for example, the Remote Router 291 | would not want a message to pass on to the local router), it must either: 292 | - return `false` (strictly `false` - not a false-like value) 293 | - call the callback function with `false` 294 | - return a promise that resolves with the value `false`. 295 | 296 | If the handler encounters an error that should be logged it must either: 297 | - throw an Error 298 | - call the callback function with the Error 299 | - return a promise that rejects with the Error 300 | 301 | If a function is defined as the two-argument version (accepting the callback function), 302 | it *must* use that callback - any value it returns will be ignored. 303 | 304 | 305 | ###### Hooks vs Events 306 | 307 | The runtime API already provides `RED.events` where handlers can be registered 308 | for certain runtime events. At a glance, that sounds very similar to the new `RED.hooks` 309 | api. So it is reasonable to ask what the difference is. Here is how we distinguish 310 | them: 311 | 312 | - **Events** - are *notifications* that something has happened. The event handler 313 | cannot have any direct influence on the event. 314 | - **Hooks** - are handlers that are inserted into the code path when something 315 | happens. The hooks can modify the data being passed through and can halt the 316 | onward processing 317 | 318 | 319 | ## Reference 320 | 321 | - https://trello.com/c/J7UDbQVP/66-pluggable-message-routing 322 | - https://github.com/node-red/node-red/wiki/Pluggable-Message-Routing 323 | 324 | ## History 325 | 326 | 327 | - 2020-10-01 - Added to 1.2 release 328 | - 2020-07-24 - Updated to add 'hooks' concept 329 | - 2019-03-27 - Initial proposal submitted 330 | -------------------------------------------------------------------------------- /designs/pluggable-message-routing/message-router-events.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/pluggable-message-routing/message-router-events.png -------------------------------------------------------------------------------- /designs/plugins.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | 6 | # Node-RED Plugins 7 | 8 | ## Summary 9 | 10 | We want to make it easier to extend Node-RED through the use of plugins. 11 | 12 | A plugin is an npm module that is loaded by Node-RED and used to provide custom 13 | functionality to either the runtime and/or editor. 14 | 15 | 16 | ## Authors 17 | 18 | - Nick O'Leary 19 | 20 | ## Details 21 | 22 | A lot can already be done using the model we have for nodes; they are installed 23 | as npm modules, loaded into the runtime and the editor. The runtime manages 24 | their lifecycle. 25 | 26 | Whilst this works for getting custom JavaScript loaded into the editor, we want 27 | to have a bit more structure in place to help plugins do the right thing. 28 | 29 | The main points this design needs to cover are: 30 | 31 | - An editor API that plugins use to register themselves - similar to 32 | `RED.nodes.registerType` used by nodes. This API would then let the editor 33 | manage the plugin lifecycle 34 | 35 | - We know there are cases like nrlint where the core linter plugin will want 36 | to load its own plugins. 37 | 38 | - Loading additional resources - a plugin might not want to provide all of its 39 | content in the single .html file. We've long thought about having a way for 40 | nodes to declare additional resources that it will want to load in the editor. 41 | The same is true for plugins. 42 | 43 | - If a plugin is runtime-only, it should not be required to provide a .js file 44 | 45 | - **Question:** should it be possible to dynamically remove/disable a plugin as you can nodes? That would 46 | require *all* UI customisations to be undoable. 47 | If plugins are there in order to provide customisations of the editor, it may not be practical to 48 | fully support removing them without a reload of the editor. 49 | This will require more work to figure out whilst implementing some real plugins. 50 | 51 | 52 | ### Plugin registration 53 | 54 | The following applies to both editor and runtime plugins. 55 | 56 | ``` 57 | RED.plugins.registerPlugin("plugin-identifier", { 58 | type: "plugin-type", 59 | onadd: function() {}, 60 | onremove: function() {} 61 | 62 | }) 63 | ``` 64 | 65 | - `plugin-identifier`: a unique identifier for the plugin. 66 | - `type`: an optional field to identify the type of the plugin (see below) 67 | - `onadd`: called when the plugin is registered 68 | - `onremove`: called when the plugin is removed due to the module being disabled or uninstalled. 69 | 70 | When this register function is called, the following things will happening 71 | 72 | - The plugin is added to the internal plugin registry 73 | - If provided, the plugin's `onadd` function is called 74 | - The `"registry:plugin-added"` event is emitted with the name of the plugin as its payload 75 | 76 | The `onremove` function is called when the plugin is removed due to the module being 77 | disabled or uninstalled. *TODO*: not clear if we'll support removing plugins dynamically. 78 | 79 | 80 | 81 | ### Loading plugins 82 | 83 | Plugins are loaded from node modules in the same way as nodes are. 84 | 85 | However, there is a decision to be made about whether the same `package.json` structure 86 | should be used, or if it should identify plugins explicitly. For example: 87 | 88 | ``` 89 | "node-red": { 90 | "plugins": { 91 | "foo": "foo.js" 92 | } 93 | } 94 | ``` 95 | 96 | If the plugin only provides editor resources, the entry can point to just an HTML 97 | file. 98 | 99 | One advantage of this is it will be ignored by older versions of Node-RED - so installing 100 | a plugin won't break things. 101 | 102 | It would also allow the runtime to provide plugin html content to the editor separately 103 | to the node html content. This means plugins can be loaded into the runtime before 104 | it loads the palette of nodes. 105 | 106 | This does mean a new admin api will be needed in order to serve the plugins. 107 | 108 | ### Settings 109 | 110 | Plugins provide a way to custom and extended both the Node-RED editor and runtime. 111 | 112 | It may not always be a good thing to allow a user to install random plugins. For example, 113 | when Node-RED is embedded into another application, such as the commercial adopters. 114 | 115 | In those scenarios, they may want to take advantage of plugins to provide customisations, 116 | but not allow the end user to install/remove those plugins. 117 | 118 | For this feature then, there are some use cases to consider: 119 | 120 | - enable/disable loading of plugins entirely 121 | - disable dynamic loading of plugins - only those loaded at startup are loaded 122 | - equivalent of `nodeExcludes`/`nodeIncludes` to identify specific modules that 123 | either may or may not be loaded. This applies beyond just plugins - we've 124 | needed a way to allow/deny modules by name for some time. 125 | 126 | The details of this are being [discussed here](https://github.com/node-red/designs/discussions/40). 127 | 128 | ### Plugin Settings 129 | 130 | A plugin may expect some settings to be provided via `settings.js`. As it stands, 131 | a runtime plugin has full access to the `RED.settings` object so will be able to 132 | access anything in there. 133 | 134 | There is a question over how an editor plugin is able to access settings. 135 | 136 | We already provide a [mechanism for nodes to do this](https://nodered.org/docs/creating-nodes/node-js#exposing-settings-to-the-editor) 137 | when their runtime code calls `RED.nodes.registerType`. 138 | 139 | Plugins should use a similar mechanism for consistency. Although there is a question 140 | of whether it can be improved to provide some more flexibility. 141 | 142 | The key requirement is how to tell the runtime to make certain settings available 143 | to the editor. 144 | 145 | To help organise the settings file, it is probably better to group plugin settings 146 | together. 147 | 148 | Lets say the plugin with id 'my-plugin' has three settings - 'color', 'shape' and 'size' 149 | 150 | With the existing node setting scheme, you'd end up with three settings `myPluginColor`, 151 | `myPluginShape` and `myPluginSize`. 152 | 153 | It would be cleaner to have a single top-level property that matches the plugin id 154 | and then group the settings under that: 155 | ``` 156 | "my-plugin": { 157 | color: ... 158 | shape: ... 159 | size: 160 | } 161 | ``` 162 | 163 | the `settings` property in the plugin definition could then look like: 164 | ``` 165 | settings: { 166 | color: { value: "red", exportable: true}, 167 | shape: { value: "square", exportable: true}, 168 | size: { value: "big", exportable: true}, 169 | } 170 | ``` 171 | 172 | However, this does require the plugin to pre-declare all possible settings. In the 173 | case of something like `nrlint`, the single top-level setting may used to collect 174 | settings of sub-plugins as well. 175 | 176 | So the following would be supported: 177 | 178 | ``` 179 | settings: { 180 | "*": { exportable : true}, 181 | "color": { exportable: false} 182 | } 183 | ``` 184 | 185 | This will cause all settings under the plugin name to be exported, but also allow 186 | for individual settings to be held back. 187 | 188 | Another example: 189 | ``` 190 | settings: { 191 | "editor": { exportable : true}, 192 | "runtime": { exportable: false} 193 | } 194 | ``` 195 | 196 | would allow for a clear separation of settings for the runtime and editor. 197 | 198 | We should consider updating the node settings naming to allow for this pattern 199 | as well. 200 | 201 | ### Plugin Resources 202 | 203 | Plugins may want to provide additional resources to the editor, such as extra JS, 204 | CSS or images. To remove the need for them to create http endpoints, we will introduce 205 | a new way for both Plugins and Nodes to provide such content. 206 | 207 | In the same manner as `icons` and `examples`, if there is a directory called `resources` 208 | at the root of the package tree, the runtime will automatically expose that directory under 209 | the path: 210 | 211 | ``` 212 | /resources//path-to-resource 213 | ``` 214 | 215 | For example, if a plugin called `test-plugin-module` has the file structure: 216 | 217 | ``` 218 | |- package.json 219 | |- plugin.js 220 | |- plugin.html 221 | \- resources 222 | |- script.js 223 | \- css 224 | \- style.css 225 | ``` 226 | 227 | The resource files will be accessible via the paths: 228 | 229 | - `/resources/test-plugin-module/script.js` 230 | - `/resources/test-plugin-module/css/style.css` 231 | 232 | 233 | ### HTTP Admin API 234 | 235 | - `GET /plugins` 236 | - `Content-Type: application/json` - returns a list of the loaded plugins (see below for format) 237 | - `Content-Type: text/html` - returns the html content of all loaded plugins 238 | - `GET /plugins/messages` - returns the message catalogs for all loaded plugins 239 | 240 | TODO: need to figure out what other admin endpoints are needed. 241 | 242 | ### `@node-red/runtime` API 243 | 244 | The following functions will be added to the external runtime API: 245 | 246 | - `plugins.getPluginList` - returns a promise that resolves to the list of loaded plugins (see below for format) 247 | - `plugins.getPluginConfigs` - returns a promise that resolves to the HTML content for loaded plugins 248 | - `plugins.getPluginCatalogs` - returns a promise that resolves to the message catalogs for all loaded plugins 249 | 250 | The following functions will be added to the internal runtime API: 251 | 252 | - `RED.plugins.registerPlugin` - register a new runtime plugin 253 | - `RED.plugins.getPlugin` - get a plugin by id 254 | - `RED.plugins.getPluginsByType` - get all plugins of a given type 255 | - `RED.plugins.getPluginList` - get a list of the loaded plugins (see below for format) 256 | - `RED.plugins.getPluginConfigs` - get the message catalogs for all loaded plugins 257 | 258 | These functions all pass through to the `@node-red/registry` module... 259 | 260 | 261 | ### `@node-red/registry` API 262 | 263 | - `registerPlugin` - register a new runtime plugin 264 | - `getPlugin` - get a plugin by id 265 | - `getPluginsByType` - get all plugins of a given type 266 | - `getPluginList` - get a list of the loaded plugins (see below for format) 267 | - `getPluginConfigs` - get the message catalogs for all loaded plugins 268 | 269 | ### Node/Plugin Runtime API 270 | 271 | The `RED` object passed to nodes/plugins will now include a `plugins` object with the functions: 272 | - `registerPlugin` - register a new runtime plugin 273 | - `getPlugin` - get a plugin by id 274 | - `getPluginsByType` - get all plugins of a given type 275 | 276 | 277 | #### Plugin List format 278 | 279 | This follows the same basic structure as the node list format: 280 | 281 | ``` 282 | [ 283 | { 284 | "id": "test-plugin/test", 285 | "name": "test", 286 | "enabled": true, 287 | "local": true, 288 | "module": "test-plugin", 289 | "plugins": [ 290 | { 291 | "type": "foo", 292 | "id": "my-test-plugin", 293 | "module": "test-plugin" 294 | } 295 | ], 296 | "editor": true, 297 | "runtime": true, 298 | "version": "1.0.0" 299 | } 300 | ] 301 | ``` 302 | 303 | Each entry in the array represents a separate set of plugins - ie a separate entry in 304 | the `package.json`. The properties are: 305 | 306 | - `id` - the module/set 307 | - `name` - the set name 308 | - `enabled` - whether the plugin is enabled (*TODO* this is a hangover from nodes. Not sure if we'll 309 | support dynamic enable/disable of nodes) 310 | - `local` - whether the module is installed in `~/.node-red` or not - only local modules can be removed. (*TODO* another hangover 311 | from nodes) 312 | - `module` - the module name 313 | - `plugins` - an array of the plugins provided by this set 314 | - `editor` - boolean flag to say if this is an editor plugin (ie, provides an html file) 315 | - `runtime` - boolean flag to say if this is a runtime plugin (ie, provides a js file) 316 | - `version` - npm module version 317 | 318 | ### Editor APIs 319 | 320 | `RED.plugins` will be a new module in the Editor. It will expose the functions: 321 | 322 | - `RED.plugins.registerPlugin(id,definition)` - called by plugins to register themselves 323 | - `RED.plugins.getPlugins(id)` - get a plugin by identifier 324 | - `RED.plugins.getPluginsByType(type)` - get all plugins of a particular type (using the optional `type` field in the plugin definition) 325 | 326 | TODO: there may be other APIs to expose 327 | 328 | ### Editor customisations 329 | 330 | For plugins that add to the UI, we need to identify what UI elements they may want 331 | to extend and have a plan in place to have proper APIs in place for them. 332 | 333 | - Adding a custom sidebar - already available 334 | - Adding custom menu items - the `RED.menu` api needs overhauling to allow easier customisation of the main menu 335 | - Adding library types - this will be covered in a separate design note on the improved library api 336 | - Adding custom widgets to the workspace footer - work in progress, need to formalise the existing API 337 | 338 | 339 | ### Example plugin - nrlint 340 | 341 | *I am using nrlint as an example to demonstrate how it could work. This is not meant to be the formal design for nrlint's plugin system. All of the names/types/ids I've used here are placeholders* 342 | 343 | - The Flow Linter tool will register a plugin called `nrlint`. 344 | - It's `onadd` function will do the following: 345 | - add a custom sidebar to the editor and all the logic to do its work. 346 | - it will listen for the `registry:plugin-added` event so it knows when other plugins are added. 347 | When that is called, it will check if the new plugin has a `type` of `nrlint-rule`. 348 | - It will also call `RED.plugins.getByType("nrlint-rule")` to get a list of existing plugins. 349 | - For the existing plugins, or newly added ones, it can then do whatever internal 350 | initialisation/registration it needs of those plugins into the lint tool. 351 | 352 | 353 | 354 | 355 | 356 | ## History 357 | 358 | - 2020-12-xx - Initial proposal submitted 359 | -------------------------------------------------------------------------------- /designs/pouchdb-context-plugin.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # PouchDB Context store plugin 6 | 7 | The PouchDB Context store plugin holds context data in the PouchDB. 8 | 9 | ## Summary 10 | 11 | This plugin uses the PouchDB database for all context scopes. 12 | PouchDB is an abstraction layer and the database to use can be selected by the adapter. 13 | In addition to the databases that PouchDB supports as standard, you can apply the NodeSQLite adapter to store your data in SQLite. 14 | 15 | PouchDB Adapters: https://pouchdb.com/adapters.html 16 | 17 | ## Author 18 | - @KazuhiroIto 19 | 20 | ## Details 21 | 22 | ### Install 23 | 24 | 1. Run the following command in your Node-RED user directory - typically `~/.node-red` 25 | 26 | npm install git+https://github.com/node-red/node-red-context-pouchdb 27 | 28 | 2. Add a configuration in settings.js: 29 | 30 | ```javascript 31 | contextStorage: { 32 | pouchdb: { 33 | module: require("node-red-context-pouchdb"), 34 | config: { 35 | // see below options 36 | } 37 | } 38 | } 39 | ``` 40 | 41 | ### Options 42 | 43 | | Options | Description | 44 | | -------- | ----------------------------------------------------------------------------- | 45 | | name | Specifies the `name` argument for creating the PouchDB databse. You can specify the following.
- SQLite : Database storage file path
- LevelDB: Database storage folder path
- CouchDB: Database URL
`default(SQLite): settings.userDir/context/context.db` | 46 | | options | Specifies the PouchDB database `options`.
Example
- SQLite : {adapter: 'websql'}
- LevelDB: {adapter: 'leveldb'} or {}
- CouchDB: {}
`default(SQLite): {adapter: 'websql'}`| 47 | 48 | Reference: [PouchDB Create a database options](https://pouchdb.com/api.html#create_database) 49 | 50 | ### Data Model 51 | 52 | - This plugin uses a PouchDB database for all context scope. 53 | - The NodeSQLite adapter is added to the PouchDB adapter. You can save the data in SQLite3. 54 | - You can also specify saving to a database (LevelDB, CouchDB) that PouchDB supports as standard. 55 | - This plugin saves a JSON object of keys and values in a document for each scope. 56 | - The keys of `global context` will be id with `global` . 57 | - The keys of `flow context` will be id with `` . 58 | - The keys of `node context` will be id with `` . 59 | - Context data is stored in `doc.data` as a document for each scope. 60 | 61 | Structure of data stored in PouchDB: 62 | ```json 63 | { 64 | "total_rows": 3, 65 | "offset": 0, 66 | "rows": [ 67 | { 68 | "id": "2052fca8.312154:a77d79a4.d1a908", 69 | "key": "2052fca8.312154:a77d79a4.d1a908", 70 | "value": { 71 | "rev": "6-55e0513ffba64a8b8efec1ba8e43c90f" 72 | }, 73 | "doc": { 74 | "data": { 75 | "NODE-KEY-1": "NODE-DATA-1", 76 | "NODE-KEY-2": "NODE-DATA-2" 77 | }, 78 | "_id": "2052fca8.312154:a77d79a4.d1a908", 79 | "_rev": "6-55e0513ffba64a8b8efec1ba8e43c90f" 80 | } 81 | }, 82 | { 83 | "id": "a77d79a4.d1a908", 84 | "key": "a77d79a4.d1a908", 85 | "value": { 86 | "rev": "61-2c2a457388db1c3859b79e4bb62e9375" 87 | }, 88 | "doc": { 89 | "data": { 90 | "FLOW-KEY-1": "FLOW-DATA-1", 91 | "FLOW-KEY-2": "FLOW-DATA-2", 92 | }, 93 | "_id": "a77d79a4.d1a908", 94 | "_rev": "61-2c2a457388db1c3859b79e4bb62e9375" 95 | } 96 | }, 97 | { 98 | "id": "global", 99 | "key": "global", 100 | "value": { 101 | "rev": "73-ee260387e51ae20132076ccc83957600" 102 | }, 103 | "doc": { 104 | "data": { 105 | "GLOBAL-KEY-1": "GLOBAL-DATA-1", 106 | "GLOBAL-KEY-2": "GLOBAL-DATA-2", 107 | }, 108 | "_id": "global", 109 | "_rev": "73-ee260387e51ae20132076ccc83957600" 110 | } 111 | } 112 | ] 113 | } 114 | ``` 115 | 116 | ### Data Structure 117 | 118 | - Data is saved in the JSON object format supported by PouchDB. The plugin does not convert JSON data to a string for storage. 119 | 120 | Code example that references database data : 121 | ```javascript 122 | var pd = require('pouchdb'); 123 | pd.plugin(require('pouchdb-adapter-node-websql')); 124 | var db; 125 | 126 | db = new pd( "/home/user/.node-red/context/context.db", { adapter: 'websql' }); 127 | db.allDocs({include_docs: true}, function(err, doc) { 128 | if (err) { 129 | return console.log(err); 130 | } else { 131 | var data = JSON.stringify(doc,null,4); 132 | console.log(data); 133 | } 134 | }); 135 | ``` 136 | 137 | ### Database replication 138 | 139 | - The data in the context store can be replicated to other DBs using the PouchDB feature. 140 | This allows you to back up the data stored in your local SQLite to a remote CouchDB.This allows you to back up context data stored in your local SQLite to a remote CouchDB. 141 | 142 | - In an environment with multiple context stores, only contexts using the PouchDB plugin will be backed up. 143 | 144 | Code example of replication to remote database(CouchDB) : 145 | ```javascript 146 | var pd = require('pouchdb'); 147 | pd.plugin(require('pouchdb-adapter-node-websql')); 148 | 149 | var source = "/home/user/.node-red/context/context.db"; 150 | var target = "http://localhost:5984/couchdb_mycouchdb_1"; 151 | 152 | var db_source = new pd(source, { adapter: 'websql' }); 153 | var db_target = new pd(target); 154 | 155 | db_source.replicate.to(db_target) 156 | .on('complete', function () { 157 | console.log ("Database replicated."); 158 | }).on('error', function (err) { 159 | console.log(err); 160 | }); 161 | ``` 162 | Reference: [PouchDB replication](https://pouchdb.com/api.html#replication) 163 | 164 | - Replication can also be filtered.You can also consider replicating a partial database (for example, only the global context part). 165 | 166 | Code example of replication filtering in global context : 167 | ```javascript 168 | db_source.replicate.to(db_target, { 169 | filter: function (doc) { 170 | return doc._id === 'global'; 171 | }}) 172 | ``` 173 | Reference: [PouchDB filtered replication](https://pouchdb.com/api.html#filtered-replication) 174 | 175 | ## History 176 | 177 | - 2021-03-03 - Initial Design Note 178 | - 2021-06-15 - Added database replication and support standard plugin databases. 179 | -------------------------------------------------------------------------------- /designs/runnable-projects.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Runnable Projects 6 | 7 | ## Summary 8 | 9 | One of the goals of the Projects feature is to treat a project as a deployable artefact. 10 | 11 | It should be possible to deploy a project without using the editor. For example: 12 | 13 | ``` 14 | 1. git clone https://example.com/my-node-red-project.git 15 | 2. cd my-node-red-project 16 | 3. npm install 17 | 4. npm start -- --credentialSecret="my-secret-key" 18 | ``` 19 | 20 | 21 | ## Authors 22 | 23 | - @knolleary 24 | - @dceejay 25 | 26 | ## Details 27 | 28 | **Note:** Added a branch `runnable-project` to experiment - https://github.com/node-red/node-red/tree/runnable-project. Currently works as described below... but maybe not final design choices. 29 | 30 | We can't quite do this today. Here are the missing pieces: 31 | 32 | 1. the project's `package.json` needs to list `node-red` as a dependency. On balance, I think this is the right approach, rather than treat node-red as an assumed prerequisite. I don't think we should do this be default; maybe add a checkbox in the project settings' dependencies page to include `node-red`. 33 | > DCJ: Yes - agree it should be a dependency. Currently not hard to add manually, but could be even easier. It would be a prerequisite dependency for inside a Docker container. 34 | 35 | >DCJ: Added a button to dependencies panel to add node-red core - As it's not "known" it shows as greyed in the list- but I actually think this is good, as it indicates it may not be required also. 36 | 37 | >NOL: Not sure a standalone button passes the "why is it there?" test. 38 | 2. it must be possible to point node-red at a project on start-up, without using the editor to do so. Currently we overload the flowFile argument to set the active project - but that only works if the project is 'known'. It would be better to point at a `projectDir` wherever it may exist. 39 | > DCJ: Are we pointing Node-RED or npm start ? Which are we calling to run it ? If npm start then don't we already have to be in the correct project directory ? (or use npm start --prefix projectDir) 40 | 41 | > DCJ: Having played with it a while - I think `npm start` from within the relevant directory is the way to go to start with. 42 | 3. it must be possible to provide the credentialSecret for the project without it being part of any of the version controlled files. Options for this include: 43 | - env var - `NODE_RED_CREDENTIAL_SECRET` 44 | - [DCJ] easy to add to settings.js - `credentialSecret: process.env.NODE_RED_CREDENTIAL_SECRET` and would suit Docker style deploy. 45 | - command-line flag - `--credentialSecret="..."` 46 | - [DCJ] added as new command line option to runnable-project dev branch 47 | >DCJ/NOL: a) If environment variable works do we need command line option ? When would you use it? b) should using env var be baked into core or just via settings file so can be removed if required. 48 | 49 | 4. Must handle running in a read-only environment. If the package is installed globally (in order to insert a command link into /usr/bin) then the flow file and settings.js will be in root space - which ought not to be writable by the user. Currently even starting Node-RED with -u option pointing somewhere protected will fail. Options include 50 | - just handling the error - i.e. log something (useful) - but fail to start. 51 | - on detecting this condition, marking the settings.readOnly as true thus stopping the library failing - but run. 52 | - setting disableEditor to `true` also. 53 | 54 | The package would provide an npm start script that runs node-red with the appropriate command-line args. 55 | > DCJ: currently just using `node-red -u . flow.json` - i.e. start Node-RED using the current directory as the base and the current flow file. 56 | 57 | >NOL: instead should it use (say) a -p option to specify a package.json file to use instead to read the extra settings from (like flow file etc) 58 | 59 | It could also provide a `bin` section that could allow starting via command line if installed globally - but that requires a command to be run (that doesn't take parameters) so in order to do that we would also need to create that script/.js file or provide a template. 60 | 61 | > DCJ: Actually by pre-req node-red it automatically adds the node-red command to the node_modules/.bin so the package.json start script can just call `node-red` and all is good. 62 | 63 | ### Runtime settings 64 | 65 | When deploying the project, there will also be a need to have the runtime settings provided. These are not normally part of a project generated by Node-RED. The question is whether we have some tooling support to generate, or we just document how to do it. For example, the user would manually add a `settings.js` file to their project repo. 66 | > DCJ: If we have the "add node-red as a dependency" button - could it not also have a ... "copy default settings.js into project" button or create a minimal settings.js (i.e. without defaults and comments) ? (but ensuring flowFile doesn't clash) e.g. 67 | 68 | ``` 69 | module.exports = { 70 | uiPort: process.env.PORT || 1880, 71 | credentialSecret: process.env.NODE_RED_CREDENTIAL_SECRET 72 | } 73 | ``` 74 | 75 | ### Dockerfile 76 | 77 | We should also have an example Dockerfile that can be used to build and run a project. 78 | 79 | > DCJ: Here is an example *Dockerfile.template* that works with various flavours of Docker - note that the flow file is set using the start command in package.json, and it needs node red as a dependency) 80 | 81 | "scripts": {"start": "node-red -u . flow.json"}, 82 | 83 | ``` 84 | # Expects -e NODE_RED_CREDENTIAL_SECRET="my_secret_key" to be part of the docker run command at start time 85 | # and settings.js to exist as per above 86 | # 87 | # Note the node:slim image doesn't have node-gyp 88 | 89 | ## Uncomment one of the following FROM lines to suit your environment 90 | # For generic servers 91 | #FROM node:8-slim 92 | # For Pi... 93 | FROM arm32v7/node:8-slim 94 | # For Resin.io 95 | #FROM resin/%%RESIN_MACHINE_NAME%%-node:8-slim 96 | 97 | # Uncomment the next three lines if you want GPIO for Pi 98 | #RUN apt-get update && apt-get install -yq \ 99 | # rpi.gpio && \ 100 | # apt-get clean && rm -rf /var/lib/apt/lists/* 101 | 102 | WORKDIR /usr/src/app 103 | 104 | COPY package.json package.json 105 | 106 | RUN JOBS=MAX npm install --production --unsafe-perm --no-optional && npm cache clean --force && rm -rf /tmp/* 107 | 108 | COPY . ./ 109 | 110 | # required for Resin.io 111 | ENV INITSYSTEM on 112 | # useful for pigpiod node to be able to talk to daemon on host. 113 | ENV HOST_IP=172.17.0.1 114 | ENV NODE_PATH=/usr/src/app/node_modules 115 | 116 | EXPOSE 1880 117 | 118 | CMD ["npm", "start"] 119 | ``` 120 | 121 | A really minimal Dockerfile to build an image (but not run) could be as simple as 122 | 123 | ``` 124 | FROM node:8-onbuild 125 | VOLUME /root/.node-red 126 | EXPOSE 1880 127 | ``` 128 | 129 | 130 | 131 | 132 | ## History 133 | 134 | - 2019-02-27 - migrated from Design note wiki 135 | -------------------------------------------------------------------------------- /designs/subflow-node-modules.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Subflow Node Modules 6 | 7 | ## Summary 8 | 9 | The goal of this feature is to allow a subflow to be packaged as an installable 10 | npm module and loaded into the runtime. 11 | 12 | It will appear in the palette in the same way as any regular node; the end-user 13 | will not need to know the node is implemented as a subflow. 14 | 15 | Node-RED Nodegen should be updated to supported creating such a module. 16 | 17 | Topics this design must cover: 18 | 19 | - how the runtime registry knows a module provides a subflow type node 20 | - how the module declares its node dependencies 21 | - how the node's help template and edit template are provided 22 | 23 | ## Authors 24 | 25 | - @knolleary 26 | 27 | ## Details 28 | 29 | ## History 30 | 31 | - 2019-02-26 - migrated from Design note wiki 32 | -------------------------------------------------------------------------------- /designs/subflow-property-ui/README.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: in-progress 3 | --- 4 | 5 | ## Subflow Property UI 6 | 7 | ### Summary 8 | 9 | 0.20 introduces subflow instance properties. They are exposed as a list of key/value 10 | pairs in the subflow edit dialog. 11 | 12 | There is a requirement to be able to generate a richer UI interface for the user 13 | to set these properties. 14 | 15 | 16 | ### Authors 17 | 18 | - @HiroyasuNishiyama 19 | 20 | ### Details 21 | 22 | ### UI definition 23 | 24 | Next figure shows a process to define new parameter settings UI for a SUBFLOW. 25 | 26 | ![Subflow-dev-process](images/Subflow-dev-process.png) 27 | 28 | **UI Edit Panel** is added to **SUBFLOW Template Edit Panel** of a SUBFLOW. Simple parameter setting UI for the SUBFLOW can be created using this panel by adding UI items on UI items list. Created UI can be previewed by clicking **preview** button or selecting preview tab. 29 | 30 | Parameter settings UI of a SUBFLOW can be show by double clicking the SUBFLOW (and selecting parameter edit tab) on flow edit pane. SUBFLOW customization parameters can be set using this UI. 31 | 32 | #### SUBFLOW Parameter Representation 33 | 34 | Each UI item defined in UI Edit Panel can store the defined value into two kinds of targets: SUBFLOW environment variable and node property within the SUBFLOW. 35 | 36 | #### Edit UI Panel 37 | 38 | **Edit UI Panel** have **add** button at the bottom. By clicking this button, new UI item can be added to UI items list. UI items are shown from top to bottom in generated parameter settings UI. Order of UI items can be rearragned by dragging an item in the list. 39 | 40 | ![UI-edit-panel](images/UI-edit-panel.png) 41 | 42 | UI item definition consists of following sub-items: 43 | 44 | ![UI-element](images/UI-element01.png) 45 | 46 | - **name** - name of defined UI item. Used to access input value. 47 | 48 | - **label** - label of defined UI item. HTML tags can be used in label definition. For internationalization (i18n) of label text, locale of the text can be selected from menu. 49 | 50 | **label** field appearance is as follows: 51 | 52 | ![field-label](images/field-label.png) 53 | 54 | - **type** - type of defined item. Currently following types are supported: 55 | 56 | | type | input item | description | 57 | | ----------------- | --------------------------------------------------- | ------------------------------------------------------------ | 58 | | *simple type* | - initial value | simple input for basic types supported by TypedInput such as string, number, etc. | 59 | | **any** | - selection of candidate types
- initial value | set of basic types supported by TypedInput | 60 | | **label** | - | only show label | 61 | | **menu** | - menu items
- initial value | select one item from list of menu items | 62 | | **checkbox** | - label
- initial value | checkbox to select from items | 63 | | **spinner** | - initial value | spinner input of numeric value | 64 | | **node** | - target node | select from set of nodes | 65 | | **node property** | - target property of a node | select from set of node properties | 66 | | **ui_group** | - group of UI widget | select group of Node-RED Dashboard | 67 | | **ui_size** | - size of UI widget | set size of Node-RED Dashboard UI Widget | 68 | 69 | **type** field appearance is as follows: 70 | 71 | ![field-type](images/field-type.png) 72 | 73 | - **tgt. type** - type of target to which input value is stored. Curtly supported tgt. types are as follow: 74 | 75 | | name | description | 76 | | ------------- | --------------------------------------- | 77 | | env var | environment variable | 78 | | node property | property of a specified node in SUBFLOW | 79 | 80 | There are some kind of UI input types of nodes in SUBFLOW that can not be specified using environment variable. For such input type, we allow directly specifying property of nodes from node settings UI. 81 | 82 | **tgt. type** field appearance is as follows: 83 | 84 | ![field-tgt](images/field-tgt.png) 85 | 86 | - **target** - specification of target for selected **tgt. type**. 87 | 88 | If **tgt. type** is **env var**, **target** field appearance is as follows: 89 | 90 | ![field-target-envvar](images/field-target-envvar.png) 91 | 92 | If **tgt. type** is **node property**, **target** field appearance is as follows: 93 | 94 | ![field-target-target](images/field-target-target.png) 95 | 96 | - **type specific inputs** - input items specific to each UI item type. 97 | 98 | #### UI Input Elements 99 | 100 | This section describes UI definition, UI appearance, and environment variable defined by UI element for each UI input element type. 101 | 102 | ##### *Simple Type* 103 | 104 | - UI Definition 105 | 106 | ![UI-def-simple](images/UI-def-simple.png) 107 | 108 | - **initial value** accepts a value of type specified by **** field. 109 | 110 | - UI Appearance 111 | 112 | ![UI-app-simple](images/UI-app-simple.png) 113 | 114 | - Environment Variable 115 | 116 | | name | value | 117 | | ----------- | ------------------------------------------------------------ | 118 | | *name* | input value represented by specified type | 119 | | *name*_type | type of input value | 120 | | *name*_info | JavaScript object that represents input information. It contains following properties:
- name: name of field
- label: label of field
- value: input value
- type: type of UI element
- target_type: “env var” or “node property”
- target: name or [id, name] | 121 | 122 | ##### **any** 123 | 124 | - UI Definition 125 | 126 | ![UI-def-any](images/UI-def-any0.png) 127 | 128 | - A set of candidate types can be selected by checkbox. 129 | - Initial value can be defined using **initial value** field. 130 | 131 | - UI Appearance 132 | 133 | ![UI-app-any](images/UI-app-any.png) 134 | 135 | - A value can be input using **TypedInput** interface. 136 | 137 | - Environment Variable 138 | 139 | same as *simple type* 140 | 141 | ##### **label** 142 | 143 | - UI Definition 144 | 145 | ![UI-def-label](images/UI-def-label.png) 146 | 147 | - UI Appearance 148 | 149 | ![UI-app-label](images/UI-app-label.png) 150 | 151 | - Environment Variable 152 | 153 | | name | value | 154 | | ----------- | ------------------------------------------------------------ | 155 | | *name*_type | “label” | 156 | | *name*_info | JavaScript object that represents input information. It contains following properties:
- name: name of field
- label: label of field
- type: “label" | 157 | 158 | ##### **menu** 159 | 160 | - UI Definition 161 | 162 | ![UI-def-menu](images/UI-def-menu.png) 163 | 164 | - menu items are defined by adding *item label* (with locale) and its *value* pair. 165 | - initial value of menu can be defined using **initial value** menu. 166 | 167 | - UI Appearance 168 | 169 | ![UI-app-menu](images/UI-app-menu.png) 170 | 171 | - Environment Variable 172 | 173 | | name | value | 174 | | ----------- | ------------------------------------------------------------ | 175 | | *name* | *value* of selected item | 176 | | *name*_type | "menu" | 177 | | *name*_info | JavaScript object that represents input information. It contains following properties:
- name: name of field
- label: label of field
- value: value of selected item
- type: “menu"
- target_type: “env var”, “node”, or “node property”
- target: name or [id, name]
- items: list of objects with following properties:
+ labels: list of objects with following properties:
* lang: locale of label
* label: label
+ value: value of item | 178 | 179 | ##### **checkbox** 180 | 181 | - UI Definition 182 | 183 | ![UI-def-checkbox](images/UI-def-checkbox.png) 184 | 185 | - initial value (**true** or **false**) can be defined by **initial value** field. 186 | 187 | - UI Appearance 188 | 189 | ![UI-app-checkbox](images/UI-app-checkbox.png) 190 | 191 | - item label is shown after check box. 192 | 193 | - Environment Variable 194 | 195 | | name | value | 196 | | ----------- | ------------------------------------------------------------ | 197 | | *name* | **true** or **false** | 198 | | *name*_type | “checkbox” | 199 | | *name*_info | JavaScript object that represents input information. It contains following properties:
- name: name of field
- label: label of field
- value: **true** or **false**
- type: “checkbox"
- target_type: “env var” or “node property”
- target: name or [id, name] | 200 | 201 | ##### spinner 202 | 203 | - UI Definition 204 | 205 | ![UI-def-spinner](images/UI-def-spinner.png) 206 | 207 | - initial value can be defined by **initial value** field. 208 | 209 | - UI Appearance 210 | 211 | ![UI-app-spinner](images/UI-app-spinner.png) 212 | 213 | - Environment Variable 214 | 215 | | name | value | 216 | | ----------- | ------------------------------------------------------------ | 217 | | *name* | numeric value | 218 | | *name*_type | “spinner” | 219 | | *name*_info | JavaScript object that represents input information. It contains following properties:
- name: name of field
- label: label of field
- value: numeric value
- type: “spinner"
- target_type: “env var” or “node property”
- target: name or [id, name] | 220 | 221 | ##### node 222 | 223 | - UI Definition 224 | 225 | ![UI-def-node](images/UI-def-node.png) 226 | 227 | - **filter** specifies a regular expression that filters node type. 228 | 229 | - UI Appearance 230 | 231 | ![UI-app-node](images/UI-app-node.png) 232 | 233 | - environment variable 234 | 235 | | name | node ID | 236 | | ----------- | ------------------------------------------------------------ | 237 | | *name*_type | "node" | 238 | | *name*_info | JavaScript object that represents input information. It contains following properties:
- name: name of field
- label: label of field
- value: numeric value
- type: “node"
- target_type: “env var” or “node property”
- target: name or [id, name] | 239 | 240 | ##### node property 241 | 242 | *T.B.D.* (will be implemented as an extension of **TypedInput** interface) 243 | 244 | ## History 245 | 246 | - 2019-02-27 - migrated from Design note wiki 247 | -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/Subflow-dev-process.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/Subflow-dev-process.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-app-any.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-app-any.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-app-checkbox.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-app-checkbox.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-app-label.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-app-label.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-app-menu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-app-menu.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-app-node.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-app-node.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-app-simple.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-app-simple.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-app-spinner.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-app-spinner.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-def-any0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-def-any0.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-def-checkbox.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-def-checkbox.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-def-label.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-def-label.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-def-menu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-def-menu.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-def-node.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-def-node.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-def-simple.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-def-simple.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-def-spinner.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-def-spinner.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-edit-panel.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-edit-panel.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/UI-element01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/UI-element01.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/field-label.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/field-label.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/field-target-envvar.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/field-target-envvar.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/field-target-target.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/field-target-target.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/field-tgt.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/field-tgt.png -------------------------------------------------------------------------------- /designs/subflow-property-ui/images/field-type.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/node-red/designs/788b74442d38469d8e8311dc23f2a23ba4210fb1/designs/subflow-property-ui/images/field-type.png -------------------------------------------------------------------------------- /designs/timeout-api.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft 3 | --- 4 | 5 | # Node Timout API 6 | 7 | ## Summary 8 | 9 | This design looks at how to add a timeout mechanism to nodes. This will allow 10 | flows to be created that can handle hangs or other unexpected events that prevent 11 | the normal operation of a flow. 12 | 13 | ## Authors 14 | 15 | - @knolleary 16 | 17 | ## Details 18 | 19 | This design builds on the new [Messsaging API design](node-messaging-api) that 20 | introduces the `node.done` function for a node to indicate it has completed 21 | handling a message. 22 | 23 | It will be possible for the runtime to timeout a node that does not complete its 24 | processing of a message within a given time. But this can only work if the runtime 25 | knows a node has been updated to call `node.done`. 26 | 27 | **By default, a Node cannot be timed out** 28 | 29 | It will be up to individual implementations to decide if it makes sense to add 30 | timeout handling. Some nodes, such as the Change and Switch nodes do not perform 31 | any long-running activity, so would not make sense to enable timeout handling in 32 | them. 33 | 34 | To enable timeout handling a node must add an event handler for the `timeout` event: 35 | 36 | this.on('timeout', function(msg) { 37 | // Process this timeout. For example, cancel any in-flight work 38 | }) 39 | 40 | By adding this handler, the node is telling the runtime it can be timed out and, 41 | therefore, that the node *will* call `node.done(msg)` when it finishes with each 42 | message. 43 | 44 | The runtime will track the messages passed to the node using a `Set` - this 45 | removes the need to rely on any particular message property value (such as `_msgId`) 46 | that could be modified by the node. 47 | 48 | 49 | If `node.done()` is not called within the required timeout the runtime will call 50 | `Node.error(err,msg)` and trigger the timeout listener if one is registered. 51 | 52 | ### Next steps 53 | 54 | Some implementation considerations and outstanding questions: 55 | 56 | - how should a Function node indicate it can be timed-out? It could be able to set 57 | a timeout handler in the same was as described above - but that would be 58 | evaluated with every single message the Function receives. 59 | 60 | - This api assumes a default timeout value is applied to all enabled nodes. 61 | - Should it be possible for individual nodes to be given a custom timeout value? 62 | - If so, how does the user provide that value? The Editor does not know if the 63 | runtime instance has added a `timeout` handler 64 | - add another flag in the node's HTML file? 65 | - add the option in the UI regardless? 66 | - An node-level api will be needed for the node to give its value to override 67 | the default - `node.setTimeout(secs)`. 68 | 69 | - The simple implementation of a timeout will be to add a `setTimeout` for every 70 | message that is passed to the node - clearing the timeout when `done` is called. 71 | That does, however, lead to a performance issue if we are creating and 72 | clearing hundreds of timers a second. A more careful design would be to have 73 | one `setTimeout` active per node that checks for the next thing to timeout. 74 | This is probably the most critical part of the implementation to get right. 75 | 76 | ## History 77 | 78 | - 2019-03-26 - split out from Messaging API design 79 | -------------------------------------------------------------------------------- /proposal-template.md: -------------------------------------------------------------------------------- 1 | --- 2 | state: draft | in-progress | complete 3 | --- 4 | 5 | > To get started with this template: 6 | > 7 | > 1. **Make a copy of this template.** Copy this template into the `designs` 8 | > folder and name it `my-title.md`. If the proposal will include images, a folder 9 | > should be created for the proposal and its images: `my-title/README.md`. 10 | > 11 | > Remember to delete this getting started text - but ensure the YAML 12 | > header above remains. 13 | > 14 | > 2. **Create an initial proposal.** The proposal should set out the high-level 15 | > goals of the feature with enough detail to review the intent and direction of the 16 | > feature. 17 | > 18 | > 3. **Create a pull request.** This will be used to drive discussion on the 19 | > proposal. The goal is to merge proposals early and not try to tackle everything 20 | > in a single go. 21 | > 22 | > 4. **Develop the design.** The discussion on the design will provide guidance 23 | > on what further material is needed. 24 | 25 | # Title 26 | 27 | This is the title of the proposal. It should communicate the key purpose of the 28 | proposal and will be how it is referred to in general discussion. 29 | 30 | ## Summary 31 | 32 | This section provides the high-level description of the feature being proposed. 33 | It should provide enough information for end-users to understand what the feature 34 | is trying to achieve. 35 | 36 | ## Authors 37 | 38 | - A list of the authors of this proposal. 39 | 40 | ## Details 41 | 42 | This section provides the detailed information for the proposal. This template 43 | does not provide a structure for this section, but it should address: 44 | 45 | 1. any public APIs it introduces 46 | 2. the user experience of the feature - what it does and how it is used 47 | 3. any migration concerns 48 | 4. mock-up UI designs 49 | 50 | The discussion on the initial PR will provide guidance on what further material 51 | is needed. 52 | 53 | ## History 54 | 55 | This should be a list of major milestones in the life of the proposal. For example: 56 | 57 | - 2019-02-25 - Initial proposal submitted 58 | - yyyy-mm-dd - Moved to in-progress 59 | - yyyy-mm-dd - Shipped in Node-RED 0.20 - moved to complete folder 60 | --------------------------------------------------------------------------------