├── CODE_OF_CONDUCT.md ├── LICENSE.md ├── README.md ├── meetings └── 12-06-18.md └── users-research └── netflix └── README.md /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Code of Conduct 2 | 3 | The [Node.js Code of Conduct][] applies to this team. 4 | 5 | # Moderation Policy 6 | 7 | The [Node.js Moderation Policy][] applies to this team. 8 | 9 | [Node.js Code of Conduct]: https://github.com/nodejs/admin/blob/master/CODE_OF_CONDUCT.md 10 | [Node.js Moderation Policy]: https://github.com/nodejs/admin/blob/master/Moderation-Policy.md 11 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | MIT License (MIT) 2 | 3 | Copyright (c) Node.js contributors. All rights reserved. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 6 | 7 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 8 | 9 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE 10 | 11 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # promises-debugging 2 | 3 | ## Users research 4 | 5 | ### Netflix promises error handling survey 6 | 7 | [A recent survey](users-research/netflix/) about promises error handling use 8 | cases that was done internally at Netflix. 9 | -------------------------------------------------------------------------------- /meetings/12-06-18.md: -------------------------------------------------------------------------------- 1 | # Node.js Promise Debugging Meeting 12-06-2018 2 | 3 | **GitHub Issue**: https://github.com/nodejs/promise-use-cases/issues/28 4 | **Minutes Google Doc**: https://docs.google.com/document/d/1HHBLd_k2-sHMOj7DHgEzKcvYclX1BHQa95SIQYGILx0/edit 5 | 6 | ## Present 7 | 8 | Benedikt Meurer (@bmeurer) 9 | Benjamin Gruenbaum (@benjamingr) 10 | Julien Gilli (@misterdjules) 11 | Peter Marton (@hekike) 12 | Ruben Bridgewater (@BridgeAR) 13 | Yunong Xiao (@yunong) 14 | Michael Dawson (@mhdawson) 15 | 16 | ## Agenda 17 | 18 | - Introductions (10m) 19 | - Presentation of the problem with postmortem debugging and promises (10m) 20 | - How to proceed and gather use cases (10m) 21 | - How to collaborate and on this (10m) 22 | 23 | 24 | ## Minutes 25 | 26 | Introductions: 27 | Yunong - Node @ Netflix 28 | Julien - used to work at Joyent where was involved in Node.js, works at Netflix now 29 | Peter - used to work on an APM solution, now working at Netflix 30 | Benedikt - Node.js perf lead at V8 31 | Reuben - Node.js @ NearForm 32 | Benjamin - likes promises 33 | Michael - Node.js core and diagnostics 34 | 35 | Presentation of the problem with postmortem debugging and promises: 36 | 37 | Yunong - Netflix team building post mortem debugging tools which other teams will use - they need to debug issues and they use core dumps. 38 | Postmortem - not just core dumps, but core dumps are the most useful tool. 39 | Use llnode and mdb 40 | Important to their adoption of Node. 41 | Julien - promises and postmortem interact at netflix - they provide a platform for developers and need to make sure their developer experience is good. 42 | 43 | With promises it is challenging to understand the semantics with regards to error handling and how it interacts with generating artifacts (core dumps or else). 44 | 45 | Want to be confident to tell developers to use async/await and promises. 46 | 47 | Yunong - we all understand promises are a thing and want to provide developers a way to use promises. They are very impatient. 48 | 49 | Julien - looking for a pragmatic solution from the OPs side 50 | 51 | Benjamin - promises make it hard to take core dumps and we can’t synchronously handle the errors. Currently core dumps are relied on. 52 | 53 | Julien - correct, and we should focus on the agenda. 54 | 55 | Yunong - we should figure the collaboration model. 56 | 57 | Julien - historically there has been a lot of repos and issues and the knowledge is spread out. We should figure out a centralized place where we can go and make sure we have the up-to-date perspective from the Node.js and V8 project. 58 | 59 | Benjamin - agreed 60 | 61 | Benedikt - how about we put this on top of `promise-use-cases` 62 | 63 | Julien - Suggests a new separate repo to solve this. 64 | 65 | Benedikt - is it specifically about promise post mortem debugging or is this more general about post mortem? 66 | 67 | Julien - in this case it’s promises and postmortem. 68 | 69 | Yunong - vet with the diagnostics WG if we want a more general repo. There are specific things about debugging promises and async functions. 70 | 71 | Julien - if we broaden the topic it’s harder to collaborate on it. 72 | 73 | Yunong - create new repo like promise-postmortem-debugging 74 | 75 | Julien - when we say postmortem we don’t mean core dumps, we mean everything that is after the program terminated. We are open to using any other type of artifact. What’s important is the time and not the core-dump itself. 76 | 77 | Benjamin - we can reuse the repo if we want to focus not just on postmortem. 78 | 79 | Benedikt - from v8 side - you’re looking for ways to debug in the wild. This has to be as useful as possible in order to debug the core issue. 80 | 81 | Ruben - the implementation details might be interesting in the context - Ruben is almost done with implementation to throw on GC. There was some thing Benjamin brought up about throwing synchronously if no catch handler is attached. 82 | 83 | Yunong - should it be a different repo? 84 | 85 | Benedikt - I think it should be promise debugging and not just postmortem because the service may die. 86 | 87 | Yunong - rename or create new repo. 88 | 89 | Benjamin - sure. 90 | 91 | Julien - promise-use-cases repo is a bit too general because it also discusses performance and testing and other stuff. 92 | 93 | Benedikt - we should not just frame it as postmortem - but it should be in general promises debugging because we care about the browsers and the web platform too. 94 | 95 | Yunong - agreed. 96 | 97 | Benjamin - should we have meetings? 98 | 99 | Julien - would prefer not to have biweekly meetings. Prefer to have a reference document. 100 | 101 | Ruben - agree with Julien. 102 | 103 | Yunong - carve out a section in the diagnostics summit to discuss this. We should be more proactive. 104 | 105 | Julien - we are not too familiar with the browser use cases of the browser. We would be able to contribute from a non-browser perspective. 106 | 107 | Benedikt - almost no one is shipping async/await in native, we see different implementation since people transpile. We want to use Node.js to understand the web story. 108 | 109 | Julien - would love information about bluebird and debugging. 110 | 111 | Benjamin - I have a lot of information about how people are using bluebird. 112 | 113 | Yunong - would be useful to be in the same repo. 114 | Benjamin - Want to make sure we are held accountable and focused. 115 | 116 | Michael - It’s been easier in other teams to have meetings to gain momentum. 117 | 118 | Yunong - we want to make sure we’re holding ourselves accountable to making forward progress. In 3 weeks let’s make sure we’ll have a bunch of use cases, in 3 weeks there’s the diagnostics summit - meet in person and go through the work we want to do. Want to make sure we don’t wait for meetings for progress. 119 | 120 | Julien - want things to be as fluid as possible. Not having meetings would encourage people to reach out. Agree about roadmap and goals. 121 | 122 | Michael - happy to try anything. As long as stuff happens it doesn’t matter, it’s just that often we see things tail off. Sometimes when you have somebody new it’s an easier way to get them ramped up. Not pushing to have another meeting, has seen in the past in some interval it helps. 123 | 124 | Benjamin - Let’s agree to have another meeting if we stall in 3 weeks. 125 | 126 | Julien - are we OK with describing use cases in issues. 127 | 128 | Benjamin - works for me 129 | 130 | Benedikt - useful for us to have concrete use cases. Wants something they can execute and 131 | 132 | Julien - GitHub repo to gather use cases and documents. 133 | 134 | Benjamin - do we want to also do terminology or a reference document? 135 | 136 | Julien - having a reference document about the current state and where you can find them. 137 | 138 | Yunong - we should have documentation/guide of the expected behavior in the Node.js runtime. 139 | 140 | Julien - that’s a possible outcome that we might like. I want to centralize an understanding of where we stand today. What would be the best way to communicate between us? Should we have slack? 141 | 142 | Benjamin - should we open slack? 143 | 144 | Michael & Ruben - slack work 145 | 146 | Benjamin - how do we deal with disagreements? Do we escalate? 147 | 148 | Yunong - I would like to avoid uncollaborative working environment. When things have to go to a vote then things have gone bad. It should be the last resort. 149 | Julien - this group is more for bouncing off ideas and discussing them. The Node.js project operates under a specific governance model. More of a support function. 150 | 151 | Benjamin - I don’t expect that many disagreements as we’ve had in the past 152 | 153 | Yunong - it’s OK to disagree about implementation and it’s fine. The entire group things a different way is helpful. 154 | 155 | Julien - It’s fine if we gather feedback even if we disagree. 156 | 157 | Yunong - we should do an ad-hoc meeting face-to-face if we get frustrated. 158 | 159 | Michael - Things come along much harder in person. 160 | 161 | Benjamin - we should discuss the actual implementation stuff after we do the repo. 162 | 163 | Yunong - Sounds good to me 164 | 165 | Michael - does it makes sense to publicize the repo to see if other people are interested? 166 | 167 | Yunong - we should be inclusive and promote. 168 | 169 | Michael - I can help with that 170 | 171 | Julien - we should be disciplined that the information in the repo is scoped towards debugging and post-mortem. We should redirect and guard our scope. 172 | 173 | Yunong - we should have a charter. 174 | 175 | Julien - we want to avoid confusion. 176 | 177 | Benjamin - should we have a goals statement? 178 | 179 | Julien & Yunong - sure 180 | 181 | Benedikt - should be in the readme. 182 | 183 | Benjamin - We should PR all the things and wait 72 hours for objections. 184 | 185 | Yunong - yes. 186 | 187 | Julien - we should use the Node.js process when it’s not process heavy and do that. 188 | 189 | 190 | 191 | 192 | ## Action Items 193 | - Create a new repository (/promises-debugging) to collaborate on this. (Benjamin) 194 | - Write README for `promises-debugging` and collaborate on this. (Julien, will be a collaborative effort) 195 | - Write goals as part of the README. 196 | - Add process to the README. The 72 hours PR thing basically. 197 | - Folks should feel free to file issues for each use case (All) 198 | - See where we are in 3 weeks and schedule a meeting if we don’t have enough progress. (Benjamin) 199 | - Make a central document as a reference about the issue, what the status is, where we’re moving - first draft - Julien 200 | - Open a slack. (Ruben) 201 | - Publicize the repo after we’ve set up and are ready. (Michael, Benedikt) 202 | - GitHub Team @nodejs/promises-debugging (Michael) 203 | - Add MayaLekova there please 204 | -------------------------------------------------------------------------------- /users-research/netflix/README.md: -------------------------------------------------------------------------------- 1 | # Netflix promises error handling survey 2 | 3 | This document presents the context and the findings of [a recent 4 | survey](https://docs.google.com/forms/d/1lM4TdTJcRbW2CDZw7Yj4Su75joGvK6DIkD95CenVRmk/viewanalytics) 5 | about promises error handling use cases that was done internally at Netflix. 6 | 7 | The goal of that survey was to get a better understanding on what promises error 8 | handling modes [recently proposed for the Node.js 9 | runtime](https://github.com/nodejs/node/pull/20097) are relevant for users, and 10 | how they would impact their ability to run applications reliably at Netflix 11 | while providing a good developer and operators experience. 12 | 13 | We hoped that it would help us inform the conversation in that PR, especially on 14 | aspects that did not reach broad consensus yet. 15 | 16 | The first section describes how this survey was conducted. Then, we provide a 17 | summary of the results that highlights what we think is relevant and 18 | interesting, but without any interpretation. 19 | 20 | The third section interprets those key items that we found interesting and 21 | presents what we think the key learnings can be. 22 | 23 | Finally, we conclude with the set of recommendations we want to make for the 24 | Node.js PR mentioned above and what other action items we think may be worth 25 | tackling in the future. 26 | 27 | ## Disclaimer 28 | 29 | We're aware that this survey doesn't follow best practices in its methodology 30 | and implementation. We didn't expect this survey to provide us with a 31 | statistical confidence in its results. 32 | 33 | This is OK because the goal of this survey was to get a _general sense_ of what 34 | promises users inside of Netflix care about when it comes to error handling use 35 | cases, not an accurate picture of what all users inside and outside of Netflix 36 | actually rely on. 37 | 38 | ## Methodology 39 | 40 | We used Google forms to conduct the survey. The total number of respondents is 41 | 32 so far. The survey had been open for > 2 weeks at the time we collected and 42 | interpreted results. 43 | 44 | No question was mandatory, which explains why some questions have more answers 45 | than others. 46 | 47 | In addition to that, the question about asynchronously adding catch handlers to 48 | promises was added when the survey had already been filled out by half of the 49 | current number of respondents. We consider this to be acceptable, and we’d 50 | rather have some data about this question than no data. 51 | 52 | The survey didn't target any specific population of JavaScript users at Netflix. 53 | This was a conscious decision, as one of the goals was to get feedback from 54 | JavaScript users that had diverse technical backgrounds and that didn’t have the 55 | same biases as the users with which the team that conducted the survey interact 56 | on a day to day basis. 57 | 58 | In parallel of this survey, we did a small number of 30 minutes in-person 59 | interviews with four selected respondents to go deeper into some of the answers 60 | that were given when we thought that they could give us more insights. 61 | 62 | Those interviews were not recorded. We understand that this is a missed 63 | opportunity and that it lowers the credibility of this document, particularly 64 | when it comes to conclusions that have been significantly influenced by those 65 | interviews. We will strive to do better in the future. 66 | 67 | ## Summary of survey results 68 | 69 | This section summarizes in prose the data that was collected by the survey and 70 | that we consider interesting. The next section (Interpretation of survey results 71 | and interviews) focuses on what we think that data means. 72 | 73 | For raw data, please consult the [survey 74 | results](https://docs.google.com/forms/d/1lM4TdTJcRbW2CDZw7Yj4Su75joGvK6DIkD95CenVRmk/viewanalytics). 75 | 76 | ### Promises usage 77 | 78 | - Most of the respondents (90%) use promises. This could be because the survey 79 | was presented as a "promises survey", and thus users who didn't use promises 80 | may have self-selected out. There are at least three counter examples of this 81 | though, so this concern might not be valid. 82 | 83 | - Most respondents of promises seem to use native promises, but a non-negligible 84 | number of respondents use non-native promises. Among non-native promises 85 | implementations, Q and Bluebird seem to have the most users. 86 | 87 | - All respondents who are not using promises are using Rx observables instead. 88 | It's not clear whether the two serve identical use cases, but those users seem 89 | to at least have identified some overlap between the two and preferred Rx 90 | observables for several reasons: 91 | 92 | - Rx observables seem to be cancellable while promises are definitely not. 93 | - Historical context: promises were not necessarily available when those users 94 | had to pick a similar abstraction. 95 | - Issues with the error handling model of promises. 96 | 97 | ### Async/Await usage 98 | 99 | - A significant number of respondents (~70%) seem to use async/await. 100 | 101 | - For others, reasons not to use async/await include: 102 | - Issues with the error handling model. 103 | - Runtime not supporting them (old version of JavaScript Core). 104 | 105 | ### Promises error handling 106 | 107 | - A significant portion of respondents seem to differentiate between fatal and 108 | non-fatal errors when using promises. 109 | 110 | - A significant number of respondents seem to use different classes of errors to 111 | represent the difference between fatal and non-fatal errors. 112 | 113 | - A significant number of respondents consider that they should always attach a 114 | catch handler to a promise, however most of them don't seem to enforce it via 115 | tooling (e.g. static analysis tools). 116 | 117 | - Only one respondent mentioned that they add catch handlers asynchronously. 118 | When discussing their use case for it, they mentioned that they could achieve 119 | similar functionality differently (e.g. by emitting/listening on a 'error' 120 | event), and that the ability to add catch handlers _asynchronously_ was only a 121 | matter of user experience. 122 | 123 | - A significant number of respondents seem to consider that exiting the Node.js 124 | process on an unhandled promise rejection is a desirable behavior. 125 | 126 | - Most of the respondents that add an `'unhandledRejection'` event listener use 127 | it to log and exit the process. 128 | 129 | - However, most respondents seem not to use it, and a non-negligible part of 130 | them were not aware that the `'unhandledRejection'` event exists. 131 | 132 | - More respondents seem to be adding a listener for the `uncaughtException` 133 | event, and those handlers also almost always log messages and exit the 134 | process. However, a non-negligible number of respondents are not aware that 135 | this event exists. 136 | 137 | - No respondent seems to consider that exiting the Node.js process once a 138 | promise with an unhandled rejection is garbage collected is a desirable 139 | behavior. 140 | 141 | ## Interpretation of survey results and interviews 142 | 143 | This section presents _our interpretation_ of the data that was collected and 144 | the four interviews we had with a selected sample of respondents. 145 | 146 | ### Promises are used significantly at Netflix 147 | 148 | It is clear that a lot of JavaScript developers use promises at Netflix, 149 | including when writing Node.js applications. It also seems like a significant 150 | number of users are at least considering to use async/await. 151 | 152 | ### Promises are confusing for users 153 | 154 | However, it's also clear that promises are confusing, even for developers who 155 | have a lot of experience with them. Every single one-on-one conversation we 156 | had with promises users highlighted at least one area of confusion with regards 157 | to how different JS runtimes implement promises and how they handle their 158 | errors. 159 | 160 | ### Developers don't use static analysis tools to validate promises usage 161 | 162 | So far, we have seen that a significant number of users do not rely on static 163 | analysis tools to help them validate their usage of promises. As a result, it 164 | seems they _exclusively rely on runtime checks_ to perform this validation. 165 | 166 | ### Exiting Node.js process on unhandled rejection seems to be considered as valid behavior 167 | 168 | A lot of respondents seem to be comfortable with Node.js exiting when a promise 169 | rejection is unhandled. The motivation for this behavior includes having better 170 | visibility on potentially hidden errors, and ensuring the process' state is 171 | consistent. 172 | 173 | The fact that the process exits synchronously or not does not seem to matter 174 | much for users. 175 | 176 | Some users are not comfortable with having the runtime dictate the behavior of 177 | the process when an unhandled rejection occurs, but they seem to be fine with 178 | being able to override the default behavior. 179 | 180 | ### Exiting on garbage collection doesn't seem to be considered as a desirable behavior 181 | 182 | As part of the survey, no respondent chose this mode as a desirable behavior of 183 | the runtime. Moreover, without giving any hint, every interview highlighted that 184 | this was considered a potential source of confusion. We failed to find evidence 185 | of respondents who considered this was a desirable behavior of the runtime. 186 | 187 | ### Attaching catch handlers asynchronously doesn't seem to be a required use case 188 | 189 | Only one respondent mentioned it as an important use case. However, after 190 | discussing this use case with them, they mentioned that the ability to 191 | asynchronously add catch handlers to promises wasn't a functional requirement, 192 | and that it was only a nice to have in order to provide better UX. 193 | 194 | ## Conclusions 195 | 196 | In this section, we present what we think are relevant conclusions based on this 197 | survey that can inform future discussions in the [PR recently proposed for the 198 | Node.js runtime](https://github.com/nodejs/node/pull/20097) and more generally 199 | on the topic of promises debugging and error handling. 200 | 201 | ### No evidence that exiting on GCed unhandled promise is desirable 202 | 203 | We haven't managed to find evidence of exiting on GC being a valid use case. As 204 | a result we are still leaning towards recommending not to add this behavior to 205 | the Node.js runtime. 206 | 207 | ### Exiting on unhandled rejections could be well accepted by Node.js users 208 | 209 | We haven't found any evidence that, when providing the ability to override this 210 | behavior, having the runtime defaults to exit on unhandled rejections is 211 | undesirable. 212 | 213 | As a result, we will continue to lean towards recommending this as the default 214 | behavior for the Node.js runtime, as well as providing the ability to add a 215 | custom override. 216 | 217 | ### Advocating for using static analysis tools could help 218 | 219 | It seems that using static code analysis tooling is not a common way for 220 | developers to validate the robustness of their code regarding promises error 221 | handling. 222 | 223 | If such static analysis tooling was broadly used, we could at least inform 224 | runtime behavior changes based on behavior that is enforced at build time. 225 | 226 | As a result, it seems worth it to explore advocating for using useful static 227 | code analysis tooling that encourages the best practices that we’d like the 228 | broader community to use and that are compatible with the debugging tools we’d 229 | like to use. 230 | 231 | ### Focus on developers education may help reach consensus in the future 232 | 233 | All our conversations with promises users showed that there are a lot of 234 | different perspectives on why to use promises, how they work, and what best 235 | practices to use. 236 | 237 | Most of the time though, when digging deeper and when providing additional 238 | insights, it seemed that all interviewees reached a common ground on those 239 | topics. 240 | 241 | Our takeaway is that socializing challenges and best practices around using 242 | promises and their error handling model could go a long way towards building a 243 | shared understanding about promises and reaching a consensus on those topics. 244 | 245 | We think reaching a broader consensus will help steer the Node.js platform 246 | towards a more consistent behavior with regards to how promises are supported. 247 | We hope to be able to work on enabling those conversations in the near future. 248 | 249 | --------------------------------------------------------------------------------