├── LICENSE ├── README.md └── SolutionDesign └── README.md /LICENSE: -------------------------------------------------------------------------------- 1 | Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International 2 | Creative Commons Corporation (“Creative Commons”) is not a law firm and does not provide legal services or legal advice. Distribution of Creative Commons public licenses does not create a lawyer-client or other relationship. Creative Commons makes its licenses and related information available on an “as-is” basis. Creative Commons gives no warranties regarding its licenses, any material licensed under their terms and conditions, or any related information. Creative Commons disclaims all liability for damages resulting from their use to the fullest extent possible. 3 | 4 | Using Creative Commons Public Licenses 5 | 6 | Creative Commons public licenses provide a standard set of terms and conditions that creators and other rights holders may use to share original works of authorship and other material subject to copyright and certain other rights specified in the public license below. The following considerations are for informational purposes only, are not exhaustive, and do not form part of our licenses. 7 | 8 | Considerations for licensors: Our public licenses are intended for use by those authorized to give the public permission to use material in ways otherwise restricted by copyright and certain other rights. Our licenses are irrevocable. Licensors should read and understand the terms and conditions of the license they choose before applying it. Licensors should also secure all rights necessary before applying our licenses so that the public can reuse the material as expected. Licensors should clearly mark any material not subject to the license. This includes other CC-licensed material, or material used under an exception or limitation to copyright. More considerations for licensors. 9 | 10 | Considerations for the public: By using one of our public licenses, a licensor grants the public permission to use the licensed material under specified terms and conditions. If the licensor’s permission is not necessary for any reason–for example, because of any applicable exception or limitation to copyright–then that use is not regulated by the license. Our licenses grant only permissions under copyright and certain other rights that a licensor has authority to grant. Use of the licensed material may still be restricted for other reasons, including because others have copyright or other rights in the material. A licensor may make special requests, such as asking that all changes be marked or described. Although not required by our licenses, you are encouraged to respect those requests where reasonable. More considerations for the public. 11 | 12 | Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License 13 | By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License ("Public License"). To the extent this Public License may be interpreted as a contract, You are granted the Licensed Rights in consideration of Your acceptance of these terms and conditions, and the Licensor grants You such rights in consideration of benefits the Licensor receives from making the Licensed Material available under these terms and conditions. 14 | 15 | Section 1 – Definitions. 16 | a. Adapted Material means material subject to Copyright and Similar Rights that is derived from or based upon the Licensed Material and in which the Licensed Material is translated, altered, arranged, transformed, or otherwise modified in a manner requiring permission under the Copyright and Similar Rights held by the Licensor. For purposes of this Public License, where the Licensed Material is a musical work, performance, or sound recording, Adapted Material is always produced where the Licensed Material is synched in timed relation with a moving image. 17 | 18 | b. Adapter's License means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License. 19 | 20 | c. BY-NC-SA Compatible License means a license listed at creativecommons.org/compatiblelicenses, approved by Creative Commons as essentially the equivalent of this Public License. 21 | 22 | d. Copyright and Similar Rights means copyright and/or similar rights closely related to copyright including, without limitation, performance, broadcast, sound recording, and Sui Generis Database Rights, without regard to how the rights are labeled or categorized. For purposes of this Public License, the rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights. 23 | 24 | e. Effective Technological Measures means those measures that, in the absence of proper authority, may not be circumvented under laws fulfilling obligations under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, and/or similar international agreements. 25 | 26 | f. Exceptions and Limitations means fair use, fair dealing, and/or any other exception or limitation to Copyright and Similar Rights that applies to Your use of the Licensed Material. 27 | 28 | g. License Elements means the license attributes listed in the name of a Creative Commons Public License. The License Elements of this Public License are Attribution, NonCommercial, and ShareAlike. 29 | 30 | h. Licensed Material means the artistic or literary work, database, or other material to which the Licensor applied this Public License. 31 | 32 | i. Licensed Rights means the rights granted to You subject to the terms and conditions of this Public License, which are limited to all Copyright and Similar Rights that apply to Your use of the Licensed Material and that the Licensor has authority to license. 33 | 34 | j. Licensor means the individual(s) or entity(ies) granting rights under this Public License. 35 | 36 | k. NonCommercial means not primarily intended for or directed towards commercial advantage or monetary compensation. For purposes of this Public License, the exchange of the Licensed Material for other material subject to Copyright and Similar Rights by digital file-sharing or similar means is NonCommercial provided there is no payment of monetary compensation in connection with the exchange. 37 | 38 | l. Share means to provide material to the public by any means or process that requires permission under the Licensed Rights, such as reproduction, public display, public performance, distribution, dissemination, communication, or importation, and to make material available to the public including in ways that members of the public may access the material from a place and at a time individually chosen by them. 39 | 40 | m. Sui Generis Database Rights means rights other than copyright resulting from Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended and/or succeeded, as well as other essentially equivalent rights anywhere in the world. 41 | 42 | n. You means the individual or entity exercising the Licensed Rights under this Public License. Your has a corresponding meaning. 43 | 44 | Section 2 – Scope. 45 | a. License grant. 46 | 47 | Subject to the terms and conditions of this Public License, the Licensor hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, irrevocable license to exercise the Licensed Rights in the Licensed Material to: 48 | 49 | A. reproduce and Share the Licensed Material, in whole or in part, for NonCommercial purposes only; and 50 | 51 | B. produce, reproduce, and Share Adapted Material for NonCommercial purposes only. 52 | 53 | Exceptions and Limitations. For the avoidance of doubt, where Exceptions and Limitations apply to Your use, this Public License does not apply, and You do not need to comply with its terms and conditions. 54 | 55 | Term. The term of this Public License is specified in Section 6(a). 56 | 57 | Media and formats; technical modifications allowed. The Licensor authorizes You to exercise the Licensed Rights in all media and formats whether now known or hereafter created, and to make technical modifications necessary to do so. The Licensor waives and/or agrees not to assert any right or authority to forbid You from making technical modifications necessary to exercise the Licensed Rights, including technical modifications necessary to circumvent Effective Technological Measures. For purposes of this Public License, simply making modifications authorized by this Section 2(a)(4) never produces Adapted Material. 58 | 59 | Downstream recipients. 60 | 61 | A. Offer from the Licensor – Licensed Material. Every recipient of the Licensed Material automatically receives an offer from the Licensor to exercise the Licensed Rights under the terms and conditions of this Public License. 62 | 63 | B. Additional offer from the Licensor – Adapted Material. Every recipient of Adapted Material from You automatically receives an offer from the Licensor to exercise the Licensed Rights in the Adapted Material under the conditions of the Adapter’s License You apply. 64 | 65 | C. No downstream restrictions. You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, the Licensed Material if doing so restricts exercise of the Licensed Rights by any recipient of the Licensed Material. 66 | 67 | No endorsement. Nothing in this Public License constitutes or may be construed as permission to assert or imply that You are, or that Your use of the Licensed Material is, connected with, or sponsored, endorsed, or granted official status by, the Licensor or others designated to receive attribution as provided in Section 3(a)(1)(A)(i). 68 | 69 | b. Other rights. 70 | 71 | Moral rights, such as the right of integrity, are not licensed under this Public License, nor are publicity, privacy, and/or other similar personality rights; however, to the extent possible, the Licensor waives and/or agrees not to assert any such rights held by the Licensor to the limited extent necessary to allow You to exercise the Licensed Rights, but not otherwise. 72 | 73 | Patent and trademark rights are not licensed under this Public License. 74 | 75 | To the extent possible, the Licensor waives any right to collect royalties from You for the exercise of the Licensed Rights, whether directly or through a collecting society under any voluntary or waivable statutory or compulsory licensing scheme. In all other cases the Licensor expressly reserves any right to collect such royalties, including when the Licensed Material is used other than for NonCommercial purposes. 76 | 77 | Section 3 – License Conditions. 78 | Your exercise of the Licensed Rights is expressly made subject to the following conditions. 79 | 80 | a. Attribution. 81 | 82 | If You Share the Licensed Material (including in modified form), You must: 83 | 84 | A. retain the following if it is supplied by the Licensor with the Licensed Material: 85 | 86 | i. identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated); 87 | 88 | ii. a copyright notice; 89 | 90 | iii. a notice that refers to this Public License; 91 | 92 | iv. a notice that refers to the disclaimer of warranties; 93 | 94 | v. a URI or hyperlink to the Licensed Material to the extent reasonably practicable; 95 | 96 | B. indicate if You modified the Licensed Material and retain an indication of any previous modifications; and 97 | 98 | C. indicate the Licensed Material is licensed under this Public License, and include the text of, or the URI or hyperlink to, this Public License. 99 | 100 | You may satisfy the conditions in Section 3(a)(1) in any reasonable manner based on the medium, means, and context in which You Share the Licensed Material. For example, it may be reasonable to satisfy the conditions by providing a URI or hyperlink to a resource that includes the required information. 101 | 102 | If requested by the Licensor, You must remove any of the information required by Section 3(a)(1)(A) to the extent reasonably practicable. 103 | 104 | b. ShareAlike. 105 | 106 | In addition to the conditions in Section 3(a), if You Share Adapted Material You produce, the following conditions also apply. 107 | 108 | The Adapter’s License You apply must be a Creative Commons license with the same License Elements, this version or later, or a BY-NC-SA Compatible License. 109 | 110 | You must include the text of, or the URI or hyperlink to, the Adapter's License You apply. You may satisfy this condition in any reasonable manner based on the medium, means, and context in which You Share Adapted Material. 111 | 112 | You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, Adapted Material that restrict exercise of the rights granted under the Adapter's License You apply. 113 | 114 | Section 4 – Sui Generis Database Rights. 115 | Where the Licensed Rights include Sui Generis Database Rights that apply to Your use of the Licensed Material: 116 | 117 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, reuse, reproduce, and Share all or a substantial portion of the contents of the database for NonCommercial purposes only; 118 | 119 | b. if You include all or a substantial portion of the database contents in a database in which You have Sui Generis Database Rights, then the database in which You have Sui Generis Database Rights (but not its individual contents) is Adapted Material, including for purposes of Section 3(b); and 120 | 121 | c. You must comply with the conditions in Section 3(a) if You Share all or a substantial portion of the contents of the database. 122 | 123 | For the avoidance of doubt, this Section 4 supplements and does not replace Your obligations under this Public License where the Licensed Rights include other Copyright and Similar Rights. 124 | 125 | Section 5 – Disclaimer of Warranties and Limitation of Liability. 126 | a. Unless otherwise separately undertaken by the Licensor, to the extent possible, the Licensor offers the Licensed Material as-is and as-available, and makes no representations or warranties of any kind concerning the Licensed Material, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to You. 127 | 128 | b. To the extent possible, in no event will the Licensor be liable to You on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this Public License or use of the Licensed Material, even if the Licensor has been advised of the possibility of such losses, costs, expenses, or damages. Where a limitation of liability is not allowed in full or in part, this limitation may not apply to You. 129 | 130 | c. The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability. 131 | 132 | Section 6 – Term and Termination. 133 | a. This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically. 134 | 135 | b. Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates: 136 | 137 | automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or 138 | 139 | upon express reinstatement by the Licensor. 140 | 141 | For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License. 142 | 143 | c. For the avoidance of doubt, the Licensor may also offer the Licensed Material under separate terms or conditions or stop distributing the Licensed Material at any time; however, doing so will not terminate this Public License. 144 | 145 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public License. 146 | 147 | Section 7 – Other Terms and Conditions. 148 | a. The Licensor shall not be bound by any additional or different terms or conditions communicated by You unless expressly agreed. 149 | 150 | b. Any arrangements, understandings, or agreements regarding the Licensed Material not stated herein are separate from and independent of the terms and conditions of this Public License. 151 | 152 | Section 8 – Interpretation. 153 | a. For the avoidance of doubt, this Public License does not, and shall not be interpreted to, reduce, limit, restrict, or impose conditions on any use of the Licensed Material that could lawfully be made without permission under this Public License. 154 | 155 | b. To the extent possible, if any provision of this Public License is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from this Public License without affecting the enforceability of the remaining terms and conditions. 156 | 157 | c. No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor. 158 | 159 | d. Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority. 160 | 161 | Creative Commons is not a party to its public licenses. Notwithstanding, Creative Commons may elect to apply one of its public licenses to material it publishes and in those instances will be considered the “Licensor.” Except for the limited purpose of indicating that material is shared under a Creative Commons public license or as otherwise permitted by the Creative Commons policies published at creativecommons.org/policies, Creative Commons does not authorize the use of the trademark “Creative Commons” or any other trademark or logo of Creative Commons without its prior written consent including, without limitation, in connection with any unauthorized modifications to any of its public licenses or any other arrangements, understandings, or agreements concerning use of licensed material. For the avoidance of doubt, this paragraph does not form part of the public licenses. 162 | 163 | Creative Commons may be contacted at creativecommons.org 164 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Brenn’s Software Engineering Principles 2 | 3 | # Preamble 4 | 5 | This is a collection of principles, guidelines, and suggestions I’ve compiled as my default “Tech standards playbook”. It’s oriented to standard webapps, SaaS applications, and similar. If you are doing a hardware startup or something that’s deep in security or something outside the area for which this is intended then parts of this document may not apply. 6 | 7 | 8 | 9 | I write this in part to help out the large number of non-tech founders I’ve met who are trying to hire a CTO or VPoE and don’t know how to choose or how to evaluate them. This gives them something they can use as a sounding board/conversation starter. 10 | 11 | 12 | 13 | That said, I offer this for free, you get what you pay for. 14 | 15 | # Culture Standards 16 | 17 | 1. Communicate early and often. 18 | 19 | 2. Don’t assume others are on the same page. Over-communicate and explain. 20 | 21 | 3. Self-organize, collaborate, be proactive in resolving your own blockers and helping others to do so. All for one, one for all. There is no such thing as “not my job” in a startup. In a bigger company, if you can’t solve a problem then help find the person who can. 22 | 23 | 4. If you don’t know what to do, feel stuck, are out of tasks, or are worried/nervous about anything regarding the tech, the product, or company, please speak up. 24 | 25 | 26 | 27 | ## Set and Communicate Shared Values 28 | 29 | Let’s start with defining why we need to establish a set of shared values. Each industry and team faces different risks and each company is in a different part of its lifecycle. The values that need to be maintained in a highly regulated or life and death industry (like healthcare) are very different from what you should value in a fast-paced internet startup where fortunes may be won or lost but lives will not be. 30 | 31 | 32 | 33 | These values are for those sorts of companies, which are the locations I generally work (along with most others in the internet industry). Highly regulated would be different (more testing, more formal validation, and scheduled penetration tests, etc). 34 | 35 | ### My Personal Values as a Dev and Team Lead 36 | 37 | 38 | 39 | - *Be experimental, fail fast, fail cheap.* The best teacher is the market. It’s better to put something out there and get feedback than try and polish something to perfection against an imaginary customer. 40 | 41 | 42 | - *Focus on learning.* Everyone on the team should not only be growing as an engineer, but also in their understanding of the whole business. 43 | 44 | 45 | - *Become cross-functional. We do not want to be siloed or too specialist.* Being excellent at one thing (FE/React, BE, etc) is fantastic, don’t be afraid to mentor others or take on things outside your specialty. We never know what challenges we will face so we need to prepare ourselves for anything. We do that by growing our skills in as many aspects of technology as we can. 46 | 47 | 48 | - *Good developers ship.* In the end, shipping working software is the most important thing. We hold ourselves to the goal of always improving how fast we can get our code to production while maintaining acceptable code quality. 49 | 50 | 51 | 52 | 53 | ### My Personal Values as a Tech Executive 54 | 55 | - *People come first.* Almost every problem I’ve seen at the organizational scale is a people issue first, a process issue second, and a tech issue third if at all. Get the right people in the right place to succeed, and the tech will usually be taken care of. Take care of your people and let them handle the rest. 56 | 57 | 58 | - *Build Trust, Earn Trust.* Trust is the foundation of relationships. Relationships are necessary between people to be effective. People come first. Ergo, communicate honestly at all times. Mistakes are OK if owned and learned from. Say what you know, what you think, what you believe, and be willing to admit what you don’t know. Be direct in communications. No gossip, no passive aggressive BS. I hold myself to this standard and must be a role model for the team. 59 | 60 | 61 | 62 | 63 | - *Align incentives, Use the whole person.* Organizations hire engineers to be smart, creative, problem solvers. Then they shield them from decisions which require smart and creative solutions just because those decisions don’t involve writing code. People are more than cogs, use and develop the full power of the human capability for creative thought. Align the development of the people with the development and success of the organization. 64 | 65 | 66 | 67 | 68 | - *Do the big things right, don’t sweat small stuff.* The world is ruled by Pareto's principle: 20% of what we do drives 80% of the results. Focus on getting that 20% done 120%, then work on the rest. Corollary: Anything that isn't really important probably shouldn’t be done at all. Typically, this means most meetings. Most meetings should be emails, most emails should be quick messages, and half the time even those get resolved before anyone needs to get involved. Killing waste is usually more impactful than trying to optimize. 69 | 70 | 71 | 72 | 73 | Those are for me personally as a tech leader. You can/should have your own and be able to articulate them and defend why they matter to you and your company. 74 | 75 | # Writing Scalable Software: The 12 Factor App Model 76 | 77 | The 12 factor app model (12factor.net) is an established approach to writing software that scales. We should obey these principles whenever possible. When not possible, we should take a pause and ask ourselves what we are doing wrong, fix it, and go back to the 12 factor model. While following the 12 factor model does not guarantee software will scale, breaking these rules is a good way to guarantee that it will not scale. So follow the rules ;) 78 | 79 | ## [I. Codebase](https://12factor.net/codebase) 80 | 81 | One codebase tracked in revision control, many deploys. 82 | 83 | ## [II. Dependencies](https://12factor.net/dependencies) 84 | 85 | Explicitly declare and isolate dependencies 86 | 87 | ## [III. Config](https://12factor.net/config) 88 | 89 | Store config in the environment using environment variables. 90 | 91 | ## [IV. Backing services](https://12factor.net/backing-services) 92 | 93 | Treat backing services as attached resources 94 | 95 | ## [V. Build, release, run](https://12factor.net/build-release-run) 96 | 97 | Strictly separate build and run stages 98 | 99 | ## [VI. Processes](https://12factor.net/processes) 100 | 101 | Execute the app as one or more stateless processes 102 | 103 | ## [VII. Port binding](https://12factor.net/port-binding) 104 | 105 | Export services via port binding 106 | 107 | ## [VIII. Concurrency](https://12factor.net/concurrency) 108 | 109 | Scale out via the process model 110 | 111 | ## [IX. Disposability](https://12factor.net/disposability) 112 | 113 | Maximize robustness with fast startup and graceful shutdown 114 | 115 | ## [X. Dev/prod parity](https://12factor.net/dev-prod-parity) 116 | 117 | Keep development, staging, and production as similar as possible 118 | 119 | ## [XI. Logs](https://12factor.net/logs) 120 | 121 | Treat logs as event streams 122 | 123 | ## [XII. Admin processes](https://12factor.net/admin-processes) 124 | 125 | Run admin/management tasks as one-off processes 126 | 127 | ## Writing Measurable Software 128 | 129 | Note: Once you have a working MVP, you want to consider an additional 7 factors 130 | 131 | [https://www.ibm.com/cloud/blog/7-missing-factors-from-12-factor-applications](https://www.ibm.com/cloud/blog/7-missing-factors-from-12-factor-applications) 132 | 133 | 134 | 135 | 136 | # API Principles 137 | 138 | ## I. Understand your Data in APIs. 139 | 140 | What I’m about to write is a VERY general suggestion, but I’ve seen this be applicable so often that I’m just gonna throw it out there. It’s a great place to start thinking about data and fixing this alone has resulted in 10x application performance when properly using an edge cache as I suggest below. 141 | 142 | 143 | 144 | ### Data Segmentation 145 | 146 | Data should be segmented into one of three types: Static, Dynamic, and Personal. 147 | 148 | 149 | 150 | API CALLS SHOULD INCLUDE ONLY ONE TYPE OF DATA. 151 | 152 | 153 | 154 | - STATIC data is data that once created rarely or never changes or changes rarely (daily, every few hours). Example: product photos and description. 155 | 156 | 157 | - DYNAMIC data is data that is known to change regularly - every few seconds or every minute but the value is global. Typically fetched from a database table. Example: Available inventory, shipping time updates, etc. 158 | 159 | 160 | 161 | 162 | - PERSONAL data is data that may or may not change regularly but is specific and unique (and likely private) to a user. Example: shopping carts, user profile, order histories. 163 | 164 | 165 | ## II. Design for Caching 166 | 167 | Data should be cached appropriately to its type (see below). 168 | 169 | 170 | 171 | - Static data should be cached either indefinitely or with a very long (hours) TTL. Ideally, the content will only update with a PUSH from source systems telling the cache that new data is available. 172 | 173 | 174 | - Dynamic data should be updated at a known frequency (e.g. 10s). Thought should be given to the nature of the data whether a persistent (update on notification) or transient (expires after a timer) cache should be set. 175 | 176 | 177 | - Unique/Personal data should be cached at the client side whenever possible. Exceptions to this rule should be for security or legal reasons, in which case data should be stored in whichever means is most appropriate. For instance, a user logs in and their profile loads. Cache in the browser or mobile app as long as possible to avoid future API calls. 178 | 179 | 180 | 181 | 182 | For Static and Dynamic data, Edge Caching (e.g CDN) should be implemented transparently at the infrastructure level. Ideally neither client nor server is aware of the existence of the cache. This typically means using a CDN or API gateway/reverse-proxy service that encapsulates this behavior. We do this because multiple levels of cache can create all sorts of race conditions and invalidation bugs that are hard to debug or reproduce. By keeping things at the infrastructure level, it’s typically easy to tune the system. Only if this layer is not enough should additional cache layers be considered. 183 | 184 | 185 | 186 | Once you are at the scale where a well placed and utilized edge cache is enough, you should have a substantial tech team and they can make solid decisions going forward. 187 | 188 | # Coding Standards 189 | 190 | ## I. Build In Security 191 | 192 | Don’t create your own security. Be aware and watchful for violations of the OWASP top 10: 193 | 194 | - [https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project](https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project) 195 | 196 | - Never trust data from client applications - internal and external. All data should be validated/escaped from all sources. If performance is an issue, increase the strictness of the requirements and demand clear client requests. For any web rendering: Use a template library that auto-escapes data values. 197 | 198 | - Never execute raw queries, always use an ORM package that has been heavily tested and validated to execute all queries. 199 | 200 | - Production databases should be secured with passwords, IP locked, hidden behind a VPC, and minimal permissions given for app behavior. 201 | 202 | - Backups of production data should be done daily and tested daily. Backups must be outside of the running production system. A good approach is to backup and load into a staging environment (which also needs to be properly secured). This ensures a hot backup and testing against real data. 203 | 204 | - Use scanning tools like [Sonar](https://www.sonarqube.org/) and [Snyk](https://snyk.io/) to auto-detect some security risks. Sonar is free. 205 | 206 | 207 | ## II. Code Quality 208 | 209 | ### Write Clean Code 210 | 211 | An enormous amount has been written about clean code. It will not be rehashed in full here. We just want to focus on a few key principles: 212 | 213 | 214 | 215 | - Automated Testability 216 | 217 | - Encapsulation of complexity (SOLID, FP principles, etc etc) 218 | 219 | - Code is for people just as much as it is for computers. 220 | 221 | - Program to interfaces not implementations. 222 | 223 | 224 | 225 | 226 | Have engineers read the following articles: 227 | 228 | - [https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-steps-to-better-code/](https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-steps-to-better-code/) 229 | 230 | - [https://blog.cleancoder.com/uncle-bob/2017/03/03/TDD-Harms-Architecture.html](https://blog.cleancoder.com/uncle-bob/2017/03/03/TDD-Harms-Architecture.html) 231 | 232 | - [https://chris.beams.io/posts/git-commit/](https://chris.beams.io/posts/git-commit/) 233 | 234 | - [https://blog.cleancoder.com/uncle-bob/2014/12/17/TheCyclesOfTDD.html](https://blog.cleancoder.com/uncle-bob/2014/12/17/TheCyclesOfTDD.html) 235 | 236 | 237 | 238 | 239 | You don’t have to agree with everything in these articles. The goal is to expand awareness in the space. 240 | 241 | 242 | Key principles to maintain code quality: 243 | 244 | 245 | - Use a code style linter. Every language has a tool or tools for this. Pick popular defaults if you aren’t sure about anything. 246 | 247 | - Use an automatic style formatter if there isn’t a language-standard style (e.g. Golang). Use a set of popular common defaults across all projects in that language. Don’t argue over this, a clear choice that you commit to is better than no choice and wasting time. 248 | 249 | - Automated code coverage percentage for tests should always go up. The goal should generally be 90% code coverage. This is not just “unit test” coverage, but overall code coverage by automated tools. There are ways for devs to cheat this. Make sure you know how to check for this or find someone who can. 250 | 251 | - Integration tests and e2e tests and BDD style tests provide more business value and test more code per line than unit tests. Use those tests as a first choice and use unit tests for tricky logic that needs to have all edge cases carefully covered. NOTE: this is not the case for industries like fintech which have huge penalties/problems with even “minor” misbehavior. In these cases you want extensive coverage at every level possible to ensure maximum robustness and defense against accidental/unexpected behavior. 252 | 253 | 254 | 255 | If you just HAVE to rush it, low-level tests should include at least one “happy path” test and one “error/unhappy path” test, to force the definition of failure not just success. 256 | 257 | ### Use the updated “[Joel Test](https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-steps-to-better-code/)” as a gut check 258 | 259 | How do you score on this list. You want to be 12/12. If you are below 9/12, you probably have an issue. 260 | 261 | 262 | 263 | 1. Do you use source control? (Should be using Git and github, gitlab, etc) 264 | 265 | 2. Can you make a build in one step? (This should be locally and CI/CD) 266 | 267 | 3. Do you make builds at least daily, preferably many times a day? 268 | 269 | 4. Do you have a bug database/tracking? 270 | 271 | 5. Do you fix bugs before writing new code? 272 | 273 | 6. Do you have an up-to-date schedule? 274 | 275 | 7. Do you have a spec? 276 | 277 | 8. Do programmers have quiet working conditions? 278 | 279 | 9. Do you use the best tools money can buy? (Good laptops, Cloud tech) 280 | 281 | 10. Do you have testers? 282 | 283 | 11. Do new candidates write code during their interview? 284 | 285 | 12. Do you do hallway usability testing? 286 | 287 | 288 | ## III. Code reviews 289 | 290 | All code should have at least one peer review before being merged to the master branch. Put some effort into this as a part of culture. Good code reviews improve quality, help onboarding, and lower the overall technical risk of the team and company. 291 | 292 | ## IV. Architecture 293 | 294 | Most typical web applications have a similar web architecture, consisting of a CDN, a set of UI rendering servers (optional), API servers, file storage, a database of some type, and a set of admin/worker/scheduled jobs. 295 | 296 | 297 | 298 | A good primer on system architecture concepts: 299 | 300 | [https://github.com/donnemartin/system-design-primer](https://github.com/donnemartin/system-design-primer) 301 | 302 | 303 | 304 | VideoBlocks produced a relatively generic web architecture diagram that I like: 305 | 306 | [https://engineering.videoblocks.com/web-architecture-101-a3224e126947](https://engineering.videoblocks.com/web-architecture-101-a3224e126947) 307 | 308 | 309 | 310 | This is a good baseline reference of what a production web architecture looks like. If you are building a software as a service application, the architecture should be at least somewhat similar in most cases. 311 | 312 | Last but not least, this is an excellent (but long) article on large-scale web services: 313 | 314 | [https://medium.com/swlh/recipe-to-build-large-scale-web-apps-f38b20ab727f](https://medium.com/swlh/recipe-to-build-large-scale-web-apps-f38b20ab727f) 315 | 316 | ### REST APIs 317 | 318 | By default, use REST APIs. They are well understood and supported and it’s easier to find people who are skilled at rest vs GraphQL vs gRPC. 319 | 320 | 321 | 322 | When architecting features, give a look to the REST Cookbook: 323 | 324 | [http://restcookbook.com/](http://restcookbook.com/) 325 | 326 | 327 | 328 | ## V. Code Deployment and QA Process 329 | 330 | ### Current process 331 | 332 | As code passes from development to production, it will pass the following stages: 333 | 334 | 1. Local Computer - a developer works on a ticket on their machine. 335 | 336 | 2. CI/Dev server - A developer can push to a dev branch to check their code running in the cloud and verify interaction with any google/cloud services. 337 | 338 | 3. Staging - After verification by the developer that their code works on the CI server, they should merge request to master for code review. After code review, the merged code will deploy to staging for manual and AQA. 339 | 340 | 4. Production. By tag on master branch only. 341 | 342 | 343 | 344 | 345 | The code architecture should allow an engineer to run the whole system (or at least enough of the whole system to validate behavior) on their own machine. Developer machines should either be very close to production (e.g. all running linux) or a diversity of OS’s and machine builds to help find tricky code issues through sheer variance. The best choice will depend on your problem domain. 346 | 347 | 348 | 349 | NOTE: This is changing since I wrote this, with cloud-native development options becoming available but I’ve not used them enough to recommend anything at this point. 350 | 351 | 352 | 353 | ### QA Options 354 | 355 | The quality of your software is a combo of how well written the code is and how well tested it is. At scale, QA is always important, and because of the effort involved, we want as much of that QA to be automated as possible. There’s a difference between “feature works” and “feature works well”. If you use humans for QA, they need to be opinionated about what counts as “feature works well” and help police usability. 356 | 357 | 358 | 359 | Testing you want to do: 360 | 361 | - Unit tests (every language has a unit testing library) 362 | 363 | - API tests ([https://github.com/pact-foundation](https://github.com/pact-foundation)) 364 | 365 | - End to End BDD style tests, (cucumber) 366 | 367 | - Manual tests (to catch weird stuff, glitches, bad usability, etc) 368 | 369 | 370 | 371 | 372 | Depending on the nature of the feature/project, some or all of these processes should be involved. It’s important that the team determine how much testing should be unit tests (tricky logic with lots of edge cases is a good example) and how much should be API tests or BDD tests. As code becomes more stable, there should be more and more of all of the above to ensure that updates don’t break older functionality. 373 | 374 | ## V. What is “Done”? 375 | 376 | First, understand the difference between the “definition of done” and “acceptance criteria”. Acceptance criteria are specific to a task/feature/ticket/etc. The definition of done are a set of standards that apply by default to ALL work regardless of what that work is for. 377 | 378 | 379 | 380 | Here’s my default: 381 | 382 | - Test code coverage must be at least as high as before (preferably higher). By default, the target should be 90% across branches/lines/etc. 383 | 384 | - CI/CD build passes all unit, integration, API, and performance checks and deploys properly without issues. 385 | 386 | - Static analysis checks do not flag any substantive problems. 387 | 388 | - The code has been peer reviewed. 389 | 390 | - Acceptance criteria in the ticket is met and validated by Manual QA. 391 | 392 | - The code obeys 12 factor app principles (for any web related code), or other architectural principles that the team agrees are more appropriate. 393 | 394 | 395 | # Git Branching and Merging 396 | 397 | Don’t overcomplicate this. Gitflow is popular but it’s author specifically says he did not have modern webapps in mind when he wrote it. Don’t overcomplicate. He recommends github flow. 398 | 399 | 400 | 401 | [https://guides.github.com/introduction/flow/](https://guides.github.com/introduction/flow/) 402 | 403 | 404 | 405 | I don’t like it, some people love it. I prefer trunk based development, but you need the right team to make that work. My suggestion is to hire the right team, but if you can’t or don’t know how, use the cludge that is gitflow. 406 | 407 | 408 | 409 | # Team Performance Metrics 410 | 411 | Measurement of performance is at the TEAM level. All for one, one for all. Because software is a team sport and so are startups, we either all succeed together or not at all. 412 | 413 | 414 | 415 | There are 4 metrics proven to be oriented with team success: 416 | 417 | 418 | 419 | - Cycle Time - How long it takes to put a change into production (see “What is Done?”) once code is completed for a feature (completely defined as submitted for manual QA). Our target is < 4hrs. Meaning that once code is ready for QA, it will be evaluated by a tester, tested for load and other functional and non-functional requirements, and deployed into production in half a day. Ideal is under 2 hours. 420 | 421 | 422 | - Lead Time - how long it takes to go from business idea to delivered software. Lead time can itself be divided into two: Requirements definition time and Requirements -> Feature time. 423 | 424 | The development team is primarily responsible for the second aspect of lead time. Once we get acceptable requirements, it’s up to us to find the fastest way to deliver working software. There is no target, we just want to continually improve this number. 425 | 426 | 427 | - QA fail and Open rates - How many issues fail final QA and how many defects are reported by production. Again, no firm target, but the trend should be downward. 428 | 429 | 430 | - Velocity - How many units of functionality we produce per unit time. This is not an objective measure but ties into the benefit perceived by the business (business velocity) and the complexity of the technical implementation (technical velocity). Maximum velocity comes from maximizing business impact with as little technical effort as possible. 431 | 432 | 433 | 434 | 435 | # Tech Org Metrics 436 | 437 | In addition to the dev team metrics above, [there 4 metrics](https://www.leanix.net/en/wiki/vsm/dora-metrics) associated with overall technical organization success. 438 | 439 | - Deployment frequency (how often you put code into production) 440 | 441 | - Lead time (just like above) 442 | 443 | - Mean time to restore failure (if things break in production, how long to restore your system) 444 | 445 | - Change failure rate (how often does code need to be rolled back) 446 | 447 | 448 | If you are hiring a manager/executive in charge of technology, ask about these metrics. If they don’t know what you are talking about, don’t hire them. 449 | -------------------------------------------------------------------------------- /SolutionDesign/README.md: -------------------------------------------------------------------------------- 1 | # Solution Design 2 | When companies build a new system, they often "dive in" and just start building. 3 | It's actually MUCH more efficient to spend a little time doing some up-front design work. 4 | However, when teams do upfront design, they rarely consider all the aspects they should. 5 | 6 | This document is meant to be a helpful checklist of things to consider and document before you "dive in" to implementation. 7 | If you do this, you'll almost certainly save a lot of time in the end. 8 | 9 | **The following template should be used when planning new systems or large upgrades.** 10 | 11 | # Design Writeup Template 12 | ## 1 System Goals 13 | What are you trying to achieve? How do you know you are successful 14 | ## 2 How do we measure customer success 15 | What are the people using this system/API trying to achieve? Have you considered their needs? How can we be sure our solution is easy for them to use and effective before we spend time building it? 16 | ## 3 Estimated system ROI 17 | Building things costs money and time. Where do we get our ROI? 18 | - Improved TTM? How much? 19 | - Can we sell this? 20 | - Improved customer retention? 21 | - Internal ease of use/efficiency gains? 22 | 23 | The point is to document the intended value creation and expected costs *up front*. 24 | ## 4 Solutions considered 25 | Don't consider just one solution. Consider a few and look at the ROI. Sometimes cheap, fast, and dirty is great. Sometimes it's a horrible idea. 26 | 27 | Choose carefully. 28 | ## 5 Solution Selected + Rationale 29 | Document why you chose the solution. This is not to CYA it's for retros and to force honesty. 30 | 31 | ## 6 C4 architecture + other helpful diagrams 32 | Use the C4 architecture to document your overall solution architecture. 33 | You can add additional diagrams if you really feel it helps. 34 | 35 | I also suggest sequence diagrams as well (see item 9) 36 | 37 | ## 7 Public API or Behavior 38 | If this is a backend system, link out to a proposed API *before building*. 39 | Is this easy to use? What about authorization (see below) 40 | 41 | ## 8 Authentication and authorization, rate limiting, system protection 42 | How is this system secured? 43 | How will you protect against abuse? 44 | 45 | Don't write "we won't". 46 | 47 | ## 9 Key Sequence Diagrams 48 | For key flows, create some sequence diagrams. These are helpful for development, QA, debugging, etc. 49 | 50 | ## 10 Data models 51 | Larger companies often suffer from an explosion of similar but not the same data models. 52 | For interoperability's sake, don't re-invent the wheel if possible. 53 | 54 | #### a. New Data models 55 | If you have a new data model, what is it. 56 | 57 | #### b. Re-used data models 58 | What other data models are you using from across the company? 59 | Were there competing standards? Better to call it out here. 60 | 61 | #### c. ISO standards utilized, considered, rejected 62 | Did you even look to see if there's a documented standard? Bet you didn't. 63 | 64 | Go check. Revise the above as needed. Please use the standards for everyone's sanity unless you have a *super good reason* not to. 65 | Lazyness is not a super good reason. 66 | 67 | ## 11 How to verify correct behavior? 68 | Again, think about how you'll test your system *as a whole* to verify correctness. 69 | Nobody likes 3am calls because something blew up. Shift left, starting now. 70 | 71 | ## 12 List of Key external systems and dependencies 72 | External means everything that isn't this system, inside your company or not. 73 | ### Strategy to minimize and encapsulate. 74 | Every dependency is a potential bug when they upgrade, change something, go down, etc. 75 | Have a plan for how you'll deal with external dependencies and services going haywire. 76 | 77 | Make reviewing that plan part of code review. 78 | 79 | ## 13 Key consequences 80 | Every system has consequences. Don't lie to yourself. 81 | ### a. Vendor or system lock in? 82 | ### b. Technology specific behavior 83 | ### c. Unique or rare skillsets needed for development or support 84 | ### d. Regulations? Others? 85 | ## 14 Anticipated Evolution considered 86 | The only constant is change. If your system works, do you expect a lot of scale? 87 | I've only listed scale here but you could consider ongoing regulations (privacy, financial, etc). 88 | A little thinking here can go a long long way. 89 | ### Scale 90 | #### Plan for 10x current load 91 | #### Plan for 100x current load 92 | 93 | ## 15 Service lifecycle management 94 | The only constant is change. You will upgrade, deprecate, alter, or have others depend on your service. 95 | How will you handle this thing AFTER it gets deployed. Thinking this through NOW makes everyone's life easier LATER. 96 | ### a. plan for versioning? 97 | ### b. How will we flag service upgrades, deprecation, etc, to end users? 98 | ### c. Intended SLO/SLA for service 99 | ### d. Intended behavior outside of SLO 100 | ### e. Replacement lead time 101 | ### f. Plan for operations and monitoring 102 | ### g. Key contacts --------------------------------------------------------------------------------